Initial simple separation to get the process of extracting dvc-objects
and dvc-data packages out of dvc.
dvc-objects is for:
* defining base Object (aka HashFile) and ObjectDB
* providing tools to work with those objects (check/transfer/etc)
* etc
dvc-data is for:
* defining higher level objects (e.g. Tree)
* bulding objects from the workspace (aka staging)
* projecting objects back into the workspace (aka checkout)
* comparing Tree objects (aka diff)
* defining misc object databases (e.g LocalObjectDB)
* higher-level transfer (e.g. with Tree indexes)
* etc
For example, the objects we use in run-cache are based on dvc-objects
and not on dvc-data, as they have absolutely nothing to do with data
management.
There are still some mixups in the code that will be addressed in the
followups.
* test: fix test failure in case of different default branch
When the git config has set global `init.defaultBranch` different than master,
a few test cases were failing.
Currently, the test case is assuming the default branch to be set as
master.
This commit will introduce **initial_branch** option to git init call
which will ensure *master* branch is used as default for out test cases.
Set default git branch to main for test CI
Since main is going to be new default branch, it will let the test case
handle the scenario where default branch is set other than master.
Introduce isolation for git configuration
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
fix: #7196
1. Refactor exp_ref_by_name in `dvc/repo/experiments/utils.py`.
In previous the exp_ref_by_name walk throught all of the references to
match only one exp name. This makes multi exp name seeking to be a job
of `O(mn)` complexity, Walking though the exp ref and return all of the
exp ref with greatly reduce this complexity to `O(n)`
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
`@memoize()` memory lives as long as decorated function lives, which is
permanent in this case. Since `self` is referenced as a memory key no
`Index` will ever be freed.
Changed to use `@cached_property` whose cache lives as long as the
parent instance.
SCM is used for both reading and writing operations, but in the former case -
as filesystem and in the latter - as actual scm/git. So we should distinguish
those and use fs directly, which follows the overall philosophy of being able
to use Repo on any fs.
Keeping `rev=` option for `Repo` because it is a very convenient way to get
repo instance from local path but for particular branch/tag/etc.
Pre-requisite for git/dvc/repofs refactorings.
When filling missing values with `ui.rich_text` (i.e. in experiments show CMD), those values were not being correctly matched against `self._fill_value`)
Adds new argument `fill_value` to ParallelCoordinatesRenderer.
For scalar columns, prevent converting to categorical by explictly inserting None when value is missing (value == fill_value).
For categorical columns, replace `fill_value` with `"Missing"` and ensure it is the last item so it gets rendered on top of the column.
related to #5489
Current dvc/command/experiments.py is too long for switch between cmd class
and the argument parser.
Split the sub command in dvc/command/experiments.py to individual files
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
We didn't mock `os.kill` in `test_dead_process` and the pid it
kills might coincidentlly be the same with some real process.
1. Add a mocker to mock out `os.kill` in `test_dead_process`.
2. Use reset_mock to replace setting it again.
With introduction of https://github.com/iterative/dvc-plugins,
all important tests are being ran there, so there is no need
for dvc repo to run them again.
This PR also removes some redundant/broken remote tests.
fix: #7062
1. Add `send_signal`, `kill`, `terminate` to processManager.
2. Add tests for each of them.
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Borrowed this idea from pandas api. Current dropna methods will drop a
column or row if any data is missing. But it is also quite useful to
drop those columns or rows only if all of its elements are missing. This
is mostly used for pruning a table.
- allow mismatched RSA signature algo names in initial
SSH_MSG_USERAUTH_PK_OK response
- unit test overridden asyncssh publickey auth compatibility with
github ssh server
fix: #6940
`--global/system/project/local` help message are in some irrelevant
command `status`, `ssh`, `create`, `destroy`. These commands are more
related to running instances rather than `machine config`, level
options didn't work for them.
1. Remove `--global/system/project/local` options in `status`, `ssh`,
`create`, `destroy`.
2. Add a test for it.
3. Correct a typo error.
fix: #6838
1. make machine status name optional.
2. show all machine status if no name provide.
3. UI changes for this cmd.
4. add a new test for it.
Co-authored-by: Dave Berenbaum <[email protected]>
