This removes callback from internal APIs: `_list_paths`, `list_hashes`,
and `_hashes_with_limit`. These are all iterators and hence can just be
replaced by wrapping a progressbar over those iterators in the caller
side.
We still have progressbars in `hashes_exist`, `_estimate_remote_size`,
`_list_hashes_traverse` and `list_hashes_exists`, which should gradually
lifted up and replaced.
fix: #7155
1. rename flags `-A/--all-commits` in `exp remove`
2. add new flag `-n/--num` in `exp remove`
3. unify the revision collection in `exp remove`
4. add unit and func tests for `exp remove`
Co-authored-by: Jorge Orpinel <[email protected]>
fix: #7154
1. rename flags `-A/--all-commits` in the `exp pull/push`
2. add new flag "--rev" and "--num" in the `exp pull/push`
3. Unify the collection of revs in `exp push/pull`
4. add unit and func tests for `exp push/pull`
fix: #7153
Unify the flags in `exp list` to `exp show`
1. Add new `-A`, `-n` flags to `exp list` command.
2. Add unit test for the flag parse.
3. unify the revs retrive logic to `exp show`.
4. Add functional test for `exp list`
5. Update the documents.
Co-authored-by: Jorge Orpinel <[email protected]>
Currently we use both absolute and relative (depend on cwd) paths, which is
terrible for an api, as their interpretation depends on external factors.
This PR makes dvcfs accept only paths relative to the repo root for regular
outputs and absolute paths for external outputs.
Pre-requisite for making repofs use same relative paths.
Introducing simple repo tree that combines all workspaces into one structure,
and allows us to detach data representation from pipelines.
Pre-requisite for dvc-data extraction and using relpaths in dvcfs/repofs.
metadata was introduced before our migration to fsspec and no longer needed
now. This is a pre-requisite to using data index in dvcfs/repofs.
Note that a bunch of awkward flags like isdvc/isout/etc are still kept, and
will be removed in followups.
With our staging process automatically capturing file metadata, we no longer
need to set isexec ourselves.
In followup PRs we will also use meta during checkout, so that we won't need
to use Output.set_exec by hand.
Removed `--include-metrics` / `--include-params` and
`--exclude-metrics` / `--exclude-params`.
Removed `--no-timestamp` . Can be done by `--drop Created`.
`--drop` and `--keep` operate directly on the table columns.
`--keep` does not perform any filtering. It's only used
to specify columns to keep despite the other filtering.
For example `--only-changed --keep foo` will prevent
`foo` from being removed by `--only-changed`.
Another example, `--drop train.* --keep train.dropout`
will remove all columns matching `train.*` except for
`train.dropout`.
Closes#7079Closes#7080
Pre-requisite for #6434
By splitting dvc.cli into a module, we'll be able to share cli-related
utilities, split parser-related stuffs into smaller parts, improve
reusability and improve API for command extensions in the future.
fix: #7196
New feature: now `dvc exp pull` can accept a list of experiment names as input.
2. add support to list input for the `exp pull`
3. add function tests for the multi exp pull
4. add unit test for the arg parse
fix: #7196
New feature: now `dvc exp push` can accept a list of experiment names as input.
1. refactor of experiment push
2. add support to list input for the `exp push`
3. add function tests for the multi exp push
4. add unit test for the arg parse
fix: #7152
1. Add `--rev` flag to `dvc exp show`.
2. Add -1 support for `--num` flag`.
3. Extract revision found logic to `utils`.
4. Brancher use this revision found logic.
5. Update command unit test
6. Add a util unit test.
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Initial simple separation to get the process of extracting dvc-objects
and dvc-data packages out of dvc.
dvc-objects is for:
* defining base Object (aka HashFile) and ObjectDB
* providing tools to work with those objects (check/transfer/etc)
* etc
dvc-data is for:
* defining higher level objects (e.g. Tree)
* bulding objects from the workspace (aka staging)
* projecting objects back into the workspace (aka checkout)
* comparing Tree objects (aka diff)
* defining misc object databases (e.g LocalObjectDB)
* higher-level transfer (e.g. with Tree indexes)
* etc
For example, the objects we use in run-cache are based on dvc-objects
and not on dvc-data, as they have absolutely nothing to do with data
management.
There are still some mixups in the code that will be addressed in the
followups.
* test: fix test failure in case of different default branch
When the git config has set global `init.defaultBranch` different than master,
a few test cases were failing.
Currently, the test case is assuming the default branch to be set as
master.
This commit will introduce **initial_branch** option to git init call
which will ensure *master* branch is used as default for out test cases.
Set default git branch to main for test CI
Since main is going to be new default branch, it will let the test case
handle the scenario where default branch is set other than master.
Introduce isolation for git configuration
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
fix: #7196
1. Refactor exp_ref_by_name in `dvc/repo/experiments/utils.py`.
In previous the exp_ref_by_name walk throught all of the references to
match only one exp name. This makes multi exp name seeking to be a job
of `O(mn)` complexity, Walking though the exp ref and return all of the
exp ref with greatly reduce this complexity to `O(n)`
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Ruslan Kuprieiev
Nick SorrosandPaweł Redzyński
David de la Iglesia Castro
Saugat Pachhai (सौगात)
karajan1001andJorge Orpinel
Peter Rowlands
dberenbaum
Daniele Trifirò
Nikita Kodenko
pre-commit-ci[bot]
Jon Burdo
Amrit GhimireandSaugat Pachhai