* tests: additional callbacks test for remaining filesystems
* requirements: bump pydrive2 to 1.9.4
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* machine: add `machine list`&`machine modify`
1. add new command `dvc machine list`
2. complete `dvc machine modify`
3. add new unit tests for this two commands to ensure the call.
4. add functionality tests for two commands
* Update dvc/config_schema.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Update tests/func/machine/test_machine_config.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Some reviewed problems
1. capitalize global variables.
2. remove precision restriction.
3. solve windows path seperator problem.
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* dvc cli: fixed error in 'dvc config --list' in non repo dir
Fixes#6496
* Issue 6496: Added test that covers the issue
* dvc cli: fixes error in 'dvc config <name>' in non dvc repo dir
Fixes the specified error. Did some refactoring based on previous commits.
Also added a test case for setting config in non dvc repo dir.
Fixes#6496
* Issue-6496: dvc config --<level> and dvc config --list --<level> will now throw config error if ran in non dvc repo with repo only level
* Issue 6496: Segregated config levels into repo and system levels that helped in simplifying the logic to decide appropriate level at the time of cli operation.
The current ci failure was caused by wrong order of the experiments.
Because our test script runs so fast that they might share the same
commit timestamp, that makes them in an uncertain order. And then gives
a wrong branch_base.
* output: use remote kwarg instead of odb
Fixes#6596
* Update dvc/output.py
Co-authored-by: Dave Berenbaum <[email protected]>
* add test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: Dave Berenbaum <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fs.download: add callback support
* fix lint issues
* compat for gdrive, use fs to upload text for tests
* fix typing issues
* skip gdrive test
* Update tests/func/test_fs.py
* fs.upload: add support for optional callback
Also removes _upload methods in other fses, and replaces them
with fsspec-compatible put_file which supports fsspec callback
even if they are not fsspec based filesystem.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Show csv format of experiments
fix#5989 and #5446
1. add --show-csv to dvc exp show
2. add a new functional test for csv format
3. add a new unit test for show_experiements
4. solve name duplication in table common headers (Created,Parent, etc... ), metrics files and param files.
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Making sure that script takes a bit more time than the await time
I am not really sure this was the cause though.
Also the tests are flaky only in MacOS, but I haven't been able to reproduce.
* Clean up remotes's exps
fix#6006
1. add a new argument `--git-remote` to `dvc exp remove`
2. support remote a special remote exp
3. add tests for it
4. fix #6421
5. add a test for #6421.
6. extract some functions from the `dvc pull`, `dvc push`, `dvc remove` to utils.
7. add a tests for this new util function `resolve_exp_ref`
8. add `__init__.py` to some test package for the pytest fail
Co-authored-by: Jorge Orpinel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dave Berenbaum <[email protected]>
* Added refresh tag
* Formatted
* Added refresh_seconds to live
* Added refresh test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add refresh_seconds to render
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update html.py
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Remove all of the experiments in the workspace
fix#5676
1. remove all the experiments in workspace
2. add tests for it
* Only remove baseline ones
* Add a new arguments --all
* Update dvc/repo/experiments/remove.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Rename and shortcut for these commands
* Solve the issue from removing a changing list.
* Update dvc/repo/experiments/remove.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Update dvc/repo/experiments/remove.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* delete workspace option
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Replacing DVC_ANALYTICS. This is following our other env conventions and
will not have effect if set to false/no/etc, and will only work if set
to true/yes/y/etc. Will disable analytics even if it is explicitly
enabled in our configs.
Kudos @skshetry
* add support for Python 3.10
pyarrow is not available, so we are skipping tests for it.
And, restricting `hdfs` extras to be not installed in Python3.10.
Pathlib removed `_init` method and `init` parameter, so just working
around that. It's not a big change.
* skip windows on 3.10; update pip/wheel/setuptools during install
pywin32 is not yet available for 3.10 Python, so skipping those for now
* install libgit2 with Python3.10 as pygit2 wheel is not available
* install libgit2
* only run on macOS, libgit2-1.1 not available on ubuntu
* Remove a special queued experiments
fix#6157
1. dvc exp remove not accept queued experiments name
2. add some tests for this feature
* Extract tests and add revision support
1. Extract remove experiments to a new file.
2. revision can be used to remove special queued experiment
* Accept shortened revisions
* Split removing committed and queued exp functions
* Api name change
* Return to the old API name
* shorten some functions
* Error message change
* Better test cases, more corner case
* Still raise exception in a mixed case
* Push running checkpoint to remote
Fix#6182
* Update dvc/repo/experiments/executor/base.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Get the branch name
* Finish this PR
1. move env name to dvc/env.py.
2. add some tests for it.
* Update after finally commit
* Some problems found in the review
1. change the behavior of self remote
2. do not use string DVC_EXP_AUTO_PUSH
3. downgrade the logger level in auto push
4. use full branch ref
* Update dvc/repo/experiments/executor/base.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* add a remote check before auto push
* Split current one env into two
* Rename DVC_EXP_CHECKPOINT_PUSH to DVC_EXP_AUTO_PUSH
* value error
* Remove getenv_bool switch to env2bool
* Better on handling Dulwich exceptions
* Name changing removing prints
* Update dvc/env.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Some changes in review
* Add validation to repo URL
* Add new type of SCMError
* Update dvc/repo/experiments/executor/base.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Use spy tests call counts and args
* Update dvc/scm/git/backend/base.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Update dvc/scm/git/backend/dulwich.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove validate_git_repo to validate_git_remote
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* demonstrate issue while sending analytics report
When sending analytics, we are not flushing the data to the temp file
which causes failure at times, especially on Windows
* close file before calling daemonized analytics send command
This way, the contents bufferred will get a chance to be flushed.
Saugat PachhaiandRuslan Kuprieiev
Nikita Kodenko
David de la Iglesia Castro
Batuhan Taskaya
GaoandPeter Rowlands
Amey Kulkarni
Casper da Costa-LuisandPeter Rowlands
Paweł Redzyński