* Fixes#4637
* use PathInfo instead of Path
* catch ValueError with PathInfo.relative_to and fallback to dvc.utils.relpath
* cleanup
* add test
Co-authored-by: Peter Rowlands <[email protected]>
* repo: make Repo.scm a cached attribute
* checkpoints: create final checkpoint after repro when needed
* use correct exception rev in _collect_and_commit
* tests/remote: adding tests for real oss
the new test suggests #4633 is a bug in the emulator.
* Update tests/func/test_data_cloud.py
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Update tests/remotes/oss.py
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* tests: correcting small lint issue
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* command: print the current cache directory when no cmd flags are provided
tests: corrected the test for cache dir cmomand
comamnd 'dvc cache dir' now prints the current cache dir
when no cmd flags are provided and modified existing test for
the same.
Fixes#1998
Signed-off-by: Girish Joshi <[email protected]>
* command: updated validation for args.value and args.unset flag
according to current convention in dvc/command/cache.py
Signed-off-by: Girish Joshi <[email protected]>
* Update dvc/command/cache.py
* dont forget logger
Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
When executing `dvc import-url remote://some-webdav/data`, the name of
the WebDAV dependency in the created stage for data used to be `None`.
This resulted in the following error:
```
Traceback (most recent call last):
File "/home/schubert/projects/dvc/dvc/main.py", line 76, in main
ret = cmd.run()
File "/home/schubert/projects/dvc/dvc/command/imp_url.py", line 14, in run
self.repo.imp_url(
File "/home/schubert/projects/dvc/dvc/repo/__init__.py", line 51, in wrapper
return f(repo, *args, **kwargs)
File "/home/schubert/projects/dvc/dvc/repo/scm_context.py", line 4, in run
result = method(repo, *args, **kw)
File "/home/schubert/projects/dvc/dvc/repo/imp_url.py", line 54, in imp_url
stage.run()
File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/site-packages/funcy/decorators.py", line 39, in wrapper
return deco(call, *dargs, **dkwargs)
File "/home/schubert/projects/dvc/dvc/stage/decorators.py", line 36, in rwlocked
return call()
File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/site-packages/funcy/decorators.py", line 60, in __call__
return self._func(*self._args, **self._kwargs)
File "/home/schubert/projects/dvc/dvc/stage/__init__.py", line 450, in run
self.save()
File "/home/schubert/projects/dvc/dvc/stage/__init__.py", line 395, in save
self.md5 = self.compute_md5()
File "/home/schubert/projects/dvc/dvc/stage/__init__.py", line 388, in compute_md5
m = compute_md5(self)
File "/home/schubert/projects/dvc/dvc/stage/utils.py", line 167, in compute_md5
return dict_md5(
File "/home/schubert/projects/dvc/dvc/utils/__init__.py", line 118, in dict_md5
return dict_hash(d, "md5", **kwargs)
File "/home/schubert/projects/dvc/dvc/utils/__init__.py", line 113, in dict_hash
byts = json.dumps(filtered, sort_keys=True).encode("utf-8")
File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
TypeError: '<' not supported between instances of 'str' and 'NoneType'
```
* state: add simple reentrance
* state: pass Repo instead of LocalCache
* state: use granularly
Users could still load state preemptively if they want to bulk up
the operations, but other than that tree/cache will just load it
when they need it.
* stage: make runs checkpointable
* api: add make_checkpoint API call
* tests: add test for checkpointed stage
* experiments: implement checkpoints callback chain
* initial working state
* use .dvc/tmp for checkpoint signal file location
* force repro for checkpoint experiments
* checkout: add flag to allow missing persistent outputs
(i.e. not yet created)
* use allow_persist_missing on checkpoint experiment repro
* executor: force checkout before checkpoints experiment repro
* move checkpoint commands to `dvc exp run`
* support resuming checkpoint runs with --continue
* fix git bugs
* experiments: include checkpoint commits in `exp show`
* use 1sec sleep timers
* fix apply_workspace test conflict
* tests: add test for `dvc exp run`
* cleanup signal file behavior
* fix bool conversion
* tests: add tests for checkpoint and checkpoint_continue
* fix styling
* fix checkpoint stage test
* fix checkpoint monitor thread condition/notification
* repro: support --pull
This option allows run-cache to pull the cache for the outputs
when restoring a stage instead of pulling everything at once during
`dvc pull --run-cache`.
Fixes#4223
* repro: adjust --pull help
* yaml: wrap ruamel's DuplicateKeyError
Unfortunately ruamel's exception doesn't contain much info that we could
reuse, so we can only reasonably extract the exc.problem and have
something like:
```
ERROR: failed to obtain data status - unable to read: 'dvc.yaml', found
duplicate key "metrics" with value "[]" (original value: "[]")
```
Fixes#4664
* status: remove unneeded error prefix
Peter Rowlands (변기호)
Saugat Pachhai
Harsh Patel
Alexei Tenitski
Musa AtlıhanandPeter Rowlands
Juliano Efson SalesandRuslan Kuprieiev
Jonny Browningandbrowningjp
Paweł Redzyński
Jorge OrpinelandRuslan Kuprieiev
omar.zhouandzwh
Frederik Schubert