Commit Graph
5942 Commits
Author SHA1 Message Date
Peter Rowlands (변기호) 87f1c42acf experiments: wrap column entries with rich.Text (#4771) 2020-10-22 16:40:08 +09:00
Peter Rowlands (변기호) 8a5dc6a781 init: move welcome message from Repo.init to command.init (#4759)
* repo: add quiet flag for Repo.init

* move welcome message to command/init
2020-10-22 12:49:48 +09:00
Paweł Redzyński 33ece011fb TestCmdCacheDir: roll back darwin skip (#4673) 2020-10-21 23:36:16 +03:00
Peter Rowlands (변기호) 79232dfedf checkpoints: --continue from last run experiment by default (#4758)
* checkpoints: `--continue` with no args resumes most recent experiment

* add test for --continue :last
2020-10-21 17:25:16 +09:00
Saugat Pachhai aa7b4c6340 Add tests for resolving stage data (#4753)
* Add test, and fix resolving of paths for params

Improve merging of vars and imported params, improve tracking
Resolve wdir correctly

* Fix windows relpath

* isort imports
2020-10-21 05:47:48 +05:45
Saugat Pachhai 668002cb01 Revert "Track flaky tests using Google's service (#4749)" (#4755)
This reverts commit 7527a220f6.
2020-10-20 08:19:00 -07:00
Peter Rowlands (변기호) 25efd0efe8 checkpoints: allow missing outputs on checkout and stage.run (#4750) 2020-10-20 21:44:01 +09:00
Saugat Pachhai b0ae5e2251 Support parametrization from params or vars section in dvc.yaml (#4751)
* Implement importing from params

* Change varname

* Add tests

* fix pylint issues
2020-10-20 04:16:13 -07:00
Ruslan Kuprieiev e48055077d plots: don't allow in non-dvc repo (#4745) 2020-10-20 11:31:57 +03:00
Saugat Pachhai 7527a220f6 Track flaky tests using Google's service (#4749)
* Track flaky tests using Google's service

https://flaky-dashboard.web.app
More info: https://github.com/GoogleCloudPlatform/flaky-service

* Increment version
2020-10-19 23:24:11 -07:00
Peter Rowlands (변기호) feaa5cfcb2 experiments: fix checkpoints sorting issues in dvc exp show (#4740)
* experiments: order experiment (branches) by timestamp

* checkpoints: sort checkpoint branches by tip commit values

* remove sort logic from dvc.scm
2020-10-20 14:54:43 +09:00
Saugat Pachhai 10ae13132d Use lxml wheel instead of legacy setup.py builds (#4748)
`lxml` now provides wheel for Python3.9. 

Also, enable windows test for Python3.9
2020-10-19 22:23:51 -07:00
Harsh Patel 924323f54f lock: added troubleshooting doc link in error message (#4325) (#4721)
* lock: added troubleshooting doc link in error message (#4325)

* updated lock error message (iterative#4325)
2020-10-19 20:47:08 +03:00
Alexei Tenitski 235d4c9a94 #1618 Custom message when cache and remote are in sync (#4694)
* #1618 Custom message when cache and remote are in sync

* Output the correct remote name during dvc status

* Simplify logic
2020-10-19 20:38:05 +03:00
Musa AtlıhanandPeter Rowlands 4b6ebc7527 make pull --run-cache work with local remote having relative path (#4670)
* Fixes #4637

* use PathInfo instead of Path

* catch ValueError with PathInfo.relative_to and fallback to dvc.utils.relpath

* cleanup

* add test

Co-authored-by: Peter Rowlands <[email protected]>
2020-10-19 17:53:01 +09:00
Peter Rowlands (변기호) 699602f4d0 experiments: use general format for floating pt numbers (#4730) 2020-10-19 15:26:29 +09:00
Peter Rowlands (변기호) 10a2eeb2b8 checkpoints: create final checkpoint commit after repro when needed (#4715)
* repo: make Repo.scm a cached attribute

* checkpoints: create final checkpoint after repro when needed

* use correct exception rev in _collect_and_commit
2020-10-19 15:26:06 +09:00
Peter Rowlands (변기호) b9f547637c checkpoints: add --reset option (#4731)
* experiments: add --reset option to reset/restart checkpoint experiments

* add CheckpointExistsError internal exception

* update tests
2020-10-19 15:25:08 +09:00
Juliano Efson SalesandRuslan Kuprieiev 052b15a2a8 tests/remote: adding tests for real oss (#4735)
* tests/remote: adding tests for real oss

the new test suggests #4633 is a bug in the emulator.

* Update tests/func/test_data_cloud.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update tests/remotes/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* tests: correcting small lint issue

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-10-18 16:21:51 +03:00
Jonny Browningandbrowningjp 68220cbea0 Added 'doctor' as an alias for 'version' (#4722)
* Added 'doctor' as an alias for 'version'

* simplified doctor alias

Co-authored-by: browningjp <[email protected]>
2020-10-17 11:10:03 +03:00
Ruslan Kuprieiev 21a6be7c58 Delete .restyled.yaml
Since we've decided to stop using it. Also disabled in the repo settings.
2020-10-15 15:07:58 +03:00
Paweł Redzyński 2b90d7dfaf tests, packages: roll back environmental packages setup (#4728) 2020-10-15 14:19:25 +03:00
Peter Rowlands (변기호) 9d7e483599 experiments: fix git stash mtime race condition (#4727) 2020-10-15 13:26:17 +03:00
Paweł Redzyński 8f8c561d1b test_api: roll back xfail (#4660) 2020-10-15 10:40:53 +02:00
Ruslan Kuprieiev 32cef509b7 run-cache: don't save always_changed stages (#4719)
Fixes #4335
2020-10-14 17:43:16 +03:00
Saugat Pachhai 6a79111047 Parametrization support on dvc.yaml (#4679)
* Custom parametrization support

* log generated data

* Add tests
2020-10-14 05:32:05 -07:00
Peter Rowlands (변기호) 09e51e655d checkpoints: cleanup --continue usage (#4716)
* warn if rev provided with --continue is ambiguous
* always provide tip of branch when suggesting to use --continue
2020-10-14 19:12:57 +09:00
Peter Rowlands (변기호) 4aa8fd0fbb tests: mark test_extend_branch as flaky (#4718) 2020-10-14 19:12:32 +09:00
Peter Rowlands (변기호) 3cebd98c54 tests: fix test_remote::test_push_order (#4714)
* tests: fix test_remote::test_push_order

* review fixes
2020-10-14 18:40:04 +09:00
Ruslan Kuprieiev b11fe83828 dvc: bump to 1.8.4 1.8.4 2020-10-13 23:57:29 +03:00
Peter Rowlands (변기호) cc8d0ef43f experiments: support exp run --checkpoint for baseline commits (#4701)
* handle untracked files when `git reset --hard` in experiments workspace

* error on checkpoint branch already exists

* terminate stage process on checkpoint errors

* update tests

* cleanup error message
2020-10-13 23:10:08 +03:00
Paweł Redzyński 3e011d70d5 github actions: windows: fix packaging (#4711) 2020-10-13 21:31:48 +02:00
Ruslan Kuprieiev 0649242578 dvc: bump to 1.8.3 1.8.3 2020-10-13 13:17:22 +03:00
Paweł Redzyński 549f342db9 gha: fix github token providing for assets upload (#4703) 2020-10-12 18:18:20 +03:00
Saugat Pachhai 4e53491f5c Use released version of Python3.9 on CI (#4704) 2020-10-12 20:24:28 +05:45
Paweł Redzyński 0da1664f3f gha: get rid of unnecessary if (#4702) 2020-10-12 13:33:15 +03:00
Jorge OrpinelandRuslan Kuprieiev 3b6a1c441e term: "metrics" plural in output messages (#4699)
* api: update to latest definitions to match iterative/dvc.org/pull/908

* Update dvc/api.py

* metrics show: update -h output to match docs
per https://github.com/iterative/dvc.org/pull/1014/commits/2c3452119545987f77b764e4aa6fc94d051092a7

* api: update docstrings to match iterative/dvc.org/pull/908

* api: refactor DVC repo check in get_url, and document it

* dvc: cosmetic edits as I explored exceptions that api functions may raise

* api: copy default info to read() docstring from open()
per https://github.com/iterative/dvc/pull/3426#pullrequestreview-366949501

* api: improve open() docstring for clarity and add example
per https://github.com/iterative/dvc/pull/3426#pullrequestreview-366956856

* api: remove unnecessary info from get_url docstring
per https://github.com/iterative/dvc/pull/3426#discussion_r386203064

* api: produces->generated in open() docstring

* api: simplify open docstring
per https://github.com/iterative/dvc/pull/3426#discussion_r388218147

* term: "metrics" plural in output messages
https://github.com/iterative/dvc.org/pull/1848#pullrequestreview-506217071

* typo

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-10-12 12:51:57 +03:00
Ruslan Kuprieiev ca490f699e ls: pass repo_tree instead of repo (#4700) 2020-10-12 12:37:19 +03:00
omar.zhouandzwh 59105ec639 Fixed the bug #4251, and fixed a minor bug which causes of OS default character. (#4646)
* bug fixed
bug description:
1, environment: OS which default character is not 'UTF-8'. For example windows 10 which default character is 'gbk'.
2, operation:
python setup.py install
Traceback (most recent call last):
  File "setup.py", line 149, in <module>
    long_description=open("README.rst", "r").read(),
UnicodeDecodeError: 'gbk' codec can't decode byte 0xa2 in position 41: illegal multibyte sequence

* bug #4251 fixed
https://github.com/iterative/dvc/issues/4251

* bug #4251 fixed
https://github.com/iterative/dvc/issues/4251

* Revert "bug #4251 fixed https://github.com/iterative/dvc/issues/4251"

This reverts commit f524c727

* Revert "bug #4251 fixed https://github.com/iterative/dvc/issues/4251"

This reverts commit fa5db477

* Fix bug #4251
We can check if the source exists by using `dep.exists`

Co-authored-by: zwh <[email protected]>
2020-10-12 09:00:16 +03:00
1d550f35af command: print the current cache directory when no cmd flags are provided (#4613)
* command: print the current cache directory when no cmd flags are provided
tests: corrected the test for cache dir cmomand

comamnd 'dvc cache dir' now prints the current cache dir
when no cmd flags are provided and modified existing test for
the same.

Fixes #1998

Signed-off-by: Girish Joshi <[email protected]>

* command: updated validation for args.value and args.unset flag
according to current convention in dvc/command/cache.py

Signed-off-by: Girish Joshi <[email protected]>

* Update dvc/command/cache.py

* dont forget logger

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-10-12 08:51:38 +03:00
Paweł Redzyński fba94a4569 tests build: buil on push/pr to master (#4680) 2020-10-10 00:22:08 +03:00
Frederik Schubert 0e257adc63 remote: add WebDAV dependency (#4690)
This adds the dependency for WebDAV and enables calls like `import-url
remote://some-webdav/data`

Related to #4374
2020-10-10 00:07:34 +03:00
Frederik Schubert 8b28a268e8 remote: set dependency name for WebDAV (#4691)
When executing `dvc import-url remote://some-webdav/data`, the name of
the WebDAV dependency in the created stage for data used to be `None`.
This resulted in the following error:
```
Traceback (most recent call last):
  File "/home/schubert/projects/dvc/dvc/main.py", line 76, in main
    ret = cmd.run()
  File "/home/schubert/projects/dvc/dvc/command/imp_url.py", line 14, in run
    self.repo.imp_url(
  File "/home/schubert/projects/dvc/dvc/repo/__init__.py", line 51, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/schubert/projects/dvc/dvc/repo/scm_context.py", line 4, in run
    result = method(repo, *args, **kw)
  File "/home/schubert/projects/dvc/dvc/repo/imp_url.py", line 54, in imp_url
    stage.run()
  File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/site-packages/funcy/decorators.py", line 39, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/home/schubert/projects/dvc/dvc/stage/decorators.py", line 36, in rwlocked
    return call()
  File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/site-packages/funcy/decorators.py", line 60, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/home/schubert/projects/dvc/dvc/stage/__init__.py", line 450, in run
    self.save()
  File "/home/schubert/projects/dvc/dvc/stage/__init__.py", line 395, in save
    self.md5 = self.compute_md5()
  File "/home/schubert/projects/dvc/dvc/stage/__init__.py", line 388, in compute_md5
    m = compute_md5(self)
  File "/home/schubert/projects/dvc/dvc/stage/utils.py", line 167, in compute_md5
    return dict_md5(
  File "/home/schubert/projects/dvc/dvc/utils/__init__.py", line 118, in dict_md5
    return dict_hash(d, "md5", **kwargs)
  File "/home/schubert/projects/dvc/dvc/utils/__init__.py", line 113, in dict_hash
    byts = json.dumps(filtered, sort_keys=True).encode("utf-8")
  File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/home/schubert/miniconda3/tmp/envs/some-project/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: '<' not supported between instances of 'str' and 'NoneType'
```
2020-10-10 00:05:18 +03:00
Paweł Redzyński 09f9506ca0 github actions: migrate to envrionmental files (#4684) 2020-10-09 11:33:06 +03:00
Peter Rowlands (변기호) f20adf5e0f experiments: add --show-json for exp show (#4689) 2020-10-09 15:07:56 +09:00
Paweł Redzyński 9f90c806d5 params, plots, metrics: unify collection (#4603)
* params, plots, metrics: unify collection

* collect: split into smaller parts
2020-10-09 06:26:11 +03:00
Peter Rowlands (변기호) c16d3fa43d experiments: migrate dvc repro -e to dvc exp run (#4677)
* experiments: migrate from `dvc repro -e` to `dvc exp run`

* tests: use experiments.run instead of reproduce

* update `repro` and `exp run` commands
2020-10-09 00:26:31 +03:00
Ruslan Kuprieiev 3ceef7dbf1 dvc: get rid of ExternalGitRepo (#4685) 2020-10-08 22:04:48 +03:00
Peter Rowlands (변기호) 34804b5636 experiments: fix checkpoints callback bug when pipeline includes data (#4678) 2020-10-08 17:47:42 +09:00
Ruslan Kuprieiev 060fc4fbb7 state: use granularly (#4676)
* state: add simple reentrance

* state: pass Repo instead of LocalCache

* state: use granularly

Users could still load state preemptively if they want to bulk up
the operations, but other than that tree/cache will just load it
when they need it.
2020-10-08 08:36:14 +03:00