Commit Graph
1425 Commits
Author SHA1 Message Date
Ivan Shcheklein 95421a5605 fix logic for unlock on unlocked lock calls 2022-08-08 14:25:10 +09:00
David de la Iglesia Castro 31635b8312 exp run: Use hydra for parsing --set-param.
- Overriding a config value : foo.bar=value
- Appending a config value : +foo.bar=value
- Appending or overriding a config value : ++foo.bar=value
- Removing a config value : ~foo.bar, ~foo.bar=value

See https://hydra.cc/docs/advanced/override_grammar/basic/#modifying-the-config-object

---

Breaking changes:

To modify a list, `foo[0]=bar` must now be passed as `foo.0=bar`.

Modifying a nested list inside a dictionary is not supported by omegaconf.

---

Closes #4883
Closes #5868
Closes #6129
2022-08-05 17:53:24 +02:00
David de la Iglesia Castro 45ac64ddde api: Fix params_show when stage has no params
Also fix when `deps=True` and no params in the entire pipeline.

Closes #8064
2022-08-05 16:26:19 +03:00
karajan1001 905c416ffc Decouple queue clear and remove commands 2022-08-05 16:09:51 +09:00
Ruslan Kuprieiev fcc7240617 odb: use odb.path instead of odb.cache_dir 2022-08-03 21:10:28 +03:00
Ivan Shcheklein 9ff18502b6 add gdrive remote profile field to config 2022-07-27 23:29:18 +03:00
karajan1001 9dfe88e2b1 exp run: workspace status wrong
fix: #8045
2022-07-26 15:02:09 +05:45
Saugat Pachhai (सौगात) c3ac4f84bf data status: fix untracked files output for subdir 2022-07-22 16:29:04 +03:00
Saugat Pachhai (सौगात) 61729ff66a use M fixture for concise tests 2022-07-22 16:29:04 +03:00
Saugat Pachhai (सौगात) 3dfcb82e5c add tests for data:status command 2022-07-22 16:29:04 +03:00
David de la Iglesia Castro ed1d8e2b8f exp: init: Use persist: true if checkpoint. 2022-07-21 13:21:06 +02:00
David de la Iglesia Castro c2668110b7 gc: Add --rev and --num.
Closes #7695
2022-07-12 20:13:07 +02:00
Peter Rowlands 80272a0770 queue: cleanup flaky tests 2022-07-12 21:39:49 +09:00
karajan1001 02681f2e65 Rename _all to all_ remove some tricky func test. 2022-07-12 21:39:49 +09:00
karajan1001 b37524603a Queue remove: Can remove successed,failed tasks msgs
Followed from #7592

1. Seperate remove method to a new file.
2. Add queued, failed, processed flags to remove.
3. Add new unit tests for queue status --queue/fail/success
4. Implement methods to remove done tasks.
5. add some new unit and functional test for celery_queue.remove
6. bump into dvc-task version 0.0.13
2022-07-12 21:39:49 +09:00
Peter Rowlands 1f7b802f79 exp show: collect running standalone tempdir experiments 2022-07-12 21:39:49 +09:00
Peter Rowlands 9e0788aa3a exp run: preserve --temp standalone behavior 2022-07-12 21:39:49 +09:00
Peter Rowlands e1cf62a583 exp queue: add separate standalone/tempdir implementation 2022-07-12 21:39:49 +09:00
Peter Rowlands 570f4345a0 exp queue: implement queue.logs() support 2022-07-12 21:39:49 +09:00
Peter Rowlands 425d0701c9 exp queue: return completed exps in queue status 2022-07-12 21:39:49 +09:00
Peter Rowlands 99a11f7056 exp: save failed queue runs in separate stash/reflog 2022-07-12 21:39:49 +09:00
Peter Rowlands 7ece26d58a tests: re-enable exp show celery tests 2022-07-12 21:39:49 +09:00
Peter Rowlands f26664a569 exp: re-enable run --run-all with celery queue 2022-07-12 21:39:49 +09:00
Peter Rowlands 4b0c65093a tests: use pytest-celery for exp tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 1c474e8f30 exp: re-enable most tmpdir/queue tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 46c734eb7e exp: temporarily mark tmpdir/queue tests as xfail 2022-07-12 21:39:49 +09:00
Peter Rowlands 7113568c60 exp: refactor base.py into refs.py and exceptions.py 2022-07-12 21:39:49 +09:00
Peter Rowlands 6143146974 exp: refactor stash handling into separate module 2022-07-12 21:39:49 +09:00
15ada9a1c3 Implement --date to exp gc
1. Implement `--time` to `exp gc`
2. add a unit test for it.
3. Rename `--time` to `--date` and change the warning msg.
4. To get a better msg for `gc` and `exp gc`

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: Dave Berenbaum <[email protected]>
2022-07-12 13:53:46 +02:00
karajan1001 9c59689aa7 Gc: Move old unittest gc tests to pytest 2022-07-12 13:53:46 +02:00
Ruslan Kuprieiev 8b3020aaf3 deps: bump dvc-data to 0.0.20 2022-07-08 21:37:38 +03:00
Alex Hall 182a22b8f6 checkout: use dvcignore
Fixes #7374
2022-07-07 17:20:30 +08:00
Alex Hall 892b235d0b metrics: support TOML files
Fixes #6402
2022-07-06 14:46:46 +02:00
David de la Iglesia Castro 954de0cae4 api: params_show: Raise exception if no params found.
Fixes #7926
2022-07-05 09:18:22 +02:00
David de la Iglesia Castro 909941308b parsing: Support dict unpacking in cmd.
Allow to use dictionaries as values for template interpolation but only inside the `cmd` key.

See tests/func/parsing/test_interpolated_entry.py::test_cmd_dict for detailed syntax.

Add `config.parsing` section for configuring behavior of ambiguous data types
like booleans and lists.
2022-07-05 09:02:15 +02:00
Ruslan Kuprieiev 0d71194741 dvc: normalize targets before entering brancher
Fixes #7888
2022-07-04 16:34:49 +03:00
Paweł Redzyński fbb98a87d1 plots: introduce flexible plots configuration to dvcfiles
Closes: #7086
2022-07-01 13:17:24 +02:00
David de la Iglesia Castro 20f69a121e api: Move to sub-package.
Split into sub-modules.

Split tests.
2022-06-21 20:52:19 +03:00
David de la Iglesia Castro b994c529a2 api: Add params_show.
Closes #6507

Uses `repo.params.show` with custom error_handler and postprocess the outputs for more user-friendly structure.

Extend `repo.params.show` to accept `stages` argument to cover the "params of current stage" use case.
2022-06-21 20:52:19 +03:00
David de la Iglesia Castro f106a68424 params.show: Fix deps for stages using same params file. 2022-06-21 20:52:19 +03:00
Peter Rowlands 31f67cd3de exp apply: preserve untracked files 2022-06-19 22:41:22 -07:00
Ruslan Kuprieiev 2e21c92c4f dvc: use dvc-data 0.0.8 2022-06-17 22:08:33 +03:00
Saugat Pachhai (सौगात) af649af462 deps: bump dvc-data to 0.0.6; fix imports 2022-06-14 17:50:57 +05:45
dependabot[bot] bdea9847b5 build(deps-dev): Bump pylint from 2.13.9 to 2.14.1
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.13.9 to 2.14.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.13.9...v2.14.1)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-06-13 22:15:27 +05:45
Ruslan Kuprieiev ef689c5593 dvc: upgrade to dvc-objects 0.0.4 2022-06-13 18:03:53 +05:45
Daniele Trifirò 793f97bdd5 fix dvc-objects/data imports 2022-06-13 18:03:53 +05:45
Ruslan Kuprieiev 9ac2f49b57 Revert "fix dvc-objects/data imports"
This reverts commit 677da2b22c.
2022-06-10 12:01:07 +03:00
Daniele Trifirò 677da2b22c fix dvc-objects/data imports 2022-06-10 14:07:33 +05:45
dberenbaum 33b3afa2f5 exp init: create output dirs 2022-06-07 17:55:24 +05:45
Ruslan Kuprieiev a3d6b1299a brancher: use scm.root_dir to determine relative cwd
Fixes #7776
2022-06-06 12:42:22 +03:00