Commit Graph
1432 Commits
Author SHA1 Message Date
Saugat Pachhai (सौगात) f72456f67c add shallow_diff 2022-08-25 13:23:18 +03:00
Daniele Trifirò 26008f1559 import-url/update: add --no-download flag
add --no-download flag to dvc import-url/dvc update to only create/update .dvc
files without downloading the associated data.

Created .dvc files can be fetched using `dvc pull` or can be
updated using `dvc update --no-download`.
2022-08-23 23:07:01 +03:00
Ruslan Kuprieiev 3fc9f323a7 fs: don't import SSHFileSystem 2022-08-18 00:23:40 +03:00
Ruslan Kuprieiev 3a429a869e tests: lockfile: use fs_path 2022-08-13 23:09:21 +03:00
skshetry 7a0b3e9377 Update tests/func/experiments/test_queue.py 2022-08-09 17:50:48 +08:00
karajan1001 d6f8e5a507 Add flaky to a queue related test.
This is a flaky, the queue tasks status might remain "Running" even if
it had already been completed, beside sometimes I also noticed it raise
a pytest fixture error.
2022-08-09 17:50:48 +08:00
Ivan Shcheklein b7c0c41262 lock test name fix 2022-08-09 00:47:04 +03:00
Ivan Shcheklein 95421a5605 fix logic for unlock on unlocked lock calls 2022-08-08 14:25:10 +09:00
David de la Iglesia Castro 31635b8312 exp run: Use hydra for parsing --set-param.
- Overriding a config value : foo.bar=value
- Appending a config value : +foo.bar=value
- Appending or overriding a config value : ++foo.bar=value
- Removing a config value : ~foo.bar, ~foo.bar=value

See https://hydra.cc/docs/advanced/override_grammar/basic/#modifying-the-config-object

---

Breaking changes:

To modify a list, `foo[0]=bar` must now be passed as `foo.0=bar`.

Modifying a nested list inside a dictionary is not supported by omegaconf.

---

Closes #4883
Closes #5868
Closes #6129
2022-08-05 17:53:24 +02:00
David de la Iglesia Castro 45ac64ddde api: Fix params_show when stage has no params
Also fix when `deps=True` and no params in the entire pipeline.

Closes #8064
2022-08-05 16:26:19 +03:00
karajan1001 905c416ffc Decouple queue clear and remove commands 2022-08-05 16:09:51 +09:00
Ruslan Kuprieiev fcc7240617 odb: use odb.path instead of odb.cache_dir 2022-08-03 21:10:28 +03:00
Ivan Shcheklein 9ff18502b6 add gdrive remote profile field to config 2022-07-27 23:29:18 +03:00
karajan1001 9dfe88e2b1 exp run: workspace status wrong
fix: #8045
2022-07-26 15:02:09 +05:45
Saugat Pachhai (सौगात) c3ac4f84bf data status: fix untracked files output for subdir 2022-07-22 16:29:04 +03:00
Saugat Pachhai (सौगात) 61729ff66a use M fixture for concise tests 2022-07-22 16:29:04 +03:00
Saugat Pachhai (सौगात) 3dfcb82e5c add tests for data:status command 2022-07-22 16:29:04 +03:00
David de la Iglesia Castro ed1d8e2b8f exp: init: Use persist: true if checkpoint. 2022-07-21 13:21:06 +02:00
David de la Iglesia Castro c2668110b7 gc: Add --rev and --num.
Closes #7695
2022-07-12 20:13:07 +02:00
Peter Rowlands 80272a0770 queue: cleanup flaky tests 2022-07-12 21:39:49 +09:00
karajan1001 02681f2e65 Rename _all to all_ remove some tricky func test. 2022-07-12 21:39:49 +09:00
karajan1001 b37524603a Queue remove: Can remove successed,failed tasks msgs
Followed from #7592

1. Seperate remove method to a new file.
2. Add queued, failed, processed flags to remove.
3. Add new unit tests for queue status --queue/fail/success
4. Implement methods to remove done tasks.
5. add some new unit and functional test for celery_queue.remove
6. bump into dvc-task version 0.0.13
2022-07-12 21:39:49 +09:00
Peter Rowlands 1f7b802f79 exp show: collect running standalone tempdir experiments 2022-07-12 21:39:49 +09:00
Peter Rowlands 9e0788aa3a exp run: preserve --temp standalone behavior 2022-07-12 21:39:49 +09:00
Peter Rowlands e1cf62a583 exp queue: add separate standalone/tempdir implementation 2022-07-12 21:39:49 +09:00
Peter Rowlands 570f4345a0 exp queue: implement queue.logs() support 2022-07-12 21:39:49 +09:00
Peter Rowlands 425d0701c9 exp queue: return completed exps in queue status 2022-07-12 21:39:49 +09:00
Peter Rowlands 99a11f7056 exp: save failed queue runs in separate stash/reflog 2022-07-12 21:39:49 +09:00
Peter Rowlands 7ece26d58a tests: re-enable exp show celery tests 2022-07-12 21:39:49 +09:00
Peter Rowlands f26664a569 exp: re-enable run --run-all with celery queue 2022-07-12 21:39:49 +09:00
Peter Rowlands 4b0c65093a tests: use pytest-celery for exp tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 1c474e8f30 exp: re-enable most tmpdir/queue tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 46c734eb7e exp: temporarily mark tmpdir/queue tests as xfail 2022-07-12 21:39:49 +09:00
Peter Rowlands 7113568c60 exp: refactor base.py into refs.py and exceptions.py 2022-07-12 21:39:49 +09:00
Peter Rowlands 6143146974 exp: refactor stash handling into separate module 2022-07-12 21:39:49 +09:00
15ada9a1c3 Implement --date to exp gc
1. Implement `--time` to `exp gc`
2. add a unit test for it.
3. Rename `--time` to `--date` and change the warning msg.
4. To get a better msg for `gc` and `exp gc`

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: Dave Berenbaum <[email protected]>
2022-07-12 13:53:46 +02:00
karajan1001 9c59689aa7 Gc: Move old unittest gc tests to pytest 2022-07-12 13:53:46 +02:00
Ruslan Kuprieiev 8b3020aaf3 deps: bump dvc-data to 0.0.20 2022-07-08 21:37:38 +03:00
Alex Hall 182a22b8f6 checkout: use dvcignore
Fixes #7374
2022-07-07 17:20:30 +08:00
Alex Hall 892b235d0b metrics: support TOML files
Fixes #6402
2022-07-06 14:46:46 +02:00
David de la Iglesia Castro 954de0cae4 api: params_show: Raise exception if no params found.
Fixes #7926
2022-07-05 09:18:22 +02:00
David de la Iglesia Castro 909941308b parsing: Support dict unpacking in cmd.
Allow to use dictionaries as values for template interpolation but only inside the `cmd` key.

See tests/func/parsing/test_interpolated_entry.py::test_cmd_dict for detailed syntax.

Add `config.parsing` section for configuring behavior of ambiguous data types
like booleans and lists.
2022-07-05 09:02:15 +02:00
Ruslan Kuprieiev 0d71194741 dvc: normalize targets before entering brancher
Fixes #7888
2022-07-04 16:34:49 +03:00
Paweł Redzyński fbb98a87d1 plots: introduce flexible plots configuration to dvcfiles
Closes: #7086
2022-07-01 13:17:24 +02:00
David de la Iglesia Castro 20f69a121e api: Move to sub-package.
Split into sub-modules.

Split tests.
2022-06-21 20:52:19 +03:00
David de la Iglesia Castro b994c529a2 api: Add params_show.
Closes #6507

Uses `repo.params.show` with custom error_handler and postprocess the outputs for more user-friendly structure.

Extend `repo.params.show` to accept `stages` argument to cover the "params of current stage" use case.
2022-06-21 20:52:19 +03:00
David de la Iglesia Castro f106a68424 params.show: Fix deps for stages using same params file. 2022-06-21 20:52:19 +03:00
Peter Rowlands 31f67cd3de exp apply: preserve untracked files 2022-06-19 22:41:22 -07:00
Ruslan Kuprieiev 2e21c92c4f dvc: use dvc-data 0.0.8 2022-06-17 22:08:33 +03:00
Saugat Pachhai (सौगात) af649af462 deps: bump dvc-data to 0.0.6; fix imports 2022-06-14 17:50:57 +05:45