Commit Graph
2706 Commits
Author SHA1 Message Date
David de la Iglesia Castro c2668110b7 gc: Add --rev and --num.
Closes #7695
2022-07-12 20:13:07 +02:00
Peter Rowlands 80272a0770 queue: cleanup flaky tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 3130a9a75f queue status: use summary style output for worker status 2022-07-12 21:39:49 +09:00
karajan1001 c0368e1756 Refactor: Move remove_revs from BaseStashQueue to expstash 2022-07-12 21:39:49 +09:00
karajan1001 02681f2e65 Rename _all to all_ remove some tricky func test. 2022-07-12 21:39:49 +09:00
karajan1001 b37524603a Queue remove: Can remove successed,failed tasks msgs
Followed from #7592

1. Seperate remove method to a new file.
2. Add queued, failed, processed flags to remove.
3. Add new unit tests for queue status --queue/fail/success
4. Implement methods to remove done tasks.
5. add some new unit and functional test for celery_queue.remove
6. bump into dvc-task version 0.0.13
2022-07-12 21:39:49 +09:00
Peter Rowlands 1f7b802f79 exp show: collect running standalone tempdir experiments 2022-07-12 21:39:49 +09:00
Peter Rowlands 9e0788aa3a exp run: preserve --temp standalone behavior 2022-07-12 21:39:49 +09:00
Peter Rowlands e1cf62a583 exp queue: add separate standalone/tempdir implementation 2022-07-12 21:39:49 +09:00
Peter Rowlands ce4cee184e exp queue: separate queue implementations into their own modules 2022-07-12 21:39:49 +09:00
karajan1001andPeter Rowlands bd74172633 queue status: add some unit test to increase the coverage
Followed by #7903

Add a new unit test to increase the coverage of test

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2022-07-12 21:39:49 +09:00
dberenbaum 164efe089f queue: ui cleanup 2022-07-12 21:39:49 +09:00
karajan1001 d7a917e2ab queue status: use a short rev to represent tasks
1. use a short rev to represent tasks
2. Add a unit test for the result shown
2022-07-12 21:39:49 +09:00
Peter Rowlands 67cbdf6de8 exp queue: remove dvc queue attach command 2022-07-12 21:39:49 +09:00
Peter Rowlands f12091002f exp queue: add dvc queue logs command 2022-07-12 21:39:49 +09:00
Peter Rowlands 570f4345a0 exp queue: implement queue.logs() support 2022-07-12 21:39:49 +09:00
Peter Rowlands 425d0701c9 exp queue: return completed exps in queue status 2022-07-12 21:39:49 +09:00
Peter Rowlands 99a11f7056 exp: save failed queue runs in separate stash/reflog 2022-07-12 21:39:49 +09:00
Peter Rowlands 8ab46c1549 exp: simplify shutdown behavior 2022-07-12 21:39:49 +09:00
Peter Rowlands c0db513cf0 dvc-task: bump to 0.0.11 2022-07-12 21:39:49 +09:00
Peter Rowlands 10aa833f80 dvc-task: bump to 0.0.10 2022-07-12 21:39:49 +09:00
Peter Rowlands 9f5af04963 exp: update tests for match_queue_entry refactor 2022-07-12 21:39:49 +09:00
karajan1001 3d0ee0d202 exp queue: monitor queue status (#7590)
fix: #7590
1. Add two new command `queue status` and `queue attach`
2. Add unit test for the CLI
3. Make `Running` exps can be found by `get_queue_entry_by_names`
2022-07-12 21:39:49 +09:00
karajan1001 6b8c87b58a Add more unit test and implement a new method
1. Add some more unit test for `celery_queue.kill`
2. Implement `kill` argument for `celery_queue.shutdown`
3. Use dataclass's internal methods instead of `__eq__`
2022-07-12 21:39:49 +09:00
Peter Rowlands 7ece26d58a tests: re-enable exp show celery tests 2022-07-12 21:39:49 +09:00
Peter Rowlands f26664a569 exp: re-enable run --run-all with celery queue 2022-07-12 21:39:49 +09:00
Peter Rowlands 4b0c65093a tests: use pytest-celery for exp tests 2022-07-12 21:39:49 +09:00
karajan1001 12fc8c4e7d exp queue: Add two command queue start and stop (#7589)
fix: #7589

1. Add two sub-command `dvc queue start` and `dvc queue stop`
2. Add a unit test to test them
2022-07-12 21:39:49 +09:00
karajan1001andPeter Rowlands a6cb4eb72e exp queue: remove queued experiments(#7591)
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2022-07-12 21:39:49 +09:00
karajan1001 4df5a1321d exp: implement internal queue.kill support(#7587)
fix: #7587
1. Add implement kill method for local queue class
2. Add a test to make sure the following job will be success after the
   original job was killed.
3. Some refactoring work on `exp remove`
2022-07-12 21:39:49 +09:00
Peter Rowlands dc4292c602 exp: implement queue shutdown 2022-07-12 21:39:49 +09:00
Peter Rowlands 56f90eb3c9 exp: add test_queue unit test helper for celery queue 2022-07-12 21:39:49 +09:00
Peter Rowlands 1c474e8f30 exp: re-enable most tmpdir/queue tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 46c734eb7e exp: temporarily mark tmpdir/queue tests as xfail 2022-07-12 21:39:49 +09:00
Peter Rowlands 7113568c60 exp: refactor base.py into refs.py and exceptions.py 2022-07-12 21:39:49 +09:00
Peter Rowlands 6143146974 exp: refactor stash handling into separate module 2022-07-12 21:39:49 +09:00
15ada9a1c3 Implement --date to exp gc
1. Implement `--time` to `exp gc`
2. add a unit test for it.
3. Rename `--time` to `--date` and change the warning msg.
4. To get a better msg for `gc` and `exp gc`

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: Dave Berenbaum <[email protected]>
2022-07-12 13:53:46 +02:00
karajan1001andPeter Rowlands 217eb13495 Add argument commit_time to gc/brancher and iter_revs
1. Add argument `commit_time` to `gc`/`brancher` and `iter_revs`
2. Add unit test for iter_revs.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2022-07-12 13:53:46 +02:00
karajan1001andDave Berenbaum 20d1f7a87d gc: add a new flag --time (#678)
fix #678
1. Add a new flag `--time` to `dvc gc`.
2. Modify the unit test of `dvc gc`
3. Fix an argument checking error and update the error msg.

Co-authored-by: Dave Berenbaum <[email protected]>
2022-07-12 13:53:46 +02:00
karajan1001 9c59689aa7 Gc: Move old unittest gc tests to pytest 2022-07-12 13:53:46 +02:00
David de la Iglesia Castro d47e3c40b8 tests: limit printed durations to 50 2022-07-11 12:36:37 +03:00
Ruslan Kuprieiev 8b3020aaf3 deps: bump dvc-data to 0.0.20 2022-07-08 21:37:38 +03:00
Alex Hall 182a22b8f6 checkout: use dvcignore
Fixes #7374
2022-07-07 17:20:30 +08:00
Alex Hall 892b235d0b metrics: support TOML files
Fixes #6402
2022-07-06 14:46:46 +02:00
David de la Iglesia Castro 954de0cae4 api: params_show: Raise exception if no params found.
Fixes #7926
2022-07-05 09:18:22 +02:00
David de la Iglesia Castro 909941308b parsing: Support dict unpacking in cmd.
Allow to use dictionaries as values for template interpolation but only inside the `cmd` key.

See tests/func/parsing/test_interpolated_entry.py::test_cmd_dict for detailed syntax.

Add `config.parsing` section for configuring behavior of ambiguous data types
like booleans and lists.
2022-07-05 09:02:15 +02:00
Ruslan Kuprieiev 0d71194741 dvc: normalize targets before entering brancher
Fixes #7888
2022-07-04 16:34:49 +03:00
Paweł Redzyński fbb98a87d1 plots: introduce flexible plots configuration to dvcfiles
Closes: #7086
2022-07-01 13:17:24 +02:00
David de la Iglesia Castro c336507e51 render: image_converter: Support slash in revision.
Fixes #7934
2022-06-27 10:53:03 +02:00
dberenbaum 2773e99afc config: remove core.jobs dead code 2022-06-22 11:36:01 +08:00