Peter Rowlands
570f4345a0
exp queue: implement queue.logs() support
2022-07-12 21:39:49 +09:00
Peter Rowlands
425d0701c9
exp queue: return completed exps in queue status
2022-07-12 21:39:49 +09:00
Peter Rowlands
99a11f7056
exp: save failed queue runs in separate stash/reflog
2022-07-12 21:39:49 +09:00
Peter Rowlands
8ab46c1549
exp: simplify shutdown behavior
2022-07-12 21:39:49 +09:00
Peter Rowlands
8b663f2719
exp: use single atomic task instead of chaining
2022-07-12 21:39:49 +09:00
Peter Rowlands
c0db513cf0
dvc-task: bump to 0.0.11
2022-07-12 21:39:49 +09:00
Peter Rowlands
bf1bc9625d
exp: use celery async wait in queue get_result
2022-07-12 21:39:49 +09:00
Peter Rowlands
686c15dab9
exp: use a single chain for all exp tasks
...
- removes separated scheduling for setup_exp
2022-07-12 21:39:49 +09:00
Peter Rowlands
10aa833f80
dvc-task: bump to 0.0.10
2022-07-12 21:39:49 +09:00
Peter Rowlands
9f5af04963
exp: update tests for match_queue_entry refactor
2022-07-12 21:39:49 +09:00
Peter Rowlands
ac5d72967a
dvc-task: bump to 0.0.9
2022-07-12 21:39:49 +09:00
karajan1001
bcc2f3b389
Reorganized get_queue_entry_by_names
2022-07-12 21:39:49 +09:00
karajan1001
3d0ee0d202
exp queue: monitor queue status ( #7590 )
...
fix : #7590
1. Add two new command `queue status` and `queue attach`
2. Add unit test for the CLI
3. Make `Running` exps can be found by `get_queue_entry_by_names`
2022-07-12 21:39:49 +09:00
karajan1001
6b8c87b58a
Add more unit test and implement a new method
...
1. Add some more unit test for `celery_queue.kill`
2. Implement `kill` argument for `celery_queue.shutdown`
3. Use dataclass's internal methods instead of `__eq__`
2022-07-12 21:39:49 +09:00
Peter Rowlands
fee1d02791
dvc-task: update to 0.0.8
2022-07-12 21:39:49 +09:00
Peter Rowlands
d766c01d2d
exp: enable late_ack to disable celery prefetch
2022-07-12 21:39:49 +09:00
Peter Rowlands
dc56c4c9b0
exp: use full task chain when checking active tasks
2022-07-12 21:39:49 +09:00
Peter Rowlands
7ece26d58a
tests: re-enable exp show celery tests
2022-07-12 21:39:49 +09:00
Peter Rowlands
f26664a569
exp: re-enable run --run-all with celery queue
2022-07-12 21:39:49 +09:00
Peter Rowlands
4b0c65093a
tests: use pytest-celery for exp tests
2022-07-12 21:39:49 +09:00
karajan1001
12fc8c4e7d
exp queue: Add two command queue start and stop ( #7589 )
...
fix : #7589
1. Add two sub-command `dvc queue start` and `dvc queue stop`
2. Add a unit test to test them
2022-07-12 21:39:49 +09:00
karajan1001 and Peter Rowlands
a6cb4eb72e
exp queue: remove queued experiments( #7591 )
...
fix : #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them
Co-authored-by: Peter Rowlands (변기호) <[email protected]
2022-07-12 21:39:49 +09:00
karajan1001
4df5a1321d
exp: implement internal queue.kill support( #7587 )
...
fix : #7587
1. Add implement kill method for local queue class
2. Add a test to make sure the following job will be success after the
original job was killed.
3. Some refactoring work on `exp remove`
2022-07-12 21:39:49 +09:00
Peter Rowlands
22deca3a6a
exp: use celery threadpool worker instead of prefork
2022-07-12 21:39:49 +09:00
Peter Rowlands
dc4292c602
exp: implement queue shutdown
2022-07-12 21:39:49 +09:00
Peter Rowlands
56f90eb3c9
exp: add test_queue unit test helper for celery queue
2022-07-12 21:39:49 +09:00
Peter Rowlands
1c474e8f30
exp: re-enable most tmpdir/queue tests
2022-07-12 21:39:49 +09:00
Peter Rowlands
24654fe16c
exp: use dvc.daemon to run worker on windows
2022-07-12 21:39:49 +09:00
Peter Rowlands
44820e74a3
exp: wait for task collection when getting exp result
2022-07-12 21:39:49 +09:00
Peter Rowlands
c16908e9fc
exp: use queue remove in gc
2022-07-12 21:39:49 +09:00
Peter Rowlands
33dbdd20af
exp: exclude celery stash ref from util iterators
2022-07-12 21:39:49 +09:00
Peter Rowlands
69d7bdd0c0
exp: generate unique worker node names per DVC repo
2022-07-12 21:39:49 +09:00
Peter Rowlands
b9f4398826
exp: start migrating info logging to ui
2022-07-12 21:39:49 +09:00
Peter Rowlands
a6e7d43668
exp: return exp results for celery runs
2022-07-12 21:39:49 +09:00
Peter Rowlands
e2ee69ddb2
dvc-task: update to 0.0.6
2022-07-12 21:39:49 +09:00
Peter Rowlands
e8448345f1
exp: follow output on run-all
2022-07-12 21:39:49 +09:00
Peter Rowlands
acab008c65
dvc-task: update to 0.0.3
2022-07-12 21:39:49 +09:00
Peter Rowlands
8c74775a4e
exp: autostart celery worker
2022-07-12 21:39:49 +09:00
Peter Rowlands
9b31a4b463
exp: support remove from celery queue
2022-07-12 21:39:49 +09:00
Peter Rowlands
9f022c576c
dvc-task: update to 0.0.2
2022-07-12 21:39:49 +09:00
Peter Rowlands
316029a9d2
exp: use queue status in exp show
2022-07-12 21:39:49 +09:00
Peter Rowlands
34fe643cee
exp: add queue.iter_active for iterating over active exps
2022-07-12 21:39:49 +09:00
Peter Rowlands
6a01cc530e
exp: use queue.iter_queued to get queued status in exp show
2022-07-12 21:39:49 +09:00
Peter Rowlands
81475bee09
dvc-task: update to 0.0.1
2022-07-12 21:39:49 +09:00
Peter Rowlands
a167dff019
exp: implement celery exp task collection
2022-07-12 21:39:49 +09:00
Peter Rowlands
cffd587952
exp: add queue-worker plumbing command
2022-07-12 21:39:49 +09:00
Peter Rowlands
a0ae9d3f45
exp: migrate workspace repro to queues
2022-07-12 21:39:49 +09:00
Peter Rowlands
e74f8f9e11
exp: add naive workspace queue
2022-07-12 21:39:49 +09:00
Peter Rowlands
e45ca1e830
exp: start moving queueing/execution from __init__
2022-07-12 21:39:49 +09:00
Peter Rowlands
46c734eb7e
exp: temporarily mark tmpdir/queue tests as xfail
2022-07-12 21:39:49 +09:00
Peter Rowlands
karajan1001