Commit Graph
8 Commits
Author SHA1 Message Date
Peter Rowlands (변기호) 176a23e291 experiments: store experiments in refs/exps instead of heads (#4910)
* experiments: commit to refs/exps/... instead of branch heads

* update exp show

* restore checkpoint_resume kwarg

* refactor new() to use experiments stash

* experiments: use refspecs to populate and collect executors

* experiments: use new stash and merge on exp checkout

* experiments: fix executor chdir

* experiments: update run and show

* skip checkout after reproduce

* checkpoints: use experiments refspecs

* check if workspace was stashed before applying

* exp show: unify checkpoint/standalone experiment collection

* git: move iter_refs to backend

* update for scm changes

* experiments: replace exp checkout in favor of exp apply

* git: handle rev_parse errors in resolve_commit

* update refs containing and baseline check

* fix sigint handling for checkpoints

* update checkpoint exists behavior

* fix checkpoint baseline behavior

* update tests

* experiments: update gc

* update show tests

* fix type checking errors

* update diff

* scm: add diff, describe

* experiments: generate human readable names

* exp run/resume: allow user specified exp name with -n/--name

* update tests

* experiments: add `exp branch` for promoting experiment to full branch

* fix test

* fix missing file

* experiments: explicitly close scm instances before cleaning up tmp dir
2020-12-02 21:20:57 +09:00
Peter Rowlands (변기호) 389efd7ae6 checkpoints: exp run and exp res[ume] refactor (#4855)
* experiments: refactor regular/checkpoint experiment separation

* fix scm_checkout pull bug

* experiments: return stages from executor.reproduce

* since executor has to move to threadpool execution anyways, we no
  longer have to work around the pickle repro result limitation

* refactor new/resume behavior

* refactor checkpoints tests

* use -f to force checkpoints reset/restart

* checkpoints: use 'exp run' and 'exp res[ume]' commands

* stage: all outs can be missing for checkpoint stages

* regular outputs may not be written at the time a checkpoint is
  generated, but subsequent checkouts of that stage should not fail due
  to the missing outs

* commit stage on checkpoint generation otherwise non-persist cached outs
are lost

* checkpoints: wait for stage proc to end after killing via terminate()

* stage: fix issues with killing checkpoint stage

* refactor signaling (use threading.Event instead of Condition)
* fix windows bug (kill all child processes via taskkill)

* experiments: refactor executor collection

* tests: move checkpoint tests into separate module

* refactor experiment tests to use stage fixture
2020-11-11 14:19:22 +09:00
Peter Rowlands (변기호) 80e181f2fc experiments: add dvc exp gc (#4803)
* experiments: add `dvc exp gc`

* tests: add exp gc tests

* add exp gc command test

* fix review issue
2020-11-06 19:05:41 +09:00
Peter Rowlands (변기호) b9f547637c checkpoints: add --reset option (#4731)
* experiments: add --reset option to reset/restart checkpoint experiments

* add CheckpointExistsError internal exception

* update tests
2020-10-19 15:25:08 +09:00
Peter Rowlands (변기호) c16d3fa43d experiments: migrate dvc repro -e to dvc exp run (#4677)
* experiments: migrate from `dvc repro -e` to `dvc exp run`

* tests: use experiments.run instead of reproduce

* update `repro` and `exp run` commands
2020-10-09 00:26:31 +03:00
Peter Rowlands (변기호) f0a729df4d experiments: checkpoints proof of concept (#4591)
* stage: make runs checkpointable

* api: add make_checkpoint API call

* tests: add test for checkpointed stage

* experiments: implement checkpoints callback chain

* initial working state

* use .dvc/tmp for checkpoint signal file location

* force repro for checkpoint experiments

* checkout: add flag to allow missing persistent outputs

(i.e. not yet created)

* use allow_persist_missing on checkpoint experiment repro

* executor: force checkout before checkpoints experiment repro

* move checkpoint commands to `dvc exp run`

* support resuming checkpoint runs with --continue

* fix git bugs

* experiments: include checkpoint commits in `exp show`

* use 1sec sleep timers

* fix apply_workspace test conflict

* tests: add test for `dvc exp run`

* cleanup signal file behavior

* fix bool conversion

* tests: add tests for checkpoint and checkpoint_continue

* fix styling

* fix checkpoint stage test

* fix checkpoint monitor thread condition/notification
2020-10-06 15:34:11 +03:00
Peter Rowlands (변기호)andRuslan Kuprieiev e2b5ab996f experiments: make dvc exp show more consistent w/viewer (#4489)
* experiments: cleanup show UI

* show git tag/branch names instead of SHAs by default
* make timestamp formatting more like the viewer

* update tests

* place timestamps in their own column

* update command test

* exp: remove unused regex

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-08-31 00:02:57 +03:00
Peter Rowlands (변기호) a56c8bbab6 Experiments proof of concept (#4199)
* experiments: add initial experiments.show

* only includes metrics + params (no code/data information)

* experiments: basic `dvc experiments show` functionality

* experiments: clone and run repro inside clone workspace

* repro: add -e/--experiment option

* experiments: hash experiments to identify duplicates

* experiments: update show

* experiments: checkout experiment after running repro

* experiments: add `dvc experiments checkout`

* experiments: add simple `dvc experiments diff` command

* just shows combined output from `metrics diff` and `plots diff` for
  now

* add very simple tests

* add some useful output messages

* experiments: suppress help messages while feature is in development

* experiment related commands will be accessible but hidden from the
  default command help messages

* add simple experiments command tests

* experiments: disable feature by default

* experiments only enabled in test environment (DVC_TEST) or when
  core.experiments config option is true

* use fetch on checkout missing revs
2020-07-17 05:41:11 +03:00