Commit Graph
29 Commits
Author SHA1 Message Date
Peter Rowlands (변기호) b072370efb experiments: handle state where tmp args file was committed to git (#5052)
* git: fix Stash.push args typing

* experiments: handle case where args file was inadvertently committed to
git

* add test for args file being committed to git
2020-12-08 18:10:53 +09:00
Peter Rowlands (변기호) 69eb86a06f experiments: ignore untracked files (#5029)
* git: add reset()

* experiments: exclude untracked files

* experiments: add test for untracked & staged files

* git: support checkout/reset for specific paths to HEAD state

* stage: track changed files when calling checkpoint callback

* experiments: ignore untracked files when generating exp commits

* add tests for dirty lock files

* stage: track uncached outs when generating checkpoints

* experiments: update checkpoints scm handling

* update checkpoints tests
2020-12-04 17:04:45 +09:00
Peter Rowlands (변기호) 9a9f46b2b8 scm/git: add support for resolving experiment names (#5026)
* experiments: move exp ref iterators into .utils

* git: support resolving experiment short names

* update tests

* remove change unrelated to this PR
2020-12-04 15:23:08 +09:00
Ruslan Kuprieiev 5152866634 scm: move logic to gitpython backend (#5022) 2020-12-03 20:25:20 +02:00
Peter Rowlands (변기호) 8252ae5f3c experiments: enable by default, support no_scm (#5019)
* experiments: enable by default, support no_scm

* mark experiments option obsolete
2020-12-03 10:21:19 +02:00
Peter Rowlands (변기호) 176a23e291 experiments: store experiments in refs/exps instead of heads (#4910)
* experiments: commit to refs/exps/... instead of branch heads

* update exp show

* restore checkpoint_resume kwarg

* refactor new() to use experiments stash

* experiments: use refspecs to populate and collect executors

* experiments: use new stash and merge on exp checkout

* experiments: fix executor chdir

* experiments: update run and show

* skip checkout after reproduce

* checkpoints: use experiments refspecs

* check if workspace was stashed before applying

* exp show: unify checkpoint/standalone experiment collection

* git: move iter_refs to backend

* update for scm changes

* experiments: replace exp checkout in favor of exp apply

* git: handle rev_parse errors in resolve_commit

* update refs containing and baseline check

* fix sigint handling for checkpoints

* update checkpoint exists behavior

* fix checkpoint baseline behavior

* update tests

* experiments: update gc

* update show tests

* fix type checking errors

* update diff

* scm: add diff, describe

* experiments: generate human readable names

* exp run/resume: allow user specified exp name with -n/--name

* update tests

* experiments: add `exp branch` for promoting experiment to full branch

* fix test

* fix missing file

* experiments: explicitly close scm instances before cleaning up tmp dir
2020-12-02 21:20:57 +09:00
Sami Jawhar 92b17e3a89 Fix exp checkout queued and cleanup (#4914) 2020-11-20 09:16:12 +09:00
Peter Rowlands (변기호) 40bdf613f0 experiments: preserve file permissions across tmpdir execution (#4890)
* tree: use file_mode in upload() for trees that support modes

* experiments: preserve file_mode for files in the workspace

* skip permissions tests on windows

* LocalTree: separate chmod from protect

* fix rebase issue
2020-11-18 00:47:57 +09:00
Peter Rowlands (변기호) 379ed8e6fc experiments: support branching w/modified params from intermediate checkpoint (#4898)
* experiments: Allow branching from checkpoints with modified hyper params

* experiments: initial checkpoints tree functionality for `exp show`

* go back to flat/sortable view

* add test for resume w/modified params

* tests: move experiments fixtures into conftest

* add show checkpoints test

* test checkpoint table rendering

* remove dead code

* test show for checkpoint branches
2020-11-17 21:09:46 +09:00
Sami Jawhar 72ec2a8541 Simpler param updates with python-benedict (#4780) 2020-11-12 16:42:28 +09:00
Peter Rowlands (변기호) 389efd7ae6 checkpoints: exp run and exp res[ume] refactor (#4855)
* experiments: refactor regular/checkpoint experiment separation

* fix scm_checkout pull bug

* experiments: return stages from executor.reproduce

* since executor has to move to threadpool execution anyways, we no
  longer have to work around the pickle repro result limitation

* refactor new/resume behavior

* refactor checkpoints tests

* use -f to force checkpoints reset/restart

* checkpoints: use 'exp run' and 'exp res[ume]' commands

* stage: all outs can be missing for checkpoint stages

* regular outputs may not be written at the time a checkpoint is
  generated, but subsequent checkouts of that stage should not fail due
  to the missing outs

* commit stage on checkpoint generation otherwise non-persist cached outs
are lost

* checkpoints: wait for stage proc to end after killing via terminate()

* stage: fix issues with killing checkpoint stage

* refactor signaling (use threading.Event instead of Condition)
* fix windows bug (kill all child processes via taskkill)

* experiments: refactor executor collection

* tests: move checkpoint tests into separate module

* refactor experiment tests to use stage fixture
2020-11-11 14:19:22 +09:00
Peter Rowlands (변기호) c85a46cdda experiments: add checkpoint output type (#4845)
* output: add checkpoint output type

* stage: handle checkpoint outs

* run/repro: handle checkpoint outs

* experiments: use checkpoint output type in tests

* tests: update run command tests
2020-11-06 14:44:21 +09:00
Peter Rowlands (변기호) 911534116e experiments: fix stash ordering bug (#4805) 2020-10-29 18:01:34 +09:00
Sami Jawhar 831e85682e Experiments: allow setting of list parameters (#4769)
* Allow setting of list parameteres

Check if dict is Mapping or list before using .get()

* Added test for dvc exp run with list paramater

* Add more tests for string parameter modification
2020-10-23 14:35:17 +09:00
Peter Rowlands (변기호) 79232dfedf checkpoints: --continue from last run experiment by default (#4758)
* checkpoints: `--continue` with no args resumes most recent experiment

* add test for --continue :last
2020-10-21 17:25:16 +09:00
Peter Rowlands (변기호) b9f547637c checkpoints: add --reset option (#4731)
* experiments: add --reset option to reset/restart checkpoint experiments

* add CheckpointExistsError internal exception

* update tests
2020-10-19 15:25:08 +09:00
Peter Rowlands (변기호) 9d7e483599 experiments: fix git stash mtime race condition (#4727) 2020-10-15 13:26:17 +03:00
Peter Rowlands (변기호) 4aa8fd0fbb tests: mark test_extend_branch as flaky (#4718) 2020-10-14 19:12:32 +09:00
Peter Rowlands (변기호) c16d3fa43d experiments: migrate dvc repro -e to dvc exp run (#4677)
* experiments: migrate from `dvc repro -e` to `dvc exp run`

* tests: use experiments.run instead of reproduce

* update `repro` and `exp run` commands
2020-10-09 00:26:31 +03:00
Peter Rowlands (변기호) f0a729df4d experiments: checkpoints proof of concept (#4591)
* stage: make runs checkpointable

* api: add make_checkpoint API call

* tests: add test for checkpointed stage

* experiments: implement checkpoints callback chain

* initial working state

* use .dvc/tmp for checkpoint signal file location

* force repro for checkpoint experiments

* checkout: add flag to allow missing persistent outputs

(i.e. not yet created)

* use allow_persist_missing on checkpoint experiment repro

* executor: force checkout before checkpoints experiment repro

* move checkpoint commands to `dvc exp run`

* support resuming checkpoint runs with --continue

* fix git bugs

* experiments: include checkpoint commits in `exp show`

* use 1sec sleep timers

* fix apply_workspace test conflict

* tests: add test for `dvc exp run`

* cleanup signal file behavior

* fix bool conversion

* tests: add tests for checkpoint and checkpoint_continue

* fix styling

* fix checkpoint stage test

* fix checkpoint monitor thread condition/notification
2020-10-06 15:34:11 +03:00
Peter Rowlands (변기호) 32bb46f34f experiments: support cloning parent in detached-head state (#4611)
* experiments: account for cloning detached-head parent repo

* add test for detached-head parent
2020-09-24 18:36:51 +03:00
Alexanderandaandrusenko e928034fac add py support for ParamsDependency (#4456)
* add py support for ParamsDependency

* add py support by ast

* add support modify py params

Co-authored-by: aandrusenko <[email protected]>
2020-09-24 17:25:44 +03:00
Peter Rowlands (변기호) 6a3c516b71 experiments: support committing to existing experiment branches (#4573)
* experiments: stash optional branch to extend

* experiments: support commit to existing experiment branch
2020-09-19 21:49:40 +03:00
Peter Rowlands (변기호) d6bb4341dd plots: add dvc plots -e/--experiment option (#4488)
* plots: add -e/--experiment option for plots diff

* experiments: add method to get baseline for a given experiment

* plots: default to diff against baseline instead of workspace/head for experiments

* use exact match for experiments RE since name-rev may return relative
commits
2020-08-29 00:57:28 +03:00
Peter Rowlands (변기호) 01a6580b3d experiments: fix inconsistent dvc exp checkout behavior (#4461)
* experiments: stage workspace changes before using stash apply

* add test for dvc exp checkout
2020-08-25 14:33:14 +03:00
Peter Rowlands (변기호) f789cdff89 experiments: fix bug where wrong git branch checked-out before pull (#4450) 2020-08-24 15:47:25 +03:00
Peter Rowlands (변기호) dac17e247d experiments: abstract executor tmpdir population using tree upload (#4432)
* executor: abstract tmpdir population using tree upload

* experiments: unmark flaky test

* fix wrong tree download

* LocalTree: use os.replace instead of os.rename in _upload
2020-08-21 17:32:39 +03:00
Saugat Pachhai 9f1bd98fcc Mark test_new_simple as flaky (#4429)
It is for short period of time till it gets resolved
See: #4401
2020-08-20 10:37:58 +05:45
Peter Rowlands (변기호) a56c8bbab6 Experiments proof of concept (#4199)
* experiments: add initial experiments.show

* only includes metrics + params (no code/data information)

* experiments: basic `dvc experiments show` functionality

* experiments: clone and run repro inside clone workspace

* repro: add -e/--experiment option

* experiments: hash experiments to identify duplicates

* experiments: update show

* experiments: checkout experiment after running repro

* experiments: add `dvc experiments checkout`

* experiments: add simple `dvc experiments diff` command

* just shows combined output from `metrics diff` and `plots diff` for
  now

* add very simple tests

* add some useful output messages

* experiments: suppress help messages while feature is in development

* experiment related commands will be accessible but hidden from the
  default command help messages

* add simple experiments command tests

* experiments: disable feature by default

* experiments only enabled in test environment (DVC_TEST) or when
  core.experiments config option is true

* use fetch on checkout missing revs
2020-07-17 05:41:11 +03:00