Experiments proof of concept (#4199)

* experiments: add initial experiments.show

* only includes metrics + params (no code/data information)

* experiments: basic `dvc experiments show` functionality

* experiments: clone and run repro inside clone workspace

* repro: add -e/--experiment option

* experiments: hash experiments to identify duplicates

* experiments: update show

* experiments: checkout experiment after running repro

* experiments: add `dvc experiments checkout`

* experiments: add simple `dvc experiments diff` command

* just shows combined output from `metrics diff` and `plots diff` for
  now

* add very simple tests

* add some useful output messages

* experiments: suppress help messages while feature is in development

* experiment related commands will be accessible but hidden from the
  default command help messages

* add simple experiments command tests

* experiments: disable feature by default

* experiments only enabled in test environment (DVC_TEST) or when
  core.experiments config option is true

* use fetch on checkout missing revs
This commit is contained in:
Peter Rowlands (변기호)
2020-07-17 05:41:11 +03:00
committed by GitHub
parent 4edf368d68
commit a56c8bbab6
17 changed files with 806 additions and 1 deletions
@@ -0,0 +1,24 @@
from tests.func.test_repro_multistage import COPY_SCRIPT
def test_new_simple(tmp_dir, scm, dvc, mocker):
tmp_dir.gen("copy.py", COPY_SCRIPT)
tmp_dir.gen("params.yaml", "foo: 1")
stage = dvc.run(
cmd="python copy.py params.yaml metrics.yaml",
metrics_no_cache=["metrics.yaml"],
params=["foo"],
name="copy-file",
)
scm.add(["dvc.yaml", "dvc.lock", "copy.py", "params.yaml", "metrics.yaml"])
scm.commit("init")
tmp_dir.gen("params.yaml", "foo: 2")
new_mock = mocker.spy(dvc.experiments, "new")
dvc.reproduce(stage.addressing, experiment=True)
new_mock.assert_called_once()
assert (
tmp_dir / ".dvc" / "experiments" / "metrics.yaml"
).read_text() == "foo: 2"