The current ci failure was caused by wrong order of the experiments.
Because our test script runs so fast that they might share the same
commit timestamp, that makes them in an uncertain order. And then gives
a wrong branch_base.
* Show csv format of experiments
fix#5989 and #5446
1. add --show-csv to dvc exp show
2. add a new functional test for csv format
3. add a new unit test for show_experiements
4. solve name duplication in table common headers (Created,Parent, etc... ), metrics files and param files.
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* params/metrics/plots/exp show: introduce onerror
* fixup
* plots/metrics/params: removing errors
* removing more errors
* removing errors
* some more removing
* new plots ux
* new ux for metrics
* new ux for params
* fixing tests
* fixups
* minor test fixes
* minor test fixes
* commands: plots/params: fix ui imports
* working on error handlng decorator, stopped at show checkpoints
* error_handler, further dvelopment
* plots just before moving loading to collect
* appending errors to results: initial passing
* fixup
* onerror: remove onerror fixture
* fixups
* plots/metrics/params/experiments: temporarily remove error warnings
* plots: extract parsing out of class method
* metrics/experiments: provide exception serializer for show-json
* params/plots/metrics: signal errors
* rebase fixups
* fix TODOs
* fixup
* skshetry's review refactor
* exp: allow fetching from arbitrary executor by git URL
* exp: write info for running executors to .dvc/tmp/exp/run
currently only includes PID and git fetch URL
* exp: refactor stash/queue cleanup
* exp: add ability to collect dict of currently running experiments
* exp show: display run/queue state in the table
* exp run: unlock main repo while running tmpdir executors
* add tests for new exp show behavior
* fix workspace checkpoint run handling
* fix issue where running stash revs were shown as queued
* use "running"/"queued"
* add executor (location) column
* compare: add TabularData.is_empty
* use td.is_empty to drop state/executor
* use "workspace" and "temp" as executor names
* apply: allow applying exp with multiple possible branches
* default to new checkpoint branch after applying ambiguous commit
* output shas in addition to exp names by default
* update show tests
* git: allow force checkout in detach_head context
* exp run: error when trying to resume from a non-head checkpoint without
modifications
* add test for apply/resume from intermediate checkpoint
* list some possible heads when multiple matches
* exp show: Allow for wildcard patterns in include and exclude params/metrics
Fixes#5642
* Add test case for nested metrics/params
* match full string in _filter_name
* pre-commit
* Update dvc/command/experiments.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* exp show: add -n/--num to display the last N commits from HEAD
* exp run: fix empty commit with no changes
* add test for exp show with multiple commits
* allow -n greater than num commits in repo
* review fix
* experiments: move executor into a module
* experiments: separate executor into base/local
* rename LocalExecutor to TempDirExecutor, run in .dvc/tmp by default
* git: restore HEAD using reset not checkout in detached_head()
* experiments: support calling BaseExecutor.reproduce directly for
workspace runs
* experiments: run single experiments in local workspace by default
* update experiments tests for workspace changes
* experiments: update checkpoints for workspace runs
* update checkpoints tests
* update test_show
* add force param for apply
* exp apply: add --force option
* exp run/res: add --temp option to force tempdir execution for single
experiments
* force apply in non-apply specific tests
* experiments: commit to refs/exps/... instead of branch heads
* update exp show
* restore checkpoint_resume kwarg
* refactor new() to use experiments stash
* experiments: use refspecs to populate and collect executors
* experiments: use new stash and merge on exp checkout
* experiments: fix executor chdir
* experiments: update run and show
* skip checkout after reproduce
* checkpoints: use experiments refspecs
* check if workspace was stashed before applying
* exp show: unify checkpoint/standalone experiment collection
* git: move iter_refs to backend
* update for scm changes
* experiments: replace exp checkout in favor of exp apply
* git: handle rev_parse errors in resolve_commit
* update refs containing and baseline check
* fix sigint handling for checkpoints
* update checkpoint exists behavior
* fix checkpoint baseline behavior
* update tests
* experiments: update gc
* update show tests
* fix type checking errors
* update diff
* scm: add diff, describe
* experiments: generate human readable names
* exp run/resume: allow user specified exp name with -n/--name
* update tests
* experiments: add `exp branch` for promoting experiment to full branch
* fix test
* fix missing file
* experiments: explicitly close scm instances before cleaning up tmp dir
* experiments: Allow branching from checkpoints with modified hyper params
* experiments: initial checkpoints tree functionality for `exp show`
* go back to flat/sortable view
* add test for resume w/modified params
* tests: move experiments fixtures into conftest
* add show checkpoints test
* test checkpoint table rendering
* remove dead code
* test show for checkpoint branches
* experiments: cleanup show UI
* show git tag/branch names instead of SHAs by default
* make timestamp formatting more like the viewer
* update tests
* place timestamps in their own column
* update command test
* exp: remove unused regex
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* experiments: add initial local tmpdir executor
* fix detached head pull
* experiments: include unchanged (unrepro'd) stages in experiment hash
* experiments: repro single experiment using tmpdir executor by default
* stage experiments as stash commits before running
* include repro args/kwargs in stashed experiments
* support running arbitrary experiment commits (including stash commits)
* experiments: use ProcessPoolExecutor
* experiments: add `dvc exp` alias for `dvc experiments`
* experiments: add `dvc repro --experiment --queue`
* `--queue` can be used to stage an experiment for future execution
* experiments: show queued (unexecuted) experiments in `dvc exp show`
* revert ProcessPoolExecutor change
* experiments: add `dvc repro --run-all [--jobs]`
* `--run-all` can be used to run all queued experiments in parallel
* cleanup output
* experiments: use ProcessPoolExecutor
* fix returning unpicklable objects error
* update tests
* fix windows cleanup issue
* on windows tempdir cannot be removed if we are chdir'd into that
directory
* experiments: add initial experiments.show
* only includes metrics + params (no code/data information)
* experiments: basic `dvc experiments show` functionality
* experiments: clone and run repro inside clone workspace
* repro: add -e/--experiment option
* experiments: hash experiments to identify duplicates
* experiments: update show
* experiments: checkout experiment after running repro
* experiments: add `dvc experiments checkout`
* experiments: add simple `dvc experiments diff` command
* just shows combined output from `metrics diff` and `plots diff` for
now
* add very simple tests
* add some useful output messages
* experiments: suppress help messages while feature is in development
* experiment related commands will be accessible but hidden from the
default command help messages
* add simple experiments command tests
* experiments: disable feature by default
* experiments only enabled in test environment (DVC_TEST) or when
core.experiments config option is true
* use fetch on checkout missing revs
David de la Iglesia Castro
Saugat Pachhai
Paweł Redzyński