Commit Graph
67 Commits
Author SHA1 Message Date
Daniele Trifirò 73b46a5789 exp save: initial implementation 2022-11-28 15:49:34 +01:00
karajan1001 884182f6f2 Add "status" to replace "running" and "queued" in output of exp show
fix: #7986
1. Add two new flags  `--hide-queued` and  `--hide-failed` to `exp show`
2. Allow `exp show` to show failed experiments.
3. Add unit test for the failed experiments shown.
4. Add name support for failed exp
5. Add error msg to the `exp show` output
2022-10-06 17:20:03 +08:00
karajan1001 2182c4508d Make exp remove flags and arguments mutually exclusive.
1. Current behavior didn't follow the either or description.
2. Make the underlying logic more clear.
2022-08-05 16:09:51 +09:00
15ada9a1c3 Implement --date to exp gc
1. Implement `--time` to `exp gc`
2. add a unit test for it.
3. Rename `--time` to `--date` and change the warning msg.
4. To get a better msg for `gc` and `exp gc`

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: Dave Berenbaum <[email protected]>
2022-07-12 13:53:46 +02:00
dberenbaum 33b3afa2f5 exp init: create output dirs 2022-06-07 17:55:24 +05:45
Ruslan Kuprieiev 844f77b4f9 dvc: drop python 3.7
Also upgraded with `pyupgrade --py38-plus`

Fixes #7708
2022-05-30 16:51:35 +03:00
dberenbaum 477f266dc5 exp init: fixes #7534; simplifies/updates exp init --live 2022-05-09 20:58:27 +05:45
Peter Rowlands 444bf27e6f exp show: add internal --no-fetch option to disable temp/queue ref fetch 2022-04-21 15:45:59 +09:00
Saugat Pachhai (सौगात) 02b2ab6804 stage/exp list: alias --names-only to --name-only
Fixes #7570.
2022-04-21 06:53:16 +05:45
Saugat Pachhai (सौगात) 7d0d6a38d7 exp init: change ui prompts structure and output 2022-03-29 16:20:32 +05:45
David de la Iglesia Castro ddd6c97f71 render: Extract to dvc_render
Create and use separate package for rendering logic.
Closes #6944
2022-03-28 19:34:05 +02:00
karajan1001andJorge Orpinel efc021da73 exp remove: rename --all-commits flag and unify the collection of revs (#7155)
fix: #7155
1. rename flags `-A/--all-commits` in `exp remove`
2. add new flag `-n/--num` in `exp remove`
3. unify the revision collection in `exp remove`
4. add unit and func tests for `exp remove`

Co-authored-by: Jorge Orpinel <[email protected]>
2022-03-09 18:54:15 +08:00
karajan1001 244637e299 exp push/pull: add --all-commits flag and unify the collection of revs (#7154)
fix: #7154
1. rename flags `-A/--all-commits` in the `exp pull/push`
2. add new flag "--rev" and "--num" in the `exp pull/push`
3. Unify the collection of revs in `exp push/pull`
4. add unit and func tests for `exp push/pull`
2022-03-09 17:43:26 +08:00
karajan1001andJorge Orpinel d033345860 exp list: add the new flags (#7153)
fix: #7153
Unify the flags in `exp list` to `exp show`

1. Add new `-A`, `-n` flags to `exp list` command.
2. Add unit test for the flag parse.
3. unify the revs retrive logic to `exp show`.
4. Add functional test for `exp list`
5. Update the documents.

Co-authored-by: Jorge Orpinel <[email protected]>
2022-03-08 16:17:33 +08:00
Peter Rowlands d80c43135f exp run: remove unsupported (repro only) arguments 2022-03-07 12:01:51 +01:00
David de la Iglesia Castro d4861d535a TabularData: Simplify drop internals.
Update `subset` type hint.
2022-01-22 09:41:29 +08:00
David de la Iglesia Castro 43953ace53 exp show: Add --drop and --keep args
Removed `--include-metrics` / `--include-params` and
`--exclude-metrics` / `--exclude-params`.

Removed `--no-timestamp` . Can be done by `--drop Created`.

`--drop` and `--keep` operate directly on the table columns.

`--keep` does not perform any filtering. It's only used
to specify columns to keep despite the other filtering.

For example `--only-changed --keep foo` will prevent
`foo` from being removed by `--only-changed`.

Another example, `--drop train.* --keep train.dropout`
will remove all columns matching `train.*` except for
`train.dropout`.

Closes #7079
Closes #7080
Pre-requisite for #6434
2022-01-20 03:28:33 +01:00
Saugat Pachhai (सौगात) 446516d788 refactor: rename dvc.command to dvc.commands 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) 4268b5a339 cli: don't inherit DvcParserError from DvcException, move to dvc.cli 2022-01-19 19:41:40 +05:45
karajan1001 57263f15fa Pull: accept multiple as input(#7196)
fix: #7196

New feature: now `dvc exp pull` can accept a list of experiment names as input.

2. add support to list input for the `exp pull`
3. add function tests for the multi exp pull
4. add unit test for the arg parse
2022-01-18 17:31:55 +08:00
karajan1001 9bb6494e55 Push: accept multiple as input(#7196)
fix: #7196

New feature: now `dvc exp push` can accept a list of experiment names as input.

1. refactor of experiment push
2. add support to list input for the `exp push`
3. add function tests for the multi exp push
4. add unit test for the arg parse
2022-01-18 17:31:55 +08:00
Peter Rowlands dd5d999644 exp: add exp run --machine flag 2022-01-13 16:34:20 +09:00
karajan1001andPeter Rowlands b673ac24d0 exp show: add --rev flag (#7152)
fix: #7152
1. Add `--rev` flag to `dvc exp show`.
2. Add -1 support for `--num` flag`.
3. Extract revision found logic to `utils`.
4. Brancher use this revision found logic.
5. Update command unit test
6. Add a util unit test.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2022-01-11 19:01:34 +08:00
David de la Iglesia Castro 0a4958bafd TabularData: Decouple pcp from render. 2022-01-03 16:30:54 +01:00
David de la Iglesia Castro 5a9f861921 exp show: Rename html to pcp
Closes #7165
2022-01-03 16:30:54 +01:00
David de la Iglesia Castro 4d5be46ab4 exp show --html: Handle fill_value.
Adds new argument `fill_value` to ParallelCoordinatesRenderer.

For scalar columns, prevent converting to categorical by explictly inserting None when value is missing (value == fill_value).

For categorical columns, replace `fill_value` with `"Missing"` and ensure it is the last item so it gets rendered on top of the column.
2021-12-17 14:45:12 +09:00
David de la Iglesia Castro 5ee82d4000 exp show: Add parallel coordinates plot.
Uses `TabularData.to_parallel_coordinates`.
Adds new arguments: `html`, `out`, `open`

Reuses `--sort-by` to define colorscale.

Closes #4455
2021-12-17 14:45:12 +09:00
karajan1001andPeter Rowlands 0deab0118a exp: split exp sub command (#5489)
related to #5489
Current dvc/command/experiments.py is too long for switch between cmd class
and the argument parser.

Split the sub command in dvc/command/experiments.py to individual files

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-12-14 19:16:04 +08:00
pre-commit-ci[bot] 0bc101964e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2021-12-07 09:55:03 +05:45
Saugat Pachhai (सौगात) 3319808ed2 exp init: use shortlinks for doclinks 2021-12-07 09:55:03 +05:45
Saugat Pachhai (सौगात) cd37ebca37 exp init: minor refactoring 2021-12-07 08:40:46 +05:45
dberenbaum 4385720fab exp init: update tests 2021-12-07 07:34:31 +05:45
Saugat Pachhai (सौगात) 389fd48b5a exp init: change type to --type=live to --type=dl 2021-12-07 05:56:23 +05:45
Saugat Pachhai (सौगात) cb941a836d tests: cover exp init 2021-11-19 18:58:23 +05:45
David de la Iglesia Castro 28dd39a1a0 exp diff: Remove --old option.
After #6814 the header of `diff_table` is `a_rev/`b_rev` instead of `Old`/`New` so the name of the option was counterintuitive.

In addition, make `a_rev` show by default, for consistency with output format of other `diff_table` uses (`dvc metrics diff` / `dvc params diff`).
2021-11-12 12:58:12 +02:00
Paweł Redzyński 36ed182f26 argparse: dont override cmd
Related: #6956, https://bugs.python.org/issue45235
2021-11-11 01:14:03 +02:00
f964a44e7c Exp show md (#6799)
* Added `--md`/`--show-md` option to `exp show`.

* Added test_show_experiments_md

* Fixed re-defining all_experiments

* Fixed all_experiments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added dest markdown

* Added textwrap.dedent

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update experiments.py

* Apply suggestions from code review

Co-authored-by: Saugat Pachhai <[email protected]>

* Update tests/unit/command/test_compat_flag.py

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai <[email protected]>
2021-10-20 11:04:57 +05:45
David de la Iglesia Castro 7e4f443002 Added test_show_experiments_sort_by (#6798)
* Added test_show_experiments_sort_by

* Added comment

* help formatting
2021-10-14 14:49:00 +03:00
Saugat Pachhai 4e17279440 exp init: add config support (#6761) 2021-10-08 20:14:01 +05:45
Saugat Pachhai 9a1d74048a cli: add short alias for --show-{csv,json,md} (#6711)
As `--csv`/`--json`/`--md`
2021-09-30 20:38:32 +05:45
GaoandPeter Rowlands 6106b1341b Show csv format of experiments (#6468)
* Show csv format of experiments

fix #5989 and #5446 

1. add --show-csv to dvc exp show
2. add a new functional test for csv format
3. add a new unit test for show_experiements
4. solve name duplication in table common headers (Created,Parent, etc... ), metrics files and param files.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-09-08 16:59:01 +08:00
4cc06339c8 Clean up remotes's exps (#6471)
* Clean up remotes's exps

fix #6006
1. add a new argument `--git-remote` to `dvc exp remove`
2. support remote a special remote exp
3. add tests for it
4. fix  #6421
5. add a test for #6421.
6. extract some functions from the `dvc pull`, `dvc push`, `dvc remove` to utils.
7. add a tests for this new util function `resolve_exp_ref`
8. add `__init__.py` to some test package for the pytest fail

Co-authored-by: Jorge Orpinel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dave Berenbaum <[email protected]>
2021-09-05 10:06:07 +08:00
GaoandPeter Rowlands 57899ee86d Remove all of the experiments in the workspace (#6394)
* Remove all of the experiments in the workspace

fix #5676
1. remove all the experiments in workspace
2. add tests for it

* Only remove baseline ones

* Add a new arguments --all

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Rename and shortcut for these commands

* Solve the issue from removing a changing list.

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* delete workspace option

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-08-19 10:18:51 +09:00
Paweł Redzyński 21af7438db plots/metrics/params/experiments show: stop throwing exceptions (#5984)
* params/metrics/plots/exp show: introduce onerror

* fixup

* plots/metrics/params: removing errors

* removing more errors

* removing errors

* some more removing

* new plots ux

* new ux for metrics

* new ux for params

* fixing tests

* fixups

* minor test fixes

* minor test fixes

* commands: plots/params: fix ui imports

* working on error handlng decorator, stopped at show checkpoints

* error_handler, further dvelopment

* plots just before moving loading to collect

* appending errors to results: initial passing

* fixup

* onerror: remove onerror fixture

* fixups

* plots/metrics/params/experiments: temporarily remove error warnings

* plots: extract parsing out of class method

* metrics/experiments: provide exception serializer for show-json

* params/plots/metrics: signal errors

* rebase fixups

* fix TODOs

* fixup

* skshetry's review refactor
2021-07-16 15:36:04 +02:00
Saugat Pachhai a172067c7d fix errors raised by pylint (#6081)
astroid==2.5.7 has improved inference and was throwing
errors that were false negatives before. Fixing these
and ignoring some.

Also added `funcy.decorator` to the `signature-mutators`,
as it changes function signature (i.e. provides `Call` as
a first argument which is not required to pass).

Example errors that we were getting because of this:

```
tests/remotes/gdrive.py:40:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
************* Module dvc.stage.serialize
dvc/stage/serialize.py:38:1: E1120: No value for argument 'func' in
function call (no-value-for-parameter)
************* Module dvc.fs.http
dvc/fs/http.py:19:1: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
************* Module dvc.fs.gdrive
dvc/fs/gdrive.py:66:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
dvc/fs/gdrive.py:459:5: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
```
2021-05-31 05:46:18 +00:00
Saugat Pachhai b9000ed926 pre-commit: update black to 21.5b1 (#6032)
* pre-commit: update black to 21.5b1

also updated others, but black is more noticeable

* remove magic trailing comma
2021-05-18 15:51:15 +00:00
Paweł RedzyńskiandJorge Orpinel c39f91ecf4 params: read from ParamsDependency if possible (#5550)
* params: read from ParamsDependency if possible

* params: introduce include_untracked flag

* params: include untracked upon targets

* experiments: add option to show tracked params

* fixing test

* params: rename tracked to deps

* params: roll back to default params.yaml behavior

* experiments: add option to show only used params on diff

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/params.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2021-04-12 17:08:02 +02:00
Peter Rowlands (변기호) a984f95cbd exp run: jobs should default to 1 (#5785) 2021-04-08 19:27:48 +09:00
Peter Rowlands (변기호) 1542025afa exp: bash tab completion (#5532)
* exp list: add --names-only option

* completion: add exp name completion

* dvc exp: use exp name completion

* exp: fix bash completion
2021-03-03 15:46:11 +09:00
Peter Rowlands (변기호) e3ee960e8e exp run: remove exp reset in favor of exp run --reset (#5442)
* exp run: remove `exp reset` in favor of `exp run --reset`

* update command unit test
2021-02-11 11:53:20 +09:00