Commit Graph
65 Commits
Author SHA1 Message Date
karajan1001 2182c4508d Make exp remove flags and arguments mutually exclusive.
1. Current behavior didn't follow the either or description.
2. Make the underlying logic more clear.
2022-08-05 16:09:51 +09:00
15ada9a1c3 Implement --date to exp gc
1. Implement `--time` to `exp gc`
2. add a unit test for it.
3. Rename `--time` to `--date` and change the warning msg.
4. To get a better msg for `gc` and `exp gc`

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: Dave Berenbaum <[email protected]>
2022-07-12 13:53:46 +02:00
dberenbaum 33b3afa2f5 exp init: create output dirs 2022-06-07 17:55:24 +05:45
Ruslan Kuprieiev 844f77b4f9 dvc: drop python 3.7
Also upgraded with `pyupgrade --py38-plus`

Fixes #7708
2022-05-30 16:51:35 +03:00
dberenbaum 477f266dc5 exp init: fixes #7534; simplifies/updates exp init --live 2022-05-09 20:58:27 +05:45
Peter Rowlands 444bf27e6f exp show: add internal --no-fetch option to disable temp/queue ref fetch 2022-04-21 15:45:59 +09:00
Saugat Pachhai (सौगात) 02b2ab6804 stage/exp list: alias --names-only to --name-only
Fixes #7570.
2022-04-21 06:53:16 +05:45
Saugat Pachhai (सौगात) 7d0d6a38d7 exp init: change ui prompts structure and output 2022-03-29 16:20:32 +05:45
David de la Iglesia Castro ddd6c97f71 render: Extract to dvc_render
Create and use separate package for rendering logic.
Closes #6944
2022-03-28 19:34:05 +02:00
karajan1001andJorge Orpinel efc021da73 exp remove: rename --all-commits flag and unify the collection of revs (#7155)
fix: #7155
1. rename flags `-A/--all-commits` in `exp remove`
2. add new flag `-n/--num` in `exp remove`
3. unify the revision collection in `exp remove`
4. add unit and func tests for `exp remove`

Co-authored-by: Jorge Orpinel <[email protected]>
2022-03-09 18:54:15 +08:00
karajan1001 244637e299 exp push/pull: add --all-commits flag and unify the collection of revs (#7154)
fix: #7154
1. rename flags `-A/--all-commits` in the `exp pull/push`
2. add new flag "--rev" and "--num" in the `exp pull/push`
3. Unify the collection of revs in `exp push/pull`
4. add unit and func tests for `exp push/pull`
2022-03-09 17:43:26 +08:00
karajan1001andJorge Orpinel d033345860 exp list: add the new flags (#7153)
fix: #7153
Unify the flags in `exp list` to `exp show`

1. Add new `-A`, `-n` flags to `exp list` command.
2. Add unit test for the flag parse.
3. unify the revs retrive logic to `exp show`.
4. Add functional test for `exp list`
5. Update the documents.

Co-authored-by: Jorge Orpinel <[email protected]>
2022-03-08 16:17:33 +08:00
Peter Rowlands d80c43135f exp run: remove unsupported (repro only) arguments 2022-03-07 12:01:51 +01:00
David de la Iglesia Castro d4861d535a TabularData: Simplify drop internals.
Update `subset` type hint.
2022-01-22 09:41:29 +08:00
David de la Iglesia Castro 43953ace53 exp show: Add --drop and --keep args
Removed `--include-metrics` / `--include-params` and
`--exclude-metrics` / `--exclude-params`.

Removed `--no-timestamp` . Can be done by `--drop Created`.

`--drop` and `--keep` operate directly on the table columns.

`--keep` does not perform any filtering. It's only used
to specify columns to keep despite the other filtering.

For example `--only-changed --keep foo` will prevent
`foo` from being removed by `--only-changed`.

Another example, `--drop train.* --keep train.dropout`
will remove all columns matching `train.*` except for
`train.dropout`.

Closes #7079
Closes #7080
Pre-requisite for #6434
2022-01-20 03:28:33 +01:00
Saugat Pachhai (सौगात) 446516d788 refactor: rename dvc.command to dvc.commands 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) 4268b5a339 cli: don't inherit DvcParserError from DvcException, move to dvc.cli 2022-01-19 19:41:40 +05:45
karajan1001 57263f15fa Pull: accept multiple as input(#7196)
fix: #7196

New feature: now `dvc exp pull` can accept a list of experiment names as input.

2. add support to list input for the `exp pull`
3. add function tests for the multi exp pull
4. add unit test for the arg parse
2022-01-18 17:31:55 +08:00
karajan1001 9bb6494e55 Push: accept multiple as input(#7196)
fix: #7196

New feature: now `dvc exp push` can accept a list of experiment names as input.

1. refactor of experiment push
2. add support to list input for the `exp push`
3. add function tests for the multi exp push
4. add unit test for the arg parse
2022-01-18 17:31:55 +08:00
Peter Rowlands dd5d999644 exp: add exp run --machine flag 2022-01-13 16:34:20 +09:00
karajan1001andPeter Rowlands b673ac24d0 exp show: add --rev flag (#7152)
fix: #7152
1. Add `--rev` flag to `dvc exp show`.
2. Add -1 support for `--num` flag`.
3. Extract revision found logic to `utils`.
4. Brancher use this revision found logic.
5. Update command unit test
6. Add a util unit test.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2022-01-11 19:01:34 +08:00
David de la Iglesia Castro 0a4958bafd TabularData: Decouple pcp from render. 2022-01-03 16:30:54 +01:00
David de la Iglesia Castro 5a9f861921 exp show: Rename html to pcp
Closes #7165
2022-01-03 16:30:54 +01:00
David de la Iglesia Castro 4d5be46ab4 exp show --html: Handle fill_value.
Adds new argument `fill_value` to ParallelCoordinatesRenderer.

For scalar columns, prevent converting to categorical by explictly inserting None when value is missing (value == fill_value).

For categorical columns, replace `fill_value` with `"Missing"` and ensure it is the last item so it gets rendered on top of the column.
2021-12-17 14:45:12 +09:00
David de la Iglesia Castro 5ee82d4000 exp show: Add parallel coordinates plot.
Uses `TabularData.to_parallel_coordinates`.
Adds new arguments: `html`, `out`, `open`

Reuses `--sort-by` to define colorscale.

Closes #4455
2021-12-17 14:45:12 +09:00
karajan1001andPeter Rowlands 0deab0118a exp: split exp sub command (#5489)
related to #5489
Current dvc/command/experiments.py is too long for switch between cmd class
and the argument parser.

Split the sub command in dvc/command/experiments.py to individual files

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-12-14 19:16:04 +08:00
pre-commit-ci[bot] 0bc101964e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2021-12-07 09:55:03 +05:45
Saugat Pachhai (सौगात) 3319808ed2 exp init: use shortlinks for doclinks 2021-12-07 09:55:03 +05:45
Saugat Pachhai (सौगात) cd37ebca37 exp init: minor refactoring 2021-12-07 08:40:46 +05:45
dberenbaum 4385720fab exp init: update tests 2021-12-07 07:34:31 +05:45
Saugat Pachhai (सौगात) 389fd48b5a exp init: change type to --type=live to --type=dl 2021-12-07 05:56:23 +05:45
Saugat Pachhai (सौगात) cb941a836d tests: cover exp init 2021-11-19 18:58:23 +05:45
David de la Iglesia Castro 28dd39a1a0 exp diff: Remove --old option.
After #6814 the header of `diff_table` is `a_rev/`b_rev` instead of `Old`/`New` so the name of the option was counterintuitive.

In addition, make `a_rev` show by default, for consistency with output format of other `diff_table` uses (`dvc metrics diff` / `dvc params diff`).
2021-11-12 12:58:12 +02:00
Paweł Redzyński 36ed182f26 argparse: dont override cmd
Related: #6956, https://bugs.python.org/issue45235
2021-11-11 01:14:03 +02:00
f964a44e7c Exp show md (#6799)
* Added `--md`/`--show-md` option to `exp show`.

* Added test_show_experiments_md

* Fixed re-defining all_experiments

* Fixed all_experiments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added dest markdown

* Added textwrap.dedent

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update experiments.py

* Apply suggestions from code review

Co-authored-by: Saugat Pachhai <[email protected]>

* Update tests/unit/command/test_compat_flag.py

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai <[email protected]>
2021-10-20 11:04:57 +05:45
David de la Iglesia Castro 7e4f443002 Added test_show_experiments_sort_by (#6798)
* Added test_show_experiments_sort_by

* Added comment

* help formatting
2021-10-14 14:49:00 +03:00
Saugat Pachhai 4e17279440 exp init: add config support (#6761) 2021-10-08 20:14:01 +05:45
Saugat Pachhai 9a1d74048a cli: add short alias for --show-{csv,json,md} (#6711)
As `--csv`/`--json`/`--md`
2021-09-30 20:38:32 +05:45
GaoandPeter Rowlands 6106b1341b Show csv format of experiments (#6468)
* Show csv format of experiments

fix #5989 and #5446 

1. add --show-csv to dvc exp show
2. add a new functional test for csv format
3. add a new unit test for show_experiements
4. solve name duplication in table common headers (Created,Parent, etc... ), metrics files and param files.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-09-08 16:59:01 +08:00
4cc06339c8 Clean up remotes's exps (#6471)
* Clean up remotes's exps

fix #6006
1. add a new argument `--git-remote` to `dvc exp remove`
2. support remote a special remote exp
3. add tests for it
4. fix  #6421
5. add a test for #6421.
6. extract some functions from the `dvc pull`, `dvc push`, `dvc remove` to utils.
7. add a tests for this new util function `resolve_exp_ref`
8. add `__init__.py` to some test package for the pytest fail

Co-authored-by: Jorge Orpinel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dave Berenbaum <[email protected]>
2021-09-05 10:06:07 +08:00
GaoandPeter Rowlands 57899ee86d Remove all of the experiments in the workspace (#6394)
* Remove all of the experiments in the workspace

fix #5676
1. remove all the experiments in workspace
2. add tests for it

* Only remove baseline ones

* Add a new arguments --all

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Rename and shortcut for these commands

* Solve the issue from removing a changing list.

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* delete workspace option

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-08-19 10:18:51 +09:00
Paweł Redzyński 21af7438db plots/metrics/params/experiments show: stop throwing exceptions (#5984)
* params/metrics/plots/exp show: introduce onerror

* fixup

* plots/metrics/params: removing errors

* removing more errors

* removing errors

* some more removing

* new plots ux

* new ux for metrics

* new ux for params

* fixing tests

* fixups

* minor test fixes

* minor test fixes

* commands: plots/params: fix ui imports

* working on error handlng decorator, stopped at show checkpoints

* error_handler, further dvelopment

* plots just before moving loading to collect

* appending errors to results: initial passing

* fixup

* onerror: remove onerror fixture

* fixups

* plots/metrics/params/experiments: temporarily remove error warnings

* plots: extract parsing out of class method

* metrics/experiments: provide exception serializer for show-json

* params/plots/metrics: signal errors

* rebase fixups

* fix TODOs

* fixup

* skshetry's review refactor
2021-07-16 15:36:04 +02:00
Saugat Pachhai a172067c7d fix errors raised by pylint (#6081)
astroid==2.5.7 has improved inference and was throwing
errors that were false negatives before. Fixing these
and ignoring some.

Also added `funcy.decorator` to the `signature-mutators`,
as it changes function signature (i.e. provides `Call` as
a first argument which is not required to pass).

Example errors that we were getting because of this:

```
tests/remotes/gdrive.py:40:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
************* Module dvc.stage.serialize
dvc/stage/serialize.py:38:1: E1120: No value for argument 'func' in
function call (no-value-for-parameter)
************* Module dvc.fs.http
dvc/fs/http.py:19:1: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
************* Module dvc.fs.gdrive
dvc/fs/gdrive.py:66:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
dvc/fs/gdrive.py:459:5: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
```
2021-05-31 05:46:18 +00:00
Saugat Pachhai b9000ed926 pre-commit: update black to 21.5b1 (#6032)
* pre-commit: update black to 21.5b1

also updated others, but black is more noticeable

* remove magic trailing comma
2021-05-18 15:51:15 +00:00
Paweł RedzyńskiandJorge Orpinel c39f91ecf4 params: read from ParamsDependency if possible (#5550)
* params: read from ParamsDependency if possible

* params: introduce include_untracked flag

* params: include untracked upon targets

* experiments: add option to show tracked params

* fixing test

* params: rename tracked to deps

* params: roll back to default params.yaml behavior

* experiments: add option to show only used params on diff

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/params.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2021-04-12 17:08:02 +02:00
Peter Rowlands (변기호) a984f95cbd exp run: jobs should default to 1 (#5785) 2021-04-08 19:27:48 +09:00
Peter Rowlands (변기호) 1542025afa exp: bash tab completion (#5532)
* exp list: add --names-only option

* completion: add exp name completion

* dvc exp: use exp name completion

* exp: fix bash completion
2021-03-03 15:46:11 +09:00
Peter Rowlands (변기호) e3ee960e8e exp run: remove exp reset in favor of exp run --reset (#5442)
* exp run: remove `exp reset` in favor of `exp run --reset`

* update command unit test
2021-02-11 11:53:20 +09:00
Peter Rowlands (변기호) 9d82739b3a exp: add exp remove command (#5443)
* exp: add remove function for removing specific experiment(s)

* add `dvc exp remove` command

* update tests
2021-02-11 11:45:25 +09:00
Peter Rowlands (변기호) d05be36694 exp show: cleanup UI issues (#5361)
* exp show: add -n/--num to display the last N commits from HEAD

* exp run: fix empty commit with no changes

* add test for exp show with multiple commits

* allow -n greater than num commits in repo

* review fix
2021-02-02 11:35:02 +09:00