Commit Graph
211 Commits
Author SHA1 Message Date
David de la Iglesia Castro d8e2c14927 config: Add plots.auto_open.
Option for enabling automatically open webbrowser with generated html.
Defaults False.

Closes #6091
2022-01-03 16:07:23 +01:00
David de la Iglesia Castro 4d5be46ab4 exp show --html: Handle fill_value.
Adds new argument `fill_value` to ParallelCoordinatesRenderer.

For scalar columns, prevent converting to categorical by explictly inserting None when value is missing (value == fill_value).

For categorical columns, replace `fill_value` with `"Missing"` and ensure it is the last item so it gets rendered on top of the column.
2021-12-17 14:45:12 +09:00
David de la Iglesia Castro 5ee82d4000 exp show: Add parallel coordinates plot.
Uses `TabularData.to_parallel_coordinates`.
Adds new arguments: `html`, `out`, `open`

Reuses `--sort-by` to define colorscale.

Closes #4455
2021-12-17 14:45:12 +09:00
Paweł Redzyński e24de3a0f3 plots: introduce --json option
Fixes: #7073
2021-12-15 11:05:57 +01:00
karajan1001andPeter Rowlands 0deab0118a exp: split exp sub command (#5489)
related to #5489
Current dvc/command/experiments.py is too long for switch between cmd class
and the argument parser.

Split the sub command in dvc/command/experiments.py to individual files

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-12-14 19:16:04 +08:00
pre-commit-ci[bot] 0bc101964e [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2021-12-07 09:55:03 +05:45
Saugat Pachhai (सौगात) 3319808ed2 exp init: use shortlinks for doclinks 2021-12-07 09:55:03 +05:45
Saugat Pachhai (सौगात) cd37ebca37 exp init: minor refactoring 2021-12-07 08:40:46 +05:45
dberenbaum 4385720fab exp init: update tests 2021-12-07 07:34:31 +05:45
Saugat Pachhai (सौगात) 389fd48b5a exp init: change type to --type=live to --type=dl 2021-12-07 05:56:23 +05:45
David de la Iglesia Castro 728731d4d7 ui: Add open_browser.
Use `webbrowser.open` and relpath for WSL plattform.
2021-12-05 15:44:13 +02:00
Saugat Pachhai (सौगात) cb941a836d tests: cover exp init 2021-11-19 18:58:23 +05:45
karajan1001 b24a31b834 Machine: use table for list outputs(#6892)
fix: #6892

Current output of machine list is quite urgly, it is originaly used to
show configurations. In this PR, we use a new table format for it.
2021-11-18 11:22:13 +08:00
karajan1001 eea67a6396 machine:remove help text for irrelevant commands (#6940)
fix: #6940

`--global/system/project/local` help message are in some irrelevant
command `status`, `ssh`, `create`, `destroy`. These commands are more
related to running instances rather than `machine config`, level
options didn't work for them.

1. Remove `--global/system/project/local` options in `status`, `ssh`,
   `create`, `destroy`.
2. Add a test for it.
3. Correct a typo error.
2021-11-17 16:25:48 +08:00
karajan1001andDave Berenbaum f43c04f935 Make machine status name optional
fix: #6838
1. make machine status name optional.
2. show all machine status if no name provide.
3. UI changes for this cmd.
4. add a new test for it.

Co-authored-by: Dave Berenbaum <[email protected]>
2021-11-17 15:44:36 +08:00
David de la Iglesia Castro 28dd39a1a0 exp diff: Remove --old option.
After #6814 the header of `diff_table` is `a_rev/`b_rev` instead of `Old`/`New` so the name of the option was counterintuitive.

In addition, make `a_rev` show by default, for consistency with output format of other `diff_table` uses (`dvc metrics diff` / `dvc params diff`).
2021-11-12 12:58:12 +02:00
Paweł Redzyński 36ed182f26 argparse: dont override cmd
Related: #6956, https://bugs.python.org/issue45235
2021-11-11 01:14:03 +02:00
David de la Iglesia Castro 4dd544121f plots: Use URI for --open.
We were previously using relpath in *all* cases, mainly to support WSL but relpath doesn't work
on macOS.

Added a check for using relpath *only* for WSL.

Fixes #6092
2021-10-29 13:27:12 +02:00
Gao 4b2bec89ec dvc machine: add a new command rename to it (#6633)
* `dvc machine`: add a new command `rename` to it

fix #6481
1. add new command `dvc machine rename`
2. add a unit test for the call.
3. add several functional tests for it.t push myself fix6481

* Move rename logic to tpi

* use tpi.state_mv instead of rename

* Rebase to a newer version

* Update tpi version

* Remove unused assert and try catch
2021-10-20 15:57:44 +09:00
f964a44e7c Exp show md (#6799)
* Added `--md`/`--show-md` option to `exp show`.

* Added test_show_experiments_md

* Fixed re-defining all_experiments

* Fixed all_experiments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added dest markdown

* Added textwrap.dedent

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update experiments.py

* Apply suggestions from code review

Co-authored-by: Saugat Pachhai <[email protected]>

* Update tests/unit/command/test_compat_flag.py

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai <[email protected]>
2021-10-20 11:04:57 +05:45
966dc8a1ca machine: Add a new command dvc machine status (#6649)
* machine: Add a new command `dvc machine status`

fix#6482
1. add a new command `dvc machien status`
2. add a unit test for the cli call.
3. add a unit test for the shown result.

* Update dvc/command/machine.py

Co-authored-by: Saugat Pachhai <[email protected]>

* Update dvc/command/machine.py

Co-authored-by: Saugat Pachhai <[email protected]>

* Update machine.py

* Remove sensitive fields

* Update dvc/machine/__init__.py

Co-authored-by: Casper da Costa-Luis <[email protected]>

* Update tests/func/machine/__init__.py

Co-authored-by: Casper da Costa-Luis <[email protected]>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Use white list to limit shown fields

Co-authored-by: Saugat Pachhai <[email protected]>
Co-authored-by: Casper da Costa-Luis <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-10-19 17:38:11 +09:00
David de la Iglesia Castro 4f16852005 exp/metrics/params diff: Replaced Old/New with a_rev/b_rev (#6814)
* Replaced Old/New with a_rev/b_rev

* Fixed tests
2021-10-18 21:09:02 +03:00
David de la Iglesia Castro 7e4f443002 Added test_show_experiments_sort_by (#6798)
* Added test_show_experiments_sort_by

* Added comment

* help formatting
2021-10-14 14:49:00 +03:00
Saugat Pachhai 4e17279440 exp init: add config support (#6761) 2021-10-08 20:14:01 +05:45
Saugat Pachhai 9a1d74048a cli: add short alias for --show-{csv,json,md} (#6711)
As `--csv`/`--json`/`--md`
2021-09-30 20:38:32 +05:45
Amey Kulkarni fbd8f5c185 Issue 6651: Added alias ls for dvc list (#6683) 2021-09-25 21:14:11 +03:00
Nikita Kodenko 696c704632 tests: replace several unittest.mock usages with mocker (#6544)
* tests: several unittest.mock usages replaced with mocker

* tests: fixes in tests/unit/repo/test_reproduce.py

* tests: rolled back WiP tests
2021-09-17 18:22:34 +03:00
GaoandPeter Rowlands 18d147e2f8 machine: add machine list&machine modify (#6602)
* machine: add `machine list`&`machine modify`

1. add new command `dvc machine list`
2. complete `dvc machine modify`
3. add new unit tests for this two commands to ensure the call.
4. add functionality tests for two commands

* Update dvc/config_schema.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Update tests/func/machine/test_machine_config.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Some reviewed problems

1. capitalize global variables.
2. remove precision restriction.
3. solve windows path seperator problem.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-09-15 15:40:16 +09:00
GaoandPeter Rowlands 6106b1341b Show csv format of experiments (#6468)
* Show csv format of experiments

fix #5989 and #5446 

1. add --show-csv to dvc exp show
2. add a new functional test for csv format
3. add a new unit test for show_experiements
4. solve name duplication in table common headers (Created,Parent, etc... ), metrics files and param files.

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-09-08 16:59:01 +08:00
4cc06339c8 Clean up remotes's exps (#6471)
* Clean up remotes's exps

fix #6006
1. add a new argument `--git-remote` to `dvc exp remove`
2. support remote a special remote exp
3. add tests for it
4. fix  #6421
5. add a test for #6421.
6. extract some functions from the `dvc pull`, `dvc push`, `dvc remove` to utils.
7. add a tests for this new util function `resolve_exp_ref`
8. add `__init__.py` to some test package for the pytest fail

Co-authored-by: Jorge Orpinel <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Dave Berenbaum <[email protected]>
2021-09-05 10:06:07 +08:00
Peter Rowlands (변기호) 9fc4c60749 machine: add dvc machine ssh (#6501)
* machine: add naive asyncssh shell client

* machine: add `dvc machine ssh` command

* add ssh requirements to terraform install extras

* update asyncssh comments

* fix py <3.7

* fix async mocking in py<3.8

* use mock instead of unittest.mock
2021-09-01 16:20:13 +09:00
Peter Rowlands (변기호) 4ee98299ee dvc machine: implement initial terraform support (#6462)
* tpi: add naive terraform-provider-iterative wrapper

* executor: add naive backend/terraform wrapper

* setup: add dvc[terraform] install extras

* config: add feature.executor feature flag

* fix named executor config sections

* executor: add poc add/modify/remove/init/destroy commands

* relax python-terraform requirement

* ci: install terraform extras

* tpi: support .tf.json format

* s/executor/machine

* terraform: use separate dirs per machine

* s/init/create/

* reserve local machine name

* add initial ssh skeleton

* add initial machine unit tests

* add initial machine command tests

* add tests for add/remove

* enable feature by default in test env

* close temp pemfile object so it can be used on windows

* cleanup pemfile syntax
2021-08-27 12:46:24 +03:00
Paweł Redzyński ab08db5065 plots: introduce support for images (#6431)
* plots: support images initial

* plots: data: unify PlotsData

* plots: data conversion refactor

* plots: renderers: abstract functionality

* plots: uncommenting tests

* plots: self implement grouping

* plots: rename group

* plots: extract rendering tests

* plots: move rendering tests to unit

* plots: add image rendering tests

* plots: dont use dpath to find vega plot

* plots: test find_vega

* plots: test render function

* plots: move rendering out of repo package

* plots: refactoring

* plots: self review refactor
2021-08-26 20:00:31 +03:00
GaoandPeter Rowlands 57899ee86d Remove all of the experiments in the workspace (#6394)
* Remove all of the experiments in the workspace

fix #5676
1. remove all the experiments in workspace
2. add tests for it

* Only remove baseline ones

* Add a new arguments --all

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Rename and shortcut for these commands

* Solve the issue from removing a changing list.

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Update dvc/repo/experiments/remove.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* delete workspace option

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-08-19 10:18:51 +09:00
Saugat Pachhai 6c33fbb617 introduce index (#6300)
* introduce index

* migrate tests to use index

* migrate checkout/diff to use index.outs

* test for index

* fix typings

* add few more tests
2021-08-05 14:23:40 +03:00
Christian Clauss d8b67f6f78 Upgrade to modern Python syntax (#6296) 2021-07-19 01:30:37 +03:00
Paweł Redzyński 21af7438db plots/metrics/params/experiments show: stop throwing exceptions (#5984)
* params/metrics/plots/exp show: introduce onerror

* fixup

* plots/metrics/params: removing errors

* removing more errors

* removing errors

* some more removing

* new plots ux

* new ux for metrics

* new ux for params

* fixing tests

* fixups

* minor test fixes

* minor test fixes

* commands: plots/params: fix ui imports

* working on error handlng decorator, stopped at show checkpoints

* error_handler, further dvelopment

* plots just before moving loading to collect

* appending errors to results: initial passing

* fixup

* onerror: remove onerror fixture

* fixups

* plots/metrics/params/experiments: temporarily remove error warnings

* plots: extract parsing out of class method

* metrics/experiments: provide exception serializer for show-json

* params/plots/metrics: signal errors

* rebase fixups

* fix TODOs

* fixup

* skshetry's review refactor
2021-07-16 15:36:04 +02:00
Hadas Arikandpre-commit-ci[bot] 9d9ff10be1 diff: only show states with values in summary (#6229)
* Fixed diff summary to only show states with values.

* Removed redundant comma by using join.

* Swithced for loop to generator for better readability.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-07-06 21:03:32 +09:00
Saugat Pachhai 476af100de ui: migrating rest to the ui.write (#6176)
* complete migration to ui.write

* fix small bugs
2021-06-21 22:14:59 +03:00
Saugat Pachhai a172067c7d fix errors raised by pylint (#6081)
astroid==2.5.7 has improved inference and was throwing
errors that were false negatives before. Fixing these
and ignoring some.

Also added `funcy.decorator` to the `signature-mutators`,
as it changes function signature (i.e. provides `Call` as
a first argument which is not required to pass).

Example errors that we were getting because of this:

```
tests/remotes/gdrive.py:40:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
************* Module dvc.stage.serialize
dvc/stage/serialize.py:38:1: E1120: No value for argument 'func' in
function call (no-value-for-parameter)
************* Module dvc.fs.http
dvc/fs/http.py:19:1: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
************* Module dvc.fs.gdrive
dvc/fs/gdrive.py:66:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
dvc/fs/gdrive.py:459:5: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
```
2021-05-31 05:46:18 +00:00
Saugat Pachhai 7700d55529 use kwargs when passing command line args to run (#6075) 2021-05-29 21:04:55 +05:45
Saugat Pachhai a74855b9f7 fix empty string parsing for the command (#6074)
Fixes #6072
2021-05-28 06:51:27 +00:00
Saugat Pachhai 63dba45c14 ui: write gets a styled param for rich console printing (#6037) 2021-05-28 09:30:56 +05:45
Saugat Pachhai b9000ed926 pre-commit: update black to 21.5b1 (#6032)
* pre-commit: update black to 21.5b1

also updated others, but black is more noticeable

* remove magic trailing comma
2021-05-18 15:51:15 +00:00
Saugat Pachhai 789faa4f11 ui: migrate checkout to ui.write (#6028) 2021-05-18 13:55:11 +02:00
Saugat Pachhai e6a2b8a2ad ui: migrate cache:dir and check-ignore to ui.write (#6021) 2021-05-17 18:50:58 +03:00
Saugat Pachhai c6f49bd23d ls: use ui.write (#6004) 2021-05-14 10:19:55 +05:45
Saugat Pachhai d6afd26bea metrics diff/show: rollback to rounding fractional digits with precision (#5904) 2021-05-02 16:56:00 +03:00
Paweł Redzyński d509d77193 list: add --show-json option (#5888)
* list: add --show-json option

* list: add test for --show-json
2021-04-30 11:40:44 +02:00
Saugat Pachhai 42cf394fde diff: use ui.table for --show-md (#5887) 2021-04-27 17:10:08 +05:45