Commit Graph
2528 Commits
Author SHA1 Message Date
Saugat Pachhai (सौगात) e80e725fe4 tests: fix test failures with rich==12 2022-03-11 19:26:54 +05:45
karajan1001andJorge Orpinel efc021da73 exp remove: rename --all-commits flag and unify the collection of revs (#7155)
fix: #7155
1. rename flags `-A/--all-commits` in `exp remove`
2. add new flag `-n/--num` in `exp remove`
3. unify the revision collection in `exp remove`
4. add unit and func tests for `exp remove`

Co-authored-by: Jorge Orpinel <[email protected]>
2022-03-09 18:54:15 +08:00
karajan1001 244637e299 exp push/pull: add --all-commits flag and unify the collection of revs (#7154)
fix: #7154
1. rename flags `-A/--all-commits` in the `exp pull/push`
2. add new flag "--rev" and "--num" in the `exp pull/push`
3. Unify the collection of revs in `exp push/pull`
4. add unit and func tests for `exp push/pull`
2022-03-09 17:43:26 +08:00
karajan1001andJorge Orpinel d033345860 exp list: add the new flags (#7153)
fix: #7153
Unify the flags in `exp list` to `exp show`

1. Add new `-A`, `-n` flags to `exp list` command.
2. Add unit test for the flag parse.
3. unify the revs retrive logic to `exp show`.
4. Add functional test for `exp list`
5. Update the documents.

Co-authored-by: Jorge Orpinel <[email protected]>
2022-03-08 16:17:33 +08:00
Peter Rowlands d80c43135f exp run: remove unsupported (repro only) arguments 2022-03-07 12:01:51 +01:00
David de la Iglesia Castro 5249207ceb live: Remove summary from tests 2022-03-07 10:09:04 +01:00
dberenbaum fc3f3ca8d7 diff: fix missing key bug (#6720) 2022-03-02 19:10:31 +01:00
Peter Rowlands 59b8498905 fs: remove unused connection pool module 2022-03-01 15:36:49 +09:00
Zhenyu Gao a378e017e1 Fix dvc add -o issue #7357 2022-02-28 15:24:16 +01:00
Paweł Redzyński 3b4d76df42 plots: json: add datapoints to vega plots 2022-02-25 17:12:45 +01:00
Saugat Pachhai (सौगात) fcddf080da exp init: refactor and simplify interactive mode
- Moves workspace tree for interactive mode after prompts. Previously
  this was displayed before the prompts.
- Remove the yaml contents and the confirmation prompt in an interactive
  mode.
- Simplify `init_interactive()`.
2022-02-22 16:59:08 +05:45
Ruslan Kuprieiev 82b0a3088f dvcfs: handle repo root
Fixes #7407
2022-02-21 04:23:43 +02:00
Ruslan Kuprieiev 30fa3fc41b fs: walk: get rid of onerror
Makes dvc/repofs and other filesystems comply with fsspec.
2022-02-18 16:17:46 +02:00
Daniele Trifirò 3767b03784 test_data_cloud: also check for the correct number of jobs 2022-02-18 02:37:48 +02:00
Daniele Trifirò c0f63efe99 fix passing of jobs kwarg
the `jobs` kwarg was not being passed down properly,
resulting in it being ignored in some cases (push/pull/gc)
2022-02-18 02:37:48 +02:00
Saugat Pachhai (सौगात) 497eac952c s/Creating/Created 2022-02-17 14:18:08 +05:45
Saugat Pachhai (सौगात) 6eb5c31dd7 fix UI message 2022-02-17 14:18:08 +05:45
Saugat Pachhai (सौगात) 69ce69e427 exp init: update message 2022-02-17 14:18:08 +05:45
Saugat Pachhai (सौगात) f8467cf375 exp init: log created dependencies in non-interactive mode 2022-02-17 14:18:08 +05:45
Saugat Pachhai (सौगात) 2735fbac71 exp init: create workspace structure by default, show workspace tree 2022-02-17 14:18:08 +05:45
karajan1001 d1f7cf4540 exp show: show exp when checkpoint is running(#7329)
fix: #7329
In a previous PR, we introduce a bug which use `HEAD` as the workspace
commit, while if there is a checkpoint is running, the `HEAD` is
actually changing all the time, we should use
`refs/exps/exec/EXEC_BASELINE` instead.

1. add `fix_exp_head` transfer in `dvc/scm.py::resolve_rev`
2. remove fix_exp_head from `diff`, `metrics diff`, `params diff` and
   `plots diff`
3. modify a functional test to simulate the running conditions
2022-02-15 21:17:42 +08:00
Daniele Trifirò 4db9d19387 destroy: also remove dvcignore file 2022-02-10 16:16:12 +01:00
David de la Iglesia Castro 31a3461465 live: Update tests 2022-02-10 16:46:32 +05:45
David de la Iglesia Castro d89555cd31 exp show: handle renaming of metrics/params.
_extend_row did not considered partially empty `items`, which can occur when metris or params are renamed between commits.

Fixes #7070
2022-02-09 21:40:15 +01:00
Nikita Kodenko b6fa1178e7 tests: migrate unittest to pytest 2022-02-08 18:51:17 +09:00
Ruslan Kuprieiev e430693fc8 dvcfs: use paths relative to the repo root
Currently we use both absolute and relative (depend on cwd) paths, which is
terrible for an api, as their interpretation depends on external factors.
This PR makes dvcfs accept only paths relative to the repo root for regular
outputs and absolute paths for external outputs.

Pre-requisite for making repofs use same relative paths.
2022-02-07 20:12:33 +02:00
Ruslan Kuprieiev cb584c6a53 dvcfs: detach from pipeline outputs
Introducing simple repo tree that combines all workspaces into one structure,
and allows us to detach data representation from pipelines.

Pre-requisite for dvc-data extraction and using relpaths in dvcfs/repofs.
2022-02-07 13:12:52 +02:00
Ruslan Kuprieiev 8497edab93 dvcfs: use info() instead of metadata()
metadata was introduced before our migration to fsspec and no longer needed
now. This is a pre-requisite to using data index in dvcfs/repofs.

Note that a bunch of awkward flags like isdvc/isout/etc are still kept, and
will be removed in followups.
2022-02-03 22:45:10 +02:00
David de la Iglesia Castro 37cfb8e8af exp show: Sort deps columns.
Closes #7312
2022-02-01 15:23:26 +01:00
Paweł Redzyński 7bc14a5ddb plots: make templates generation explicit
Fixes: #6389
2022-02-01 15:09:25 +01:00
pre-commit-ci[bot] d213ccb686 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2022-02-01 19:53:13 +05:45
Daniele Trifirò 0562b4fd5f tests: update test_exceptions for rich 11.1.0 behaviour 2022-02-01 13:20:07 +01:00
Ruslan Kuprieiev aca6d229da output: move isexec to meta
With our staging process automatically capturing file metadata, we no longer
need to set isexec ourselves.

In followup PRs we will also use meta during checkout, so that we won't need
to use Output.set_exec by hand.
2022-01-27 18:55:32 +02:00
David de la Iglesia Castro 8c2f8095da debug: Remove test_viztracer_internal 2022-01-27 09:36:06 +05:45
David de la Iglesia Castro 5e80fcce4f exp show: Include outs in --json output. 2022-01-26 18:45:48 +01:00
David de la Iglesia Castro 1a1e28282d exp show: Include deps columns.
Use `repo.index.deps` to collect dependencies associated with each experiment.
2022-01-26 18:45:48 +01:00
David de la Iglesia Castro 3276c76a02 tests: Add test_debug 2022-01-25 20:28:41 +05:45
Saugat Pachhai (सौगात) 2f15f30b0e exp run: do not log message twice on workspace executor
Fixes #7276.
2022-01-25 19:01:48 +05:45
Peter Rowlands 47d10c4a8b utils: add decorator to wrap diskcache pickle errors 2022-01-25 16:21:53 +09:00
David de la Iglesia Castro d4861d535a TabularData: Simplify drop internals.
Update `subset` type hint.
2022-01-22 09:41:29 +08:00
David de la Iglesia Castro 43953ace53 exp show: Add --drop and --keep args
Removed `--include-metrics` / `--include-params` and
`--exclude-metrics` / `--exclude-params`.

Removed `--no-timestamp` . Can be done by `--drop Created`.

`--drop` and `--keep` operate directly on the table columns.

`--keep` does not perform any filtering. It's only used
to specify columns to keep despite the other filtering.

For example `--only-changed --keep foo` will prevent
`foo` from being removed by `--only-changed`.

Another example, `--drop train.* --keep train.dropout`
will remove all columns matching `train.*` except for
`train.dropout`.

Closes #7079
Closes #7080
Pre-requisite for #6434
2022-01-20 03:28:33 +01:00
David de la Iglesia Castro 255e6a1aec TabularData: Support protected`columns.
TabularData: Add subset arg to dropna and drop_duplicates.

Closes #7083
2022-01-20 03:28:33 +01:00
Saugat Pachhai (सौगात) b0f65cbd17 tests: increase timeout for exp init tests 2022-01-19 20:26:41 +05:45
Saugat Pachhai (सौगात) 672f079cc6 tests: clean repositories before each test runs 2022-01-19 20:21:48 +05:45
Saugat Pachhai (सौगात) 446516d788 refactor: rename dvc.command to dvc.commands 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) 1b3e1251e6 cli refactor: move dvc.command.base to dvc.cli.command 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) 4268b5a339 cli: don't inherit DvcParserError from DvcException, move to dvc.cli 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) f5baf990cf cli: refactor into a module
By splitting dvc.cli into a module, we'll be able to share cli-related
utilities, split parser-related stuffs into smaller parts, improve
reusability and improve API for command extensions in the future.
2022-01-19 12:22:07 +02:00
Saugat Pachhai (सौगात) 4ba504be3e repro/exp run: display stage instead of filepath on failure
Before it used to say 'failed to reproduce dvc.yaml', which is now changed
to print 'failed to reproduce train' or something similar.

For stages that are in dvc.yaml in the current working directory, it will
just print the stage names. For stages in other dvc.yaml files, it'll print

in <relative_path>:<stage_name>.
2022-01-19 06:58:40 +05:45
karajan1001 57263f15fa Pull: accept multiple as input(#7196)
fix: #7196

New feature: now `dvc exp pull` can accept a list of experiment names as input.

2. add support to list input for the `exp pull`
3. add function tests for the multi exp pull
4. add unit test for the arg parse
2022-01-18 17:31:55 +08:00