Commit Graph
7271 Commits
Author SHA1 Message Date
Daniele Trifirò 0562b4fd5f tests: update test_exceptions for rich 11.1.0 behaviour 2022-02-01 13:20:07 +01:00
tapadipti 95731e9a37 Doc URL correction for GitAuthError 2022-01-31 14:50:37 +05:45
Ruslan Kuprieiev aca6d229da output: move isexec to meta
With our staging process automatically capturing file metadata, we no longer
need to set isexec ourselves.

In followup PRs we will also use meta during checkout, so that we won't need
to use Output.set_exec by hand.
2022-01-27 18:55:32 +02:00
dependabot[bot] 60e502d361 build(deps-dev): Bump google-cloud-storage from 2.0.0 to 2.1.0
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 2.0.0 to 2.1.0.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v2.0.0...v2.1.0)

---
updated-dependencies:
- dependency-name: google-cloud-storage
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-01-27 14:48:46 +05:45
David de la Iglesia Castro 8c2f8095da debug: Remove test_viztracer_internal 2022-01-27 09:36:06 +05:45
David de la Iglesia Castro 5e80fcce4f exp show: Include outs in --json output. 2022-01-26 18:45:48 +01:00
David de la Iglesia Castro 1a1e28282d exp show: Include deps columns.
Use `repo.index.deps` to collect dependencies associated with each experiment.
2022-01-26 18:45:48 +01:00
Daniele Trifirò e3b97cb3a7 ignore: move debug statetement to the proper location 2022-01-26 18:44:01 +02:00
David de la Iglesia Castro d882e21b9b debug: Remove viztracer-depth default 2022-01-25 20:28:41 +05:45
David de la Iglesia Castro 3276c76a02 tests: Add test_debug 2022-01-25 20:28:41 +05:45
Saugat Pachhai 758dc43575 Update dvc/_debug.py 2022-01-25 20:28:41 +05:45
David de la Iglesia Castro f1dc9e50a7 debug: Add support for viztracer profiler. 2022-01-25 20:28:41 +05:45
Saugat Pachhai (सौगात) 2f15f30b0e exp run: do not log message twice on workspace executor
Fixes #7276.
2022-01-25 19:01:48 +05:45
Peter Rowlands 47d10c4a8b utils: add decorator to wrap diskcache pickle errors 2022-01-25 16:21:53 +09:00
David de la Iglesia Castro 822b501069 add: Update help of --jobs 2022-01-24 21:06:47 +01:00
Saugat Pachhai (सौगात) 943481f0a3 exp init: handle exception properly on missing parameters file 2022-01-24 17:43:29 +05:45
Jeff Hale b30385885b Fix 3 README links so that they render in GitHub 2022-01-24 15:55:12 +09:00
David de la Iglesia Castro d4861d535a TabularData: Simplify drop internals.
Update `subset` type hint.
2022-01-22 09:41:29 +08:00
dberenbaum 646097a735 update exp init help text 2022-01-21 11:07:32 +05:45
David de la Iglesia Castro 43953ace53 exp show: Add --drop and --keep args
Removed `--include-metrics` / `--include-params` and
`--exclude-metrics` / `--exclude-params`.

Removed `--no-timestamp` . Can be done by `--drop Created`.

`--drop` and `--keep` operate directly on the table columns.

`--keep` does not perform any filtering. It's only used
to specify columns to keep despite the other filtering.

For example `--only-changed --keep foo` will prevent
`foo` from being removed by `--only-changed`.

Another example, `--drop train.* --keep train.dropout`
will remove all columns matching `train.*` except for
`train.dropout`.

Closes #7079
Closes #7080
Pre-requisite for #6434
2022-01-20 03:28:33 +01:00
David de la Iglesia Castro 255e6a1aec TabularData: Support protected`columns.
TabularData: Add subset arg to dropna and drop_duplicates.

Closes #7083
2022-01-20 03:28:33 +01:00
Saugat Pachhai (सौगात) 73fee513b9 cli: improve startup speed
fsspec callbacks were making us lose 35ms (~207 ms vs 170ms now).
These are now extracted from dvc.progress to dvc.fs._callback.
2022-01-19 20:38:19 +02:00
Saugat Pachhai (सौगात) b0f65cbd17 tests: increase timeout for exp init tests 2022-01-19 20:26:41 +05:45
Saugat Pachhai (सौगात) 672f079cc6 tests: clean repositories before each test runs 2022-01-19 20:21:48 +05:45
Saugat Pachhai (सौगात) 446516d788 refactor: rename dvc.command to dvc.commands 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) 1b3e1251e6 cli refactor: move dvc.command.base to dvc.cli.command 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) 4268b5a339 cli: don't inherit DvcParserError from DvcException, move to dvc.cli 2022-01-19 19:41:40 +05:45
Saugat Pachhai (सौगात) f5baf990cf cli: refactor into a module
By splitting dvc.cli into a module, we'll be able to share cli-related
utilities, split parser-related stuffs into smaller parts, improve
reusability and improve API for command extensions in the future.
2022-01-19 12:22:07 +02:00
dependabot[bot] 69ff760e4b build(deps-dev): Bump pytest-timeout from 2.0.2 to 2.1.0
Bumps [pytest-timeout](https://github.com/pytest-dev/pytest-timeout) from 2.0.2 to 2.1.0.
- [Release notes](https://github.com/pytest-dev/pytest-timeout/releases)
- [Commits](https://github.com/pytest-dev/pytest-timeout/compare/2.0.2...2.1.0)

---
updated-dependencies:
- dependency-name: pytest-timeout
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-01-19 10:57:17 +05:45
Saugat Pachhai (सौगात) 4ba504be3e repro/exp run: display stage instead of filepath on failure
Before it used to say 'failed to reproduce dvc.yaml', which is now changed
to print 'failed to reproduce train' or something similar.

For stages that are in dvc.yaml in the current working directory, it will
just print the stage names. For stages in other dvc.yaml files, it'll print

in <relative_path>:<stage_name>.
2022-01-19 06:58:40 +05:45
Ruslan Kuprieiev 377b0bc689 find_root: unify error message
We used to use that message for special external repo cases, but we don't
need it anymore.

Fixes #7281

Leftover from #7214
2022-01-19 01:39:48 +02:00
Matt Seddon 89f2d42df8 plots: update linear template so colors can be externally overwritten 2022-01-18 12:42:17 +01:00
karajan1001 57263f15fa Pull: accept multiple as input(#7196)
fix: #7196

New feature: now `dvc exp pull` can accept a list of experiment names as input.

2. add support to list input for the `exp pull`
3. add function tests for the multi exp pull
4. add unit test for the arg parse
2022-01-18 17:31:55 +08:00
karajan1001 9bb6494e55 Push: accept multiple as input(#7196)
fix: #7196

New feature: now `dvc exp push` can accept a list of experiment names as input.

1. refactor of experiment push
2. add support to list input for the `exp push`
3. add function tests for the multi exp push
4. add unit test for the arg parse
2022-01-18 17:31:55 +08:00
Ruslan Kuprieiev 13238e9716 objects: fix imports 2022-01-15 21:39:39 +02:00
Ruslan Kuprieiev 85181e396f objects: fix empty line 2022-01-15 21:20:09 +02:00
Ruslan Kuprieiev 4c3c492b90 objects: don't forget __init__.py 2022-01-15 21:16:20 +02:00
dependabot[bot] 474812195e build(deps-dev): Bump wsgidav from 3.1.1 to 4.0.1
Bumps [wsgidav](https://github.com/mar10/wsgidav) from 3.1.1 to 4.0.1.
- [Release notes](https://github.com/mar10/wsgidav/releases)
- [Changelog](https://github.com/mar10/wsgidav/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mar10/wsgidav/compare/v3.1.1...v4.0.1)

---
updated-dependencies:
- dependency-name: wsgidav
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-01-14 09:12:16 +05:45
Saugat Pachhai c83612be71 deps-dev: don't specify pygments explicitly
pygments is a transitive dependency for us, from `rich` and here it was for `collective.checkdocs`.
2022-01-14 09:11:20 +05:45
dependabot[bot] 161e7892ba build(deps-dev): Bump google-cloud-storage from 1.43.0 to 2.0.0
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 1.43.0 to 2.0.0.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v1.43.0...v2.0.0)

---
updated-dependencies:
- dependency-name: google-cloud-storage
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-01-14 09:02:38 +05:45
dependabot[bot] 0a0e3c3f9c build(deps-dev): Bump types-requests from 2.27.5 to 2.27.7
Bumps [types-requests](https://github.com/python/typeshed) from 2.27.5 to 2.27.7.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-01-14 09:02:25 +05:45
dependabot[bot] ac2feb2340 build(deps): Bump pyinstaller from 4.7 to 4.8
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 4.7 to 4.8.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/develop/doc/CHANGES-3.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v4.7...v4.8)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2022-01-13 17:00:03 +09:00
Peter Rowlands dd5d999644 exp: add exp run --machine flag 2022-01-13 16:34:20 +09:00
Peter Rowlands 3c7695b39d exp: add SSHExecutor.reproduce via exp exec-run 2022-01-13 16:34:20 +09:00
Peter Rowlands 367e5d1429 exp: add exec-run plumbing command to run exp from ExecutorInfo 2022-01-13 16:34:20 +09:00
Peter Rowlands fb16cd06e3 machine: update behavior for exp ssh executors
- log startup script finished message
- add setup_script option for runtime env setup
2022-01-13 16:34:20 +09:00
Peter Rowlands 39eee9b5a6 exp: implement git + DVC cache collection for ssh executors 2022-01-13 16:34:20 +09:00
Peter Rowlands 84da78a62b exp: initial SSHExecutorManager implementation 2022-01-13 16:34:20 +09:00
Peter Rowlands 4b9d39c901 push/fetch: support push/fetch to specific ODB
- prerequisite for remote execution
- allow explicitly pushing imports which would normally be skipped
  (internal-use only, does not apply to `dvc push`)
2022-01-13 16:34:20 +09:00
David de la Iglesia Castro 5b8bf52eab plots: Fix --json requiring -o.
`-o` was being required even when renderers don't need output path.
2022-01-12 18:00:08 +01:00