4823 Commits
Author SHA1 Message Date
skshetry 8968405770 data status: fix path for committed changes in Windows (#8220)
The path we get from GitFileSystem is posixpath, even on Windows. We need to
convert that posixpath to os.path here.
2022-09-01 10:02:08 +05:45
David de la Iglesia Castro 91f12395f1 exp run: Support composing and dumping Hydra config.
The feature will be used depending on whether `config.hydra.enabled` is True or False.

Uses `hydra.initialize_config_dir` and `hydra.compose` (from https://hydra.cc/docs/advanced/compose_api/) to build the config and dump it to `params.yaml`. The content of the output file will be overwritten.

`config.hydra.config_dir` and `config.hydra.config_name` can be used to customize the values passed to `hydra.initialize_config_dir` and `hydra.compose`.

Can be combined with `--set-param` overrides.

Closes #8082
2022-08-31 17:24:09 +02:00
Saugat Pachhai (सौगात) d3569035c8 output.get_obj: catch ObjectCorruptedError 2022-08-30 21:31:33 +03:00
David de la Iglesia Castro daee52e176 info: skip missing subprojects 2022-08-30 20:15:41 +03:00
skshetry a8415b01d0 data status: fix quoting on command hints for untracked files (#8211)
data status: fix quoting on command hints
2022-08-30 22:51:39 +05:45
Daniele Trifirò 77f56090ff do not checkout partials imports on fetch 2022-08-30 20:05:09 +03:00
Dave Berenbaum 1d3102f29b data status: update hints to include fetch and checkout (#8209) 2022-08-30 22:05:32 +05:45
David de la Iglesia Castro ab780879d1 info: Include subprojects. 2022-08-30 09:22:39 +02:00
skshetry bfad82a2e3 data status: remove --withdirs, show unknowns in CLI (#8189)
* data status: remove --withdirs, show unknowns in CLI

* data status: only consider unknown from newer object during diff
2022-08-30 12:11:02 +05:45
skshetry e5eec59cb8 test on 3.11 (#8196)
We don't have aiohttp support yet, due to which we have to install
without cythonized extension.

Similarly, there has been some changes in how dataclasses disallows some
mutable arguments to be set as a class property, due to which
`hydra.conf` fails to import in 3.11.

See https://github.com/python/cpython/pull/29867. For now, I have
patched the module and avoid importing as a pytest-plugin.

dvc-hdfs/pyarrow is not there yet, we'll usually see a release 2-3
months after official reelase.

dvc-hdfs is skipped in 3.11 in extras_requires. `--set-params` will
return a nice error message if hydra cannot be imported in 3.11. I
decided to only patch that in tests.
2022-08-29 19:58:17 +05:45
Paweł Redzyński 2a4b83060e plots: support dirs in top level definitions
Fixes: #7914
2022-08-29 10:51:16 +02:00
David de la Iglesia Castro 9e2ebe9fcc resolve_rev: raise DvcException on empty repo. 2022-08-29 01:50:14 +03:00
David de la Iglesia Castro 2cef130191 repo: Handle no commits for exp show and plots diff.
Make it consistent with all the other commands that use `brancher` (i.e. `dvc diff`, `metrics diff`, `params diff`)

Closes #8163
2022-08-29 01:50:14 +03:00
David de la Iglesia Castro 17b9b1571e api: Add details forparams_show stages syntax.
Explain `stages` syntax for cases where the api is used from a directory different to the one the `dvc.yam` is located.
2022-08-29 01:49:38 +03:00
skshetry 37c8ef3406 Update dvc/commands/plots.py 2022-08-29 01:46:47 +03:00
b7aea414e6 plots templates: change ui to not dump to file
Update dvc/commands/plots.py

Co-authored-by: David de la Iglesia Castro <[email protected]>

Update tests/unit/command/test_plots.py

Co-authored-by: David de la Iglesia Castro <[email protected]>

Update dvc/commands/plots.py

Co-authored-by: Paweł Redzyński <[email protected]>

fixed test
2022-08-29 01:46:47 +03:00
pre-commit-ci[bot] 126a64618b [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2022-08-29 01:43:43 +03:00
Alex Hall 8bdf9b1138 Better error message when using remove on file instead of dvc file 2022-08-29 01:43:43 +03:00
David de la Iglesia Castro 7b3b4b373b api: Fix params_show for untracked targets.
If an untracked target is provided alongside `stages` or `deps`, don't load the parameters from the target and raise no params found instead.

Closes #8161
2022-08-26 12:00:45 +02:00
skshetry 54ee8c0d77 data status: make untracked files to be relative paths (#8184) 2022-08-25 19:29:43 +05:45
skshetry 063eb6904d data status: test cleanup and improvements (#8185) 2022-08-25 18:50:46 +05:45
David de la Iglesia Castro e815bbe211 metrics: show: Fix show from subdir.
This was a regression introduced in #7712.
Calling `repo.metrics.show` from a subdir stopped returning metrics because `fs.isfile(metric)` check returned false.
2022-08-25 13:02:22 +02:00
Saugat Pachhai (सौगात) 2374c72105 status: add unknown state 2022-08-25 13:23:18 +03:00
Saugat Pachhai (सौगात) 5cf1421c61 fix tests 2022-08-25 13:23:18 +03:00
Saugat Pachhai (सौगात) 6c1d00c2d9 fix diff when no object exists by falling back to shallow_diff 2022-08-25 13:23:18 +03:00
Saugat Pachhai (सौगात) f72456f67c add shallow_diff 2022-08-25 13:23:18 +03:00
skshetry 2c556f58d3 data status: fix not-in-cache hint (#8181)
See https://github.com/iterative/dvc.org/pull/3812\#discussion_r936625617.
2022-08-25 15:39:59 +05:45
Ronan Lamy cc6deb24c8 fs: remove obsolete special cases 2022-08-23 23:20:42 +03:00
Ronan Lamy e681030c72 fs: simplify filtering out dvcfiles 2022-08-23 23:20:42 +03:00
Ronan Lamy 7a51548214 fs: Add fast path for the no subrepo case in _get_fs_pair() 2022-08-23 23:20:42 +03:00
Ronan Lamy 8e99fd0091 ls: Use fs.walk(..., detail=True) instead of separate call to fs.info() 2022-08-23 23:20:42 +03:00
Ronan Lamy 7e5a586f85 fs: Remove redundant .isdvc() check
fs.walk(..., dvc_only=True) already outputs dvc objects only, so
fs.isdvc(path, recursive=True) is always True.
2022-08-23 23:20:42 +03:00
Ronan Lamy c76a84b7b1 fs: remove one call to ._get_key() in _DvcFS._get_fs_pair() 2022-08-23 23:20:42 +03:00
Daniele Trifirò 26008f1559 import-url/update: add --no-download flag
add --no-download flag to dvc import-url/dvc update to only create/update .dvc
files without downloading the associated data.

Created .dvc files can be fetched using `dvc pull` or can be
updated using `dvc update --no-download`.
2022-08-23 23:07:01 +03:00
Daniele Trifirò c69c46d08c fix error message on fetch/pull/checkout failure 2022-08-23 23:07:01 +03:00
skshetry 055641a51e deps: bump dvc-data to 0.1.21 (#8147) 2022-08-19 13:22:56 +05:45
Ruslan Kuprieiev a67fea2235 doctor: ignore missing plugins 2022-08-18 21:02:41 +03:00
Paweł Redzyński 25166f3e02 plots: show-vega: adjust renderers before show vega
Fixes: iterative/cml#1112
2022-08-18 12:52:52 +02:00
Ruslan Kuprieiev 8db9a2f70f deps: bump dvc-data to 0.1.19 2022-08-18 12:43:12 +03:00
Ruslan Kuprieiev 3fc9f323a7 fs: don't import SSHFileSystem 2022-08-18 00:23:40 +03:00
Ruslan Kuprieiev bf0aa5e8aa repo: data: ignore pylint false-positives 2022-08-17 23:02:22 +03:00
Ruslan Kuprieiev 3b3866323e deps: bump dvc-data to 0.1.18 2022-08-17 23:02:22 +03:00
Ruslan Kuprieiev 4a95020155 gs: add endpointurl 2022-08-14 13:39:03 +03:00
Ruslan Kuprieiev fcdad4d229 deps: bump dvc-data to 0.1.16 2022-08-13 22:25:24 +03:00
Ivan Shcheklein 206e00ccbe guard rwlock file to prevent not locked repo edits 2022-08-11 15:54:42 +03:00
Peter Rowlands f13443477d exp run: account for repo root when staging git-deps 2022-08-11 13:48:28 +09:00
Dave Berenbaum 9837747eef exp run: ui tweak (#8124) 2022-08-11 09:14:43 +05:45
Alex HallandDavid de la Iglesia Castro 2ca1d6c7c1 refactor: extract load_path function from common code using LOADERS (#7983)
Following up on https://github.com/iterative/dvc/pull/7965#discussion_r912684074

Co-authored-by: David de la Iglesia Castro <[email protected]>
2022-08-09 09:09:43 +00:00
Ivan Shcheklein f7fbac1eed stage add: fix hint message consistency (#8112) 2022-08-09 11:18:37 +05:45
skshetry 7c1e339650 data status: handle path joining properly for tree leaves/items (#8107) 2022-08-09 09:34:15 +05:45