1110 Commits
Author SHA1 Message Date
Saugat Pachhai (सौगात) a48a19ce31 tests: ignore versions for subprojects 2022-11-07 12:45:32 +01:00
David de la Iglesia Castro 1de983f0de Remove live section support. 2022-11-07 12:45:32 +01:00
Saugat Pachhai (सौगात) 4d17c171f1 tests: use mocker 2022-11-03 23:07:28 +05:45
Saugat Pachhai (सौगात) c02bd7f318 migrate some files to pytest 2022-11-03 23:07:28 +05:45
Saugat Pachhai (सौगात) 1cad8ea7fd migrate test_{add,analytics,init,unprotect,local,process}.py to pytest 2022-11-03 23:07:28 +05:45
skshetry 4ca2158b9d tests: retry celery queue on timeout (#8488)
retry celery queue on timeout
2022-10-31 19:38:05 +05:45
Peter Rowlands 3478a7e6a9 import-url: support cloud versioning 2022-10-19 17:12:16 +03:00
skshetry bbaf57c5b2 _debug: minor refactor on debugtools (#8454)
* _debug: minor refactor on debugtools

* delete test
2022-10-19 18:01:20 +05:45
David de la Iglesia Castro 61f8478885 tests: Remove test_ssh
Moved to dvc-ssh repo in https://github.com/iterative/dvc-ssh/pull/13
2022-10-19 12:35:11 +03:00
David de la Iglesia Castro 3bbf8fa7d2 config: ssh: Add passphrase and ask_passphrase options.
Bump `dvc-ssh`.

Fixes #8137
2022-10-14 12:11:31 +03:00
karajan1001 884182f6f2 Add "status" to replace "running" and "queued" in output of exp show
fix: #7986
1. Add two new flags  `--hide-queued` and  `--hide-failed` to `exp show`
2. Allow `exp show` to show failed experiments.
3. Add unit test for the failed experiments shown.
4. Add name support for failed exp
5. Add error msg to the `exp show` output
2022-10-06 17:20:03 +08:00
Ronan Lamy 22655fa46d ls-url: update command test for --recursive 2022-10-04 14:37:29 +03:00
Ronan Lamy fa7619dfee ls-url: create basic version of dvc ls-url command
For now, this only takes a URL (or local path) and lists its contents
if the protocol supports it. The command fails if the target doesn't
exist and returns the (absolute) path to the target if it's not a
directory.
2022-10-04 14:37:29 +03:00
Peter Rowlands 4904286307 config: wrap UnicodeDecodeErrors on load 2022-09-30 11:45:43 +09:00
Ronan Lamy 04af944e22 fs: test and slightly refactor get_fs_config() 2022-09-28 22:06:40 +03:00
Paweł Redzyński 25ea5f35ec plots: pass all types of plots in --json option
Related: iterative/vscode-dvc#1757
2022-09-27 14:23:42 +02:00
karajan1001 6bd14b21ef Move infofile operation to executor and
Currently infofile dump was spread in both executor and queue classes.
put them all into executor to make them more managable. Move all ending
dump operations to the cleanup function, because the collect_result progress
might failed because of git operations, in which can guarantee them to
be run in a `finally` scope.
2022-09-26 16:25:47 +08:00
karajan1001 c2829f4ce8 Make executor info compatible with old versions. 2022-09-26 16:25:47 +08:00
karajan1001 7fb9dcec12 Add tests for executor status
1. Move some basic test script from function tests to unit test.
2. Add success/failed tests for the status change of `tempdir`, `celery`, `workspace` running case.
2022-09-26 16:25:47 +08:00
karajan1001 90109bccd7 Refactor seperate the initialization of executor and setup environment
Reason:
1. Need some more granular control over initialization progress of
   executor.

What done:
1. Seperate `init_git` and `init_cache` progress out from setup_executor
2. Move `set_ref` into `init_git`
2022-09-26 16:25:47 +08:00
Paweł Redzyńskiandpre-commit-ci[bot] c617769a1d deps: bump codespell (#8199)
[pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/codespell-project/codespell: v2.1.0 → v2.2.1](https://github.com/codespell-project/codespell/compare/v2.1.0...v2.2.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-09-22 19:31:24 +05:45
Ruslan Kuprieiev 35e6f082a4 analytics: use iterative-telemetry for user_id lookup 2022-09-17 00:45:45 +03:00
skshetry 391bb29be2 dvcfs: rename DvcFileSystem to DVCFileSystem (#8307) 2022-09-16 18:18:55 +05:45
skshetry 8e5f4a8f35 data ls: new command to show metadata with outputs (#8252)
data ls: add new command to show metadata for the outputs
2022-09-09 17:36:24 +05:45
Saugat Pachhai (सौगात) c73764156e metadata: support custom metadata in .dvc files
This adds support for custom metadata in outputs under `meta` key.
Also adds `--meta` flag to `add`/`import`/`import-url` that supports
`key=value` values.
2022-09-08 14:10:35 +03:00
skshetry df60df5242 add metadata fields: label, type to data (#8232)
gto like metadata fields: labels and type
2022-09-08 16:33:47 +05:45
Ruslan Kuprieiev a124d7e866 deps: bump dvc-data to 0.6.3 2022-09-07 02:23:29 +03:00
Ruslan Kuprieiev 628be2ae49 deps: bump dvc-data to 0.6.1 2022-09-06 20:30:20 +03:00
Peter Rowlands 660c17f654 output: save version_id meta 2022-09-01 19:36:53 +03:00
skshetry 49da2eef02 tests: set celery ping_task_timeout 3x the default (#8221)
tests: set celery ping_timeout 3x the default
2022-09-01 16:48:19 +05:45
skshetry a8415b01d0 data status: fix quoting on command hints for untracked files (#8211)
data status: fix quoting on command hints
2022-08-30 22:51:39 +05:45
Dave Berenbaum 1d3102f29b data status: update hints to include fetch and checkout (#8209) 2022-08-30 22:05:32 +05:45
David de la Iglesia Castro ab780879d1 info: Include subprojects. 2022-08-30 09:22:39 +02:00
skshetry bfad82a2e3 data status: remove --withdirs, show unknowns in CLI (#8189)
* data status: remove --withdirs, show unknowns in CLI

* data status: only consider unknown from newer object during diff
2022-08-30 12:11:02 +05:45
skshetry e5eec59cb8 test on 3.11 (#8196)
We don't have aiohttp support yet, due to which we have to install
without cythonized extension.

Similarly, there has been some changes in how dataclasses disallows some
mutable arguments to be set as a class property, due to which
`hydra.conf` fails to import in 3.11.

See https://github.com/python/cpython/pull/29867. For now, I have
patched the module and avoid importing as a pytest-plugin.

dvc-hdfs/pyarrow is not there yet, we'll usually see a release 2-3
months after official reelase.

dvc-hdfs is skipped in 3.11 in extras_requires. `--set-params` will
return a nice error message if hydra cannot be imported in 3.11. I
decided to only patch that in tests.
2022-08-29 19:58:17 +05:45
David de la Iglesia Castro 9e2ebe9fcc resolve_rev: raise DvcException on empty repo. 2022-08-29 01:50:14 +03:00
b7aea414e6 plots templates: change ui to not dump to file
Update dvc/commands/plots.py

Co-authored-by: David de la Iglesia Castro <[email protected]>

Update tests/unit/command/test_plots.py

Co-authored-by: David de la Iglesia Castro <[email protected]>

Update dvc/commands/plots.py

Co-authored-by: Paweł Redzyński <[email protected]>

fixed test
2022-08-29 01:46:47 +03:00
skshetry 063eb6904d data status: test cleanup and improvements (#8185) 2022-08-25 18:50:46 +05:45
skshetry 2c556f58d3 data status: fix not-in-cache hint (#8181)
See https://github.com/iterative/dvc.org/pull/3812\#discussion_r936625617.
2022-08-25 15:39:59 +05:45
Daniele Trifirò 26008f1559 import-url/update: add --no-download flag
add --no-download flag to dvc import-url/dvc update to only create/update .dvc
files without downloading the associated data.

Created .dvc files can be fetched using `dvc pull` or can be
updated using `dvc update --no-download`.
2022-08-23 23:07:01 +03:00
Daniele Trifirò 92d680ff94 tests: fix typo in test name 2022-08-23 23:07:01 +03:00
Ruslan Kuprieiev a79ea15827 tests: webhdfs: fix linter 2022-08-18 12:43:12 +03:00
Ruslan Kuprieiev 8db9a2f70f deps: bump dvc-data to 0.1.19 2022-08-18 12:43:12 +03:00
Ruslan Kuprieiev 3fc9f323a7 fs: don't import SSHFileSystem 2022-08-18 00:23:40 +03:00
Ruslan Kuprieiev 3b3866323e deps: bump dvc-data to 0.1.18 2022-08-17 23:02:22 +03:00
Ruslan Kuprieiev fcdad4d229 deps: bump dvc-data to 0.1.16 2022-08-13 22:25:24 +03:00
Ivan Shcheklein 206e00ccbe guard rwlock file to prevent not locked repo edits 2022-08-11 15:54:42 +03:00
Peter Rowlands f13443477d exp run: account for repo root when staging git-deps 2022-08-11 13:48:28 +09:00
karajan1001 9180a718c1 queue status: show final experiment name
fix: #8059

1. showing final experiment name in `queue status` for successful tasks.
2. Add a unit test for `celery_queue.status`
2022-08-08 15:49:51 +09:00
karajan1001 306456d453 queue status: ERROR: Invalid experiment
fix: #8014
> ERROR: Invalid experiment '{entry.stash_rev[:7]}'.

This happens when the queue task failed from a scm error.

{"exc_type": "GitMergeError", "exc_message": ["Cannot fast-forward HEAD
to '05100047a341f2fa4a02421289d48f84d8c45e86'"], "exc_module": "dvc.scm"}.

And the failed task neither create a infofile nor create a fail_stash.

1. Didn't raise DvcException if no infofile found for failed tasks.
   (success tasks still raise it)
2. Add a new unit test for this.
2022-08-08 15:07:11 +09:00