Commit Graph
2757 Commits
Author SHA1 Message Date
David de la Iglesia Castro 9e2ebe9fcc resolve_rev: raise DvcException on empty repo. 2022-08-29 01:50:14 +03:00
David de la Iglesia Castro 2cef130191 repo: Handle no commits for exp show and plots diff.
Make it consistent with all the other commands that use `brancher` (i.e. `dvc diff`, `metrics diff`, `params diff`)

Closes #8163
2022-08-29 01:50:14 +03:00
David de la Iglesia Castro 17b9b1571e api: Add details forparams_show stages syntax.
Explain `stages` syntax for cases where the api is used from a directory different to the one the `dvc.yam` is located.
2022-08-29 01:49:38 +03:00
b7aea414e6 plots templates: change ui to not dump to file
Update dvc/commands/plots.py

Co-authored-by: David de la Iglesia Castro <[email protected]>

Update tests/unit/command/test_plots.py

Co-authored-by: David de la Iglesia Castro <[email protected]>

Update dvc/commands/plots.py

Co-authored-by: Paweł Redzyński <[email protected]>

fixed test
2022-08-29 01:46:47 +03:00
Alex Hall 8bdf9b1138 Better error message when using remove on file instead of dvc file 2022-08-29 01:43:43 +03:00
David de la Iglesia Castro 7b3b4b373b api: Fix params_show for untracked targets.
If an untracked target is provided alongside `stages` or `deps`, don't load the parameters from the target and raise no params found instead.

Closes #8161
2022-08-26 12:00:45 +02:00
Peter Rowlands (변기호) 512bfc4305 tests: mark exp queue func tests xfail (#8186) 2022-08-25 21:04:05 +05:45
skshetry 54ee8c0d77 data status: make untracked files to be relative paths (#8184) 2022-08-25 19:29:43 +05:45
skshetry 063eb6904d data status: test cleanup and improvements (#8185) 2022-08-25 18:50:46 +05:45
David de la Iglesia Castro e815bbe211 metrics: show: Fix show from subdir.
This was a regression introduced in #7712.
Calling `repo.metrics.show` from a subdir stopped returning metrics because `fs.isfile(metric)` check returned false.
2022-08-25 13:02:22 +02:00
Saugat Pachhai (सौगात) 2374c72105 status: add unknown state 2022-08-25 13:23:18 +03:00
Saugat Pachhai (सौगात) 5cf1421c61 fix tests 2022-08-25 13:23:18 +03:00
Saugat Pachhai (सौगात) 6c1d00c2d9 fix diff when no object exists by falling back to shallow_diff 2022-08-25 13:23:18 +03:00
Saugat Pachhai (सौगात) f72456f67c add shallow_diff 2022-08-25 13:23:18 +03:00
skshetry 2c556f58d3 data status: fix not-in-cache hint (#8181)
See https://github.com/iterative/dvc.org/pull/3812\#discussion_r936625617.
2022-08-25 15:39:59 +05:45
Daniele Trifirò 26008f1559 import-url/update: add --no-download flag
add --no-download flag to dvc import-url/dvc update to only create/update .dvc
files without downloading the associated data.

Created .dvc files can be fetched using `dvc pull` or can be
updated using `dvc update --no-download`.
2022-08-23 23:07:01 +03:00
Daniele Trifirò 92d680ff94 tests: fix typo in test name 2022-08-23 23:07:01 +03:00
Ruslan Kuprieiev a79ea15827 tests: webhdfs: fix linter 2022-08-18 12:43:12 +03:00
Ruslan Kuprieiev 8db9a2f70f deps: bump dvc-data to 0.1.19 2022-08-18 12:43:12 +03:00
Ruslan Kuprieiev 3fc9f323a7 fs: don't import SSHFileSystem 2022-08-18 00:23:40 +03:00
Ruslan Kuprieiev 3b3866323e deps: bump dvc-data to 0.1.18 2022-08-17 23:02:22 +03:00
Ruslan Kuprieiev 3a429a869e tests: lockfile: use fs_path 2022-08-13 23:09:21 +03:00
Ruslan Kuprieiev fcdad4d229 deps: bump dvc-data to 0.1.16 2022-08-13 22:25:24 +03:00
Ivan Shcheklein 206e00ccbe guard rwlock file to prevent not locked repo edits 2022-08-11 15:54:42 +03:00
Peter Rowlands f13443477d exp run: account for repo root when staging git-deps 2022-08-11 13:48:28 +09:00
skshetry 7a0b3e9377 Update tests/func/experiments/test_queue.py 2022-08-09 17:50:48 +08:00
karajan1001 d6f8e5a507 Add flaky to a queue related test.
This is a flaky, the queue tasks status might remain "Running" even if
it had already been completed, beside sometimes I also noticed it raise
a pytest fixture error.
2022-08-09 17:50:48 +08:00
Ivan Shcheklein b7c0c41262 lock test name fix 2022-08-09 00:47:04 +03:00
karajan1001 9180a718c1 queue status: show final experiment name
fix: #8059

1. showing final experiment name in `queue status` for successful tasks.
2. Add a unit test for `celery_queue.status`
2022-08-08 15:49:51 +09:00
karajan1001 306456d453 queue status: ERROR: Invalid experiment
fix: #8014
> ERROR: Invalid experiment '{entry.stash_rev[:7]}'.

This happens when the queue task failed from a scm error.

{"exc_type": "GitMergeError", "exc_message": ["Cannot fast-forward HEAD
to '05100047a341f2fa4a02421289d48f84d8c45e86'"], "exc_module": "dvc.scm"}.

And the failed task neither create a infofile nor create a fail_stash.

1. Didn't raise DvcException if no infofile found for failed tasks.
   (success tasks still raise it)
2. Add a new unit test for this.
2022-08-08 15:07:11 +09:00
Ivan Shcheklein 95421a5605 fix logic for unlock on unlocked lock calls 2022-08-08 14:25:10 +09:00
David de la Iglesia Castro 31635b8312 exp run: Use hydra for parsing --set-param.
- Overriding a config value : foo.bar=value
- Appending a config value : +foo.bar=value
- Appending or overriding a config value : ++foo.bar=value
- Removing a config value : ~foo.bar, ~foo.bar=value

See https://hydra.cc/docs/advanced/override_grammar/basic/#modifying-the-config-object

---

Breaking changes:

To modify a list, `foo[0]=bar` must now be passed as `foo.0=bar`.

Modifying a nested list inside a dictionary is not supported by omegaconf.

---

Closes #4883
Closes #5868
Closes #6129
2022-08-05 17:53:24 +02:00
David de la Iglesia Castro 45ac64ddde api: Fix params_show when stage has no params
Also fix when `deps=True` and no params in the entire pipeline.

Closes #8064
2022-08-05 16:26:19 +03:00
Sami Abdul-Wahid c7b22b467c Upgrade from toml to tomlkit library (#7468)
* Upgrade from toml to tomlkit library
2022-08-05 18:58:22 +05:45
karajan1001 ae83e3c787 Now exp remove can clean up failed tasks and associated successful tasks
fix: #8018
fix: #8071

1. Add one line to make `exp remove` can remove failed tasks, and will
   also delete associated successful tasks
2. Add a new unit test for this behaviour.
3. Solve the unstable test_queue problem.
2022-08-05 16:09:51 +09:00
karajan1001 2182c4508d Make exp remove flags and arguments mutually exclusive.
1. Current behavior didn't follow the either or description.
2. Make the underlying logic more clear.
2022-08-05 16:09:51 +09:00
karajan1001 905c416ffc Decouple queue clear and remove commands 2022-08-05 16:09:51 +09:00
Ruslan Kuprieiev 0ce4e769f1 repo: make lock/rwlock kinda work on non local fs 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev fcc7240617 odb: use odb.path instead of odb.cache_dir 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev 6a7b55fdf7 datafs: use dvc-data's datafs 2022-08-02 03:13:06 +03:00
Ivan Shcheklein 9ff18502b6 add gdrive remote profile field to config 2022-07-27 23:29:18 +03:00
Paweł Redzyński 6543683c0c project: update dvc render and dvclive test dependencies 2022-07-26 13:40:59 +02:00
karajan1001 9dfe88e2b1 exp run: workspace status wrong
fix: #8045
2022-07-26 15:02:09 +05:45
Saugat Pachhai (सौगात) 896acedb07 data status: optimize printing status list
`--untracked-files` and `--unchanged` coupled with `--granular`
puts a lot of stress on `show_status`. Currently, with those options,
dvc spends ~26s on `_show_status` function.

Unhappy with the above result, I looked into improving the situation with
`console.out`. It did reduce that to 9s but it still felt like a lot.

Then I dropped to good old colorama, which dropped to 3s(term)/1.24(redirected).
I tested with buffering output which dropped this further to
~600ms(term)/~100ms(redirected).

This may seem like premature optimization, but this was preventing me
from figuring out perf issues in granular outputs, and I believe the
user should not be penalize for using feature that we already offer.
2022-07-25 17:47:23 +05:45
karajan1001 a4763593c7 queue: clean local EXEC_HEAD & EXEC_BASELINE after we run exp with queue.
fix: #8031

Currently when we use tmp directory to run a experiment, we set
`EXEC_HEAD` , `EXEC_BASELINE` and `EXEC_MERGE` heads locally but forget
to remove it. This cause some problem when we use `exp show`

1. Clean `EXEC_HEAD` , `EXEC_BASELINE` and `EXEC_MERGE` after we pushed
   them to tmp_dir
2. Add a unit test for it.

Co-authored-by:
2022-07-25 18:17:12 +08:00
Saugat Pachhai (सौगात) c3ac4f84bf data status: fix untracked files output for subdir 2022-07-22 16:29:04 +03:00
Saugat Pachhai (सौगात) 61729ff66a use M fixture for concise tests 2022-07-22 16:29:04 +03:00
Saugat Pachhai (सौगात) 3dfcb82e5c add tests for data:status command 2022-07-22 16:29:04 +03:00
Saugat Pachhai (सौगात) 340a158595 introduce new dvc data status command 2022-07-22 16:29:04 +03:00
Ruslan Kuprieiev b9b2e5bbd9 Delete test_gdrive.py
Moved to dvc-gdrive.
2022-07-21 19:34:04 +03:00