* [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.282 → v0.0.285](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.282...v0.0.285)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix ruff issues
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
* use _describe in exp list
* fix pre-commit checks
* simplistic fix
* single _describe, too complex
* slightly simpler
* even simpler
* simple, but with regex
* less substitutions
* describe public
* name handling in UI (tests need updating)
* fixed old tests
* lazy imports
---------
Co-authored-by: Tibor Mach <[email protected]>
`dvc commit --help` was also completely broken, which is also fixed.
This adds unit tests for completion generation for both zsh/bash, and
tests that --help works for all commands.
Adds support for `--config` to all three commands. For `ls` and `get`, it is
a purely runtime option that will merge the config you provide with target
repo's configs (similar to how .dvc/config.local works). E.g.
```
$ cat myconfig
[remote "myremote"]
access_key_id = 12345
secret_access_key = 98765
$ dvc get https://github.com/efiop/mydataregistry recent-grads.csv --config myconfig
```
In case of `dvc import`, the semantics are the same, but the value is also
recorded in the resulting dvcfile. E.g.
```
$ dvc import https://github.com/efiop/mydataregistry recent-grads.csv --config myconfig
...
$ cat recent-grads.csv.dvc
md5: 85c86eeb3afa6d1f7d11beedc644d28e
frozen: true
deps:
- path: recent-grads.csv
repo:
url: https://github.com/efiop/mydataregistry
config: myconfig
rev_lock: af22e06bbfe34d62a140437abf32f48b535944a7
outs:
- md5: f447e23a170a9f25a7799f069a1ba934
size: 26872
hash: md5
path: recent-grads.csv
```
This is the most general and powerful mechanism that we can give, that works
for everything from reconfiguring your default remote to completely changing
your remote layout and beyond (will be handy in the future).
Fixes#2466
* repro: introduce --keep-going/--ignore to continue on error
`-k`/`--keep-going` will skip all dependents of a stage if it fails.
It'll still try to reproduce the other dependencies of the pending
targets, before it exits with a non-zero status.
`--ignore-errors` will treat any error during stage execution as a
success, and will continue executing other dependencies as if it
succeeded. Exits with 0 even on failure.
Also fixes:
* --force-downstream used to force everything even if one upstream nodes changed.
This used to work when we supported a single targets, but there could be independent
pipelines, so we should not force everything if any node changes.
Now we'll force reproduce the stage only if one of its dependency changed upstream
or it's upstream node was also forced.
* Makes `--downstream` more correct, in relation to `--allow-missing` and `--pull`.
Now we use an active graph for downstream/upstream logic instead of a subgraph.
Note that this does not affect the downstream logic though, as dvc always goes
downstream, so subgraph was no different than active graph.
The subgraph only did not have the upstream nodes.
* add tests
* add spacing in the message
The logic is now moved to experiments, where it tracks all the stage related files
itself. The `_get_stage_files()` has been modified to also return lockfile path,
so that it can be staged, and has been moved to `stage/utils.py`, and out of `reproduce.py`.
Fixes#9688.
* repro: make stages run in order of the graph.
In case of --single-item, this means it'll run in the order of arguments passed.
For --downstream, this fixes running of stage out of order when multiple arguments are passed.
For normal case, this enforces the same logic, so that they are never run out-of-order.
* add tests
* tests: repro: add test for --single-item with multiple targets
* graph: support when no nodes are provided
With all fetch and checkout related things that we've done in the recent past,
we now have all the mechanisms necessary to provide actual progress (no total,
but still better than spinner) during `dvc data status` operations.
Had the "pleasure" of finding a mocker.patch leak into other tests and one
of the suspicions is that mixing unittest.mock and mocker might be somehow
related.
In any case, we should use mocker where possible anyway.
PKG has to be specified in dvc/_build.py now, and can be
imported using:
from dvc import PKG
This is done to avoid importing from dvc/utils pkg which can
be expensive.
Also it's not an utility, so it is better that it remains at
the top-level namespace.
Note that we may have to fix the path for a few packages that we build,
like conda/snap/chocolatey.
Ruslan Kuprieiev
pre-commit-ci[bot]andSaugat Pachhai
skshetry
Dave BerenbaumandRuslan Kuprieiev
Peter Rowlands
Tibor MachandTibor Mach
Oded Messer
David de la Iglesia Castro