* repro: introduce --keep-going/--ignore to continue on error
`-k`/`--keep-going` will skip all dependents of a stage if it fails.
It'll still try to reproduce the other dependencies of the pending
targets, before it exits with a non-zero status.
`--ignore-errors` will treat any error during stage execution as a
success, and will continue executing other dependencies as if it
succeeded. Exits with 0 even on failure.
Also fixes:
* --force-downstream used to force everything even if one upstream nodes changed.
This used to work when we supported a single targets, but there could be independent
pipelines, so we should not force everything if any node changes.
Now we'll force reproduce the stage only if one of its dependency changed upstream
or it's upstream node was also forced.
* Makes `--downstream` more correct, in relation to `--allow-missing` and `--pull`.
Now we use an active graph for downstream/upstream logic instead of a subgraph.
Note that this does not affect the downstream logic though, as dvc always goes
downstream, so subgraph was no different than active graph.
The subgraph only did not have the upstream nodes.
* add tests
* add spacing in the message
* environment variable to show debug options
* lint
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add message to debug opts
* spelling
* Add argument group and move cprofile-dump
line length
linelen
lint
* use DVC_DEBUG instead of DVC_SHOW_DEBUG_OPTIONS
* use -v instead of environ
* Update dvc/_debug.py
Co-authored-by: skshetry <[email protected]>
* fix
---------
Co-authored-by: skshetry <[email protected]>
The logic is now moved to experiments, where it tracks all the stage related files
itself. The `_get_stage_files()` has been modified to also return lockfile path,
so that it can be staged, and has been moved to `stage/utils.py`, and out of `reproduce.py`.
Fixes#9688.
* repro: make stages run in order of the graph.
In case of --single-item, this means it'll run in the order of arguments passed.
For --downstream, this fixes running of stage out of order when multiple arguments are passed.
For normal case, this enforces the same logic, so that they are never run out-of-order.
* add tests
* tests: repro: add test for --single-item with multiple targets
* graph: support when no nodes are provided
Looks like flufl-lock switched to pdm in 8.0 which triggered a
```
pkg_resources.DistributionNotFound: The 'flufl.lock>=5' distribution was not found and is required by dvc
```
error in multiple scenarious. Can't put my finger on anything specific
in flufl-lock yet, so will just limit its version for now.
Fixes https://github.com/iterative/dvc/issues/9654
Relpath might not exist (e.g. on windows between different drives),
we should just use path as is and not get too fancy with the ui,
unless we want to constantly keep ValueError in mind when using
relpath for ui needs all over the place.
These operations should use index, where we always have current
directory and relative paths, so the ui will change accordingly
anyway.
* output: save: error out on cached external outputs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
skshetry
dependabot[bot]
pre-commit-ci[bot]andSaugat Pachhai
Peter Rowlands
Ruslan Kuprieiev
dberenbaum
Jon Crallandskshetry