* exp show: show metrics that include separator
* sort using partition instead of lpartition
* handle ':' in file and metric names
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix sorting with leading :
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* update msg for multiple overlapping outputs
* update msg for multiple overlapping outputs
* fix type checking
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* use sets of stages to fix mypy errors
* fix tests/func/test_add.py::test_try_adding_multiple_overlaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.282 → v0.0.285](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.282...v0.0.285)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix ruff issues
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
* use _describe in exp list
* fix pre-commit checks
* simplistic fix
* single _describe, too complex
* slightly simpler
* even simpler
* simple, but with regex
* less substitutions
* describe public
* name handling in UI (tests need updating)
* fixed old tests
* lazy imports
---------
Co-authored-by: Tibor Mach <[email protected]>
* hydra: enable config modules
Lets the user specify Hydra configs in installed Python modules as well
as inside directories. Adds `hydra.config_module` to `.dvc/config`.
Fixes#9740
old tests green
tests for config modules
* hydra: fix linter errors
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* hydra: fix mypy error
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* hydra: fix tests
* Update dvc/utils/hydra.py
Co-authored-by: David de la Iglesia Castro <[email protected]>
* address comments
* convince mypy not to worry
* split long line
* fix test
* check for raised error in test
---------
Co-authored-by: Dom Miketa <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: David de la Iglesia Castro <[email protected]>
* repro: introduce --keep-going/--ignore to continue on error
`-k`/`--keep-going` will skip all dependents of a stage if it fails.
It'll still try to reproduce the other dependencies of the pending
targets, before it exits with a non-zero status.
`--ignore-errors` will treat any error during stage execution as a
success, and will continue executing other dependencies as if it
succeeded. Exits with 0 even on failure.
Also fixes:
* --force-downstream used to force everything even if one upstream nodes changed.
This used to work when we supported a single targets, but there could be independent
pipelines, so we should not force everything if any node changes.
Now we'll force reproduce the stage only if one of its dependency changed upstream
or it's upstream node was also forced.
* Makes `--downstream` more correct, in relation to `--allow-missing` and `--pull`.
Now we use an active graph for downstream/upstream logic instead of a subgraph.
Note that this does not affect the downstream logic though, as dvc always goes
downstream, so subgraph was no different than active graph.
The subgraph only did not have the upstream nodes.
* add tests
* add spacing in the message
* repro: make stages run in order of the graph.
In case of --single-item, this means it'll run in the order of arguments passed.
For --downstream, this fixes running of stage out of order when multiple arguments are passed.
For normal case, this enforces the same logic, so that they are never run out-of-order.
* add tests
* tests: repro: add test for --single-item with multiple targets
* graph: support when no nodes are provided
* output: set remove to recursive=false
* try to make test pass on windows
* Update dvc/output.py
---------
Co-authored-by: Ruslan Kuprieiev <[email protected]>
For some unknown reason the mock in this test leaks into the outside tests.
I wasn't able to determine why in the reasonable amount of time, so instead just
simplifying it to not use mocker at all.
That whole file should really be migrated to dvc-data, but that needs some time.
Had the "pleasure" of finding a mocker.patch leak into other tests and one
of the suspicions is that mixing unittest.mock and mocker might be somehow
related.
In any case, we should use mocker where possible anyway.
This avoids recollecting the cache files over and over again.
For example, `dvc fetch` noop in `dvc-bench` went from ~30sec down to ~10sec
for me.
Next step is to also cache remote index, so we don't have to waste those
~10sec checking files in the remote.
Related https://github.com/iterative/dvc-data/pull/376
Ruslan Kuprieiev
Brady Johnson
Dave Berenbaum
dberenbaum
David de la Iglesia Castro
Peter Rowlands (변기호)
skshetry
Matt Seddon
Tibor MachandTibor Mach
Ivan Shcheklein