Cuts down `dvc ls . data/mnist/dataset --recursive` for me from ~37 to ~25
seconds.
Note that this also changes behaviour for ls with a broken symlink, as we
were not able to see it before because of a failing `info()` call, but now
we can follow a normal UNIX ls behaviour more closely.
fix: #8612
For checkpoint experiment in some case users might want to give it a early
stopping to reduce variance. But currently if we interrupt/kill the
experiment it will be marked as failed, and all of the completed
checkpoints will be removed as we cleanup all the running directly after
the process failed.
1. We raise CheckpointKilledError intead of StageCmdFailed error if at
least one checkpoint had been commited.
2. Temp_dir executor will continue collecting data if the Checkpoint
stage was interrupted.
3. Raise warnings if a checkpoint stage was incomplete and the other
stages were not forwarded.
4. Add a new functional test for this
- Add `get_random_exp_name` to utils.
Generates a name (`f"{adjective}-{noun}"`) by randomly chosing pairs of adjetive, noun.
Recieves `scm` and `baseline_rev` and retries the random choice until a non-existing ref is found.
Closes https://github.com/iterative/dvc/issues/8650
Allow to include a list of potentially untracked files.
Covers the DVCLive use case where in the first run the generated files won't be tracked in Git yet.
* hydra: Use `OmegaConf.to_yaml` for dumping `.yaml` output.
Combining `OmegaConf.to_object` with our yaml DUMPER (ruaml.yaml) was not correctly handling strings like 'no'.
`OmegaConf.to_yaml` uses a custom string representer that correctly handles those cases.
Fixes#8583
* Update dvc/utils/hydra.py
Co-authored-by: Dave Berenbaum <[email protected]>
Co-authored-by: Dave Berenbaum <[email protected]>
* plots: vega: converter: support multi series data split and props inference
Fixes: #7754, #8399
Related: iterative/studio#4369
* plots: converter: make converter infer x and y labels if not provided
* plots: vega: converter: roll back to datapoints format
* plots: vega: converter: refactor
* plots: infer labels on
* plots: get_plot_id: fix naming
* plots: vega: converter: notes regarding converting
fix: #8350
We shouldn't add locks to exp show to prevent race conditions
as this command is calling continuously by the extension.
So we make a better way to handle all kinds of reference failures.
1. Drop all based experiments if got a SCMError in baseline.
2. Drop all checkpoint experiments if any commit got a SCMError in
one of checkpoint commit.
3. Drop the entire experiments if got a SCMError.
4. Add tests for each of the conditions above.
Ruslan Kuprieiev
Saugat Pachhai (सौगात)
dberenbaum
Daniele Trifirò
karajan1001
David de la Iglesia Castro
Ivan Shcheklein
Yury
Paweł Redzyński
Peter Rowlands
blakeNaccarato