Closes#9058 .
Ensures behavior of `_stash_exp` in `exp run` is matched in `exp save` without duplicating logic.
The simpler implementation ended up being worse for maintenance when wanting to make `exp run` and `exp save` behavior in sync.
`plots.show` returned `definitions` in native `nt` format but sources in `posixpath` format so the matching was failing.
Closes#8689
- Add `test_show_plots_defined_with_native_os_path`.
* studio: Move `post_live_metrics` from `queue/workspace` to `executor/base`.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update dvc/repo/experiments/utils.py
Co-authored-by: skshetry <[email protected]>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: skshetry <[email protected]>
If we want `add`, `commit` and `move` to work for cloud versioning, option must be always True so it is the same as not having it.
Closes#8828
(We were not passing `merge_versioned=True` in `dvc commit`)
- Add `get_random_exp_name` to utils.
Generates a name (`f"{adjective}-{noun}"`) by randomly chosing pairs of adjetive, noun.
Recieves `scm` and `baseline_rev` and retries the random choice until a non-existing ref is found.
Closes https://github.com/iterative/dvc/issues/8650
Allow to include a list of potentially untracked files.
Covers the DVCLive use case where in the first run the generated files won't be tracked in Git yet.
* ui: Fix WSL check in `open_browser`
* Update dvc/ui/__init__.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* hydra: Use `OmegaConf.to_yaml` for dumping `.yaml` output.
Combining `OmegaConf.to_object` with our yaml DUMPER (ruaml.yaml) was not correctly handling strings like 'no'.
`OmegaConf.to_yaml` uses a custom string representer that correctly handles those cases.
Fixes#8583
* Update dvc/utils/hydra.py
Co-authored-by: Dave Berenbaum <[email protected]>
Co-authored-by: Dave Berenbaum <[email protected]>
Defaults List overrides (`defaults/foo=1,2`) were not working because of the inclusion of the `/` character.
Fixed it by updating how we construct the `merged_overrides` and added a regression test.
Requires `--queue` to be used alongside `--set-param`.
Doesn't strictly require `hydra.config.enabled`.
Add `get_hydra_sweeps` to `dvc/utils/hydra.py`. Uses hydra `BasicSweeper` to generate the different sweeps.
`glob` syntax for sweeps is not supported in this P.R.
Closes#8083
---
Using setup from https://hydra.cc/docs/tutorials/basic/your_first_app/composition/ :
```console
$ dvc exp run -S "db=mysql,postgresql" -S "schema=warehouse,school" --queue
Queueing with '{'params.yaml': ['db=mysql', 'schema=warehouse']}'.
Queued experiment '5ab98b8' for future execution.
Queueing with '{'params.yaml': ['db=mysql', 'schema=school']}'.
Queued experiment '57c2fb6' for future execution.
Queueing with '{'params.yaml': ['db=postgresql', 'schema=warehouse']}'.
Queued experiment 'b9d6391' for future execution.
Queueing with '{'params.yaml': ['db=postgresql', 'schema=school']}'.
Queued experiment '145cd55' for future execution.
```
The feature will be used depending on whether `config.hydra.enabled` is True or False.
Uses `hydra.initialize_config_dir` and `hydra.compose` (from https://hydra.cc/docs/advanced/compose_api/) to build the config and dump it to `params.yaml`. The content of the output file will be overwritten.
`config.hydra.config_dir` and `config.hydra.config_name` can be used to customize the values passed to `hydra.initialize_config_dir` and `hydra.compose`.
Can be combined with `--set-param` overrides.
Closes#8082
If an untracked target is provided alongside `stages` or `deps`, don't load the parameters from the target and raise no params found instead.
Closes#8161
This was a regression introduced in #7712.
Calling `repo.metrics.show` from a subdir stopped returning metrics because `fs.isfile(metric)` check returned false.
- Overriding a config value : foo.bar=value
- Appending a config value : +foo.bar=value
- Appending or overriding a config value : ++foo.bar=value
- Removing a config value : ~foo.bar, ~foo.bar=value
See https://hydra.cc/docs/advanced/override_grammar/basic/#modifying-the-config-object
---
Breaking changes:
To modify a list, `foo[0]=bar` must now be passed as `foo.0=bar`.
Modifying a nested list inside a dictionary is not supported by omegaconf.
---
Closes#4883Closes#5868Closes#6129
