Commit Graph
103 Commits
Author SHA1 Message Date
Dave Berenbaum 12955da3c3 exp: fail fast in empty git repo (#8840) 2023-03-14 17:16:17 +00:00
David de la Iglesia Castro e73cc9b40a exp: Include STUDIO_REPO_URL in run_env.
Closes #8951.
2023-03-14 13:43:03 +01:00
Peter Rowlands 8cbc4c43f1 tests: use session-scoped exp celery fixtures when possible 2023-03-09 16:25:09 +09:00
skshetry 93eea0cd42 deps: bump scmrepo to 0.1.15; close subrepo.dvc on tests teardown (#9122) 2023-03-06 20:21:42 +05:45
Peter Rowlands 78fb5275e5 exp: add dvc exp clean command 2023-03-02 12:35:54 +09:00
Peter Rowlands 648edd71eb exp: move apply tests into their own module 2023-02-20 14:36:38 +09:00
Peter Rowlands 0c95cde712 exp: remove unused ApplyConflictError 2023-02-20 14:36:38 +09:00
skshetry a86470ccbc update black and change line-length to 88 (#8953) 2023-02-04 05:28:07 +02:00
skshetry 7ab4c59d70 test: run pytest-style checks (#8901) 2023-01-28 14:10:53 +05:45
David de la Iglesia Castro 9af6fcad4a exp run: Fix --dry option.
Closes #8831
2023-01-17 13:32:20 +01:00
skshetry da27174e02 dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups, etc. (#8812)
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups etc
2023-01-15 12:45:10 +05:45
dependabot[bot]andSaugat Pachhai 178244f666 build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10 (#8807)
* build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10

Bumps [dvc-task](https://github.com/iterative/dvc-task) from 0.1.9 to 0.1.10.
- [Release notes](https://github.com/iterative/dvc-task/releases)
- [Commits](https://github.com/iterative/dvc-task/compare/0.1.9...0.1.10)

---
updated-dependencies:
- dependency-name: dvc-task
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* remove xfail on json.JSONDecodeError/PermissionError

JSONDecodeError should be fixed by the new release. Next we need to investigate
PermissionError.

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-01-13 17:25:36 +05:45
skshetry a814f04ee0 tests: xfail test_apply_failed on JSONDecodeError/PermissionError (#8796) 2023-01-11 18:04:23 +05:45
Saugat Pachhai (सौगात) 3dfdd577e7 tests: xfail experiments-related flakey tests 2023-01-06 18:59:53 +05:45
dberenbaum b0372918ea exp: stop checking for unchanged experiments 2023-01-04 17:29:12 +01:00
karajan1001 f386f7c082 exp apply can accept exp names and failed exps as target
fix: #8182

1. `exp apply` can apply to failed experiments.
2. `exp apply` can accept exp names as target.
3. Add a new functional test for it.
2022-12-30 17:40:18 +08:00
David de la Iglesia Castro bb7512687c exp: Generate a human-readable name beforehand.
- Add `get_random_exp_name` to utils.
Generates a name (`f"{adjective}-{noun}"`) by randomly chosing pairs of adjetive, noun.
Recieves `scm` and `baseline_rev` and retries the random choice until a non-existing ref is found.

Closes https://github.com/iterative/dvc/issues/8650
2022-12-09 10:32:38 +01:00
David de la Iglesia Castro 43e281521e exp save: cleanup.
Remove metrics.

Move tests to separate file.

Fix `experiments.get_exact_name` usage.

Remove unused checkpoint logic.
2022-11-28 15:49:34 +01:00
Daniele Trifirò 73b46a5789 exp save: initial implementation 2022-11-28 15:49:34 +01:00
David de la Iglesia Castro 8124396218 exp: Expose baseline and name via run_env.
Required by DVCLive for the experiments onboarding and live metrics features.

Related (not sure if closes): https://github.com/iterative/dvc/issues/6668
2022-11-28 13:13:04 +01:00
Ivan Shcheklein d7587556c9 exp list: cleanup and move logic inside repo api 2022-11-25 16:15:17 -08:00
Ivan Shcheklein 76cc649f5e fix/8550+: cover with test 2022-11-16 15:16:37 +05:45
karajan1001 fa54c1af7c Solve run_all flaky tests
1. Modify run all to include currently running exps.
2. bump dvc-task to 0.1.5
2022-11-11 07:04:43 +05:45
Saugat Pachhai (सौगात) c02bd7f318 migrate some files to pytest 2022-11-03 23:07:28 +05:45
karajan1001 d56376c70a exp show: Use batch call on scm.describe
fix: #8451

1. Change the call of scm.describe from individual revision to a
   collection of them. Accelerate the run speed of `exp show`
2. Bump scmrepo to 0.1.2
2022-10-20 17:22:25 +08:00
David de la Iglesia Castro 91f12395f1 exp run: Support composing and dumping Hydra config.
The feature will be used depending on whether `config.hydra.enabled` is True or False.

Uses `hydra.initialize_config_dir` and `hydra.compose` (from https://hydra.cc/docs/advanced/compose_api/) to build the config and dump it to `params.yaml`. The content of the output file will be overwritten.

`config.hydra.config_dir` and `config.hydra.config_name` can be used to customize the values passed to `hydra.initialize_config_dir` and `hydra.compose`.

Can be combined with `--set-param` overrides.

Closes #8082
2022-08-31 17:24:09 +02:00
David de la Iglesia Castro 31635b8312 exp run: Use hydra for parsing --set-param.
- Overriding a config value : foo.bar=value
- Appending a config value : +foo.bar=value
- Appending or overriding a config value : ++foo.bar=value
- Removing a config value : ~foo.bar, ~foo.bar=value

See https://hydra.cc/docs/advanced/override_grammar/basic/#modifying-the-config-object

---

Breaking changes:

To modify a list, `foo[0]=bar` must now be passed as `foo.0=bar`.

Modifying a nested list inside a dictionary is not supported by omegaconf.

---

Closes #4883
Closes #5868
Closes #6129
2022-08-05 17:53:24 +02:00
karajan1001 9dfe88e2b1 exp run: workspace status wrong
fix: #8045
2022-07-26 15:02:09 +05:45
Peter Rowlands 80272a0770 queue: cleanup flaky tests 2022-07-12 21:39:49 +09:00
karajan1001 b37524603a Queue remove: Can remove successed,failed tasks msgs
Followed from #7592

1. Seperate remove method to a new file.
2. Add queued, failed, processed flags to remove.
3. Add new unit tests for queue status --queue/fail/success
4. Implement methods to remove done tasks.
5. add some new unit and functional test for celery_queue.remove
6. bump into dvc-task version 0.0.13
2022-07-12 21:39:49 +09:00
Peter Rowlands 9e0788aa3a exp run: preserve --temp standalone behavior 2022-07-12 21:39:49 +09:00
Peter Rowlands 570f4345a0 exp queue: implement queue.logs() support 2022-07-12 21:39:49 +09:00
Peter Rowlands 425d0701c9 exp queue: return completed exps in queue status 2022-07-12 21:39:49 +09:00
Peter Rowlands 99a11f7056 exp: save failed queue runs in separate stash/reflog 2022-07-12 21:39:49 +09:00
Peter Rowlands f26664a569 exp: re-enable run --run-all with celery queue 2022-07-12 21:39:49 +09:00
Peter Rowlands 4b0c65093a tests: use pytest-celery for exp tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 1c474e8f30 exp: re-enable most tmpdir/queue tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 46c734eb7e exp: temporarily mark tmpdir/queue tests as xfail 2022-07-12 21:39:49 +09:00
Peter Rowlands 7113568c60 exp: refactor base.py into refs.py and exceptions.py 2022-07-12 21:39:49 +09:00
Peter Rowlands 6143146974 exp: refactor stash handling into separate module 2022-07-12 21:39:49 +09:00
David de la Iglesia Castro 20f69a121e api: Move to sub-package.
Split into sub-modules.

Split tests.
2022-06-21 20:52:19 +03:00
Peter Rowlands 31f67cd3de exp apply: preserve untracked files 2022-06-19 22:41:22 -07:00
Ruslan Kuprieiev 005d56eedf fs: rename RepoFileSystem to DvcFileSystem 2022-05-12 00:39:41 +03:00
Ruslan Kuprieiev a80a85e6bd dvc: detach gitfs from local paths
Our gitfs paths are currently tied to the local path in which the cloned
repository resides in, which results in a lot of hacky assumptions throughout
the code.

This PR detaches gitfs from local paths completely, making dvc work with a
proper fsspec gitfs implementation.

Part of #7543
2022-05-11 18:17:39 +03:00
Ruslan Kuprieiev 9d75e8261b repofs: use repo paths 2022-04-18 14:17:20 +03:00
karajan1001andJorge Orpinel d033345860 exp list: add the new flags (#7153)
fix: #7153
Unify the flags in `exp list` to `exp show`

1. Add new `-A`, `-n` flags to `exp list` command.
2. Add unit test for the flag parse.
3. unify the revs retrive logic to `exp show`.
4. Add functional test for `exp list`
5. Update the documents.

Co-authored-by: Jorge Orpinel <[email protected]>
2022-03-08 16:17:33 +08:00
Saugat Pachhai (सौगात) 2f15f30b0e exp run: do not log message twice on workspace executor
Fixes #7276.
2022-01-25 19:01:48 +05:45
Saugat Pachhai (सौगात) f5baf990cf cli: refactor into a module
By splitting dvc.cli into a module, we'll be able to share cli-related
utilities, split parser-related stuffs into smaller parts, improve
reusability and improve API for command extensions in the future.
2022-01-19 12:22:07 +02:00
Saugat Pachhai (सौगात) 4ba504be3e repro/exp run: display stage instead of filepath on failure
Before it used to say 'failed to reproduce dvc.yaml', which is now changed
to print 'failed to reproduce train' or something similar.

For stages that are in dvc.yaml in the current working directory, it will
just print the stage names. For stages in other dvc.yaml files, it'll print

in <relative_path>:<stage_name>.
2022-01-19 06:58:40 +05:45
dependabot[bot] 96acb64fec build(deps): Bump scmrepo from 0.0.6 to 0.0.7
Bumps [scmrepo](https://github.com/iterative/scmrepo) from 0.0.6 to 0.0.7.
- [Release notes](https://github.com/iterative/scmrepo/releases)
- [Commits](https://github.com/iterative/scmrepo/compare/0.0.6...0.0.7)

---
updated-dependencies:
- dependency-name: scmrepo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

tests: fix invalid set_ref in exp func test
2021-12-22 17:52:30 +09:00