Commit Graph
110 Commits
Author SHA1 Message Date
Peter Rowlands c71a4082e6 testing: expose force=... in scm_add 2023-04-10 13:17:47 +03:00
Peter Rowlands ce269ffa58 exp: use explicit scm.add(force=True) when necessary
- add test for dir dep containing gitignored files
2023-04-10 13:17:47 +03:00
David de la Iglesia Castro 3312a90949 exp run: Add --copy-paths arg.
List of paths to copy inside the temp directory. Only used if `--temp` or `--queue` is specified.

Closes #5800
2023-04-05 23:17:07 +02:00
David de la Iglesia Castro 7c8e6e3156 exp run: Track top level files. 2023-03-31 19:23:40 +02:00
David de la Iglesia Castro 1d8b2e5fa6 exp run: Propagate .dvc/config.local in TempDirExecutor.
Closes #7254
2023-03-31 09:02:22 +02:00
David de la Iglesia CastroandDave Berenbaum e9fb0b92aa exp branch: Make branch arg optional.
Default to `f"{ref_info.name}-branch"` where ref_info is inferred from the `exp_rev` argument.

Co-authored-by: Dave Berenbaum <[email protected]>
2023-03-24 19:11:06 +01:00
pre-commit-ci[bot]andDavid de la Iglesia Castro 48b56c83dc [pre-commit.ci] pre-commit autoupdate (#9168)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.254 → v0.0.257](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.254...v0.0.257)
- [github.com/codespell-project/codespell: v2.2.2 → v2.2.4](https://github.com/codespell-project/codespell/compare/v2.2.2...v2.2.4)

* pre-commit: fix ruff errors

* pre-commit: fix flake8

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: David de la Iglesia Castro <[email protected]>
2023-03-23 13:05:31 +00:00
Dave Berenbaum 12955da3c3 exp: fail fast in empty git repo (#8840) 2023-03-14 17:16:17 +00:00
David de la Iglesia Castro e73cc9b40a exp: Include STUDIO_REPO_URL in run_env.
Closes #8951.
2023-03-14 13:43:03 +01:00
Peter Rowlands 8cbc4c43f1 tests: use session-scoped exp celery fixtures when possible 2023-03-09 16:25:09 +09:00
skshetry 93eea0cd42 deps: bump scmrepo to 0.1.15; close subrepo.dvc on tests teardown (#9122) 2023-03-06 20:21:42 +05:45
Peter Rowlands 78fb5275e5 exp: add dvc exp clean command 2023-03-02 12:35:54 +09:00
Peter Rowlands 648edd71eb exp: move apply tests into their own module 2023-02-20 14:36:38 +09:00
Peter Rowlands 0c95cde712 exp: remove unused ApplyConflictError 2023-02-20 14:36:38 +09:00
skshetry a86470ccbc update black and change line-length to 88 (#8953) 2023-02-04 05:28:07 +02:00
skshetry 7ab4c59d70 test: run pytest-style checks (#8901) 2023-01-28 14:10:53 +05:45
David de la Iglesia Castro 9af6fcad4a exp run: Fix --dry option.
Closes #8831
2023-01-17 13:32:20 +01:00
skshetry da27174e02 dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups, etc. (#8812)
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups etc
2023-01-15 12:45:10 +05:45
dependabot[bot]andSaugat Pachhai 178244f666 build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10 (#8807)
* build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10

Bumps [dvc-task](https://github.com/iterative/dvc-task) from 0.1.9 to 0.1.10.
- [Release notes](https://github.com/iterative/dvc-task/releases)
- [Commits](https://github.com/iterative/dvc-task/compare/0.1.9...0.1.10)

---
updated-dependencies:
- dependency-name: dvc-task
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* remove xfail on json.JSONDecodeError/PermissionError

JSONDecodeError should be fixed by the new release. Next we need to investigate
PermissionError.

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-01-13 17:25:36 +05:45
skshetry a814f04ee0 tests: xfail test_apply_failed on JSONDecodeError/PermissionError (#8796) 2023-01-11 18:04:23 +05:45
Saugat Pachhai (सौगात) 3dfdd577e7 tests: xfail experiments-related flakey tests 2023-01-06 18:59:53 +05:45
dberenbaum b0372918ea exp: stop checking for unchanged experiments 2023-01-04 17:29:12 +01:00
karajan1001 f386f7c082 exp apply can accept exp names and failed exps as target
fix: #8182

1. `exp apply` can apply to failed experiments.
2. `exp apply` can accept exp names as target.
3. Add a new functional test for it.
2022-12-30 17:40:18 +08:00
David de la Iglesia Castro bb7512687c exp: Generate a human-readable name beforehand.
- Add `get_random_exp_name` to utils.
Generates a name (`f"{adjective}-{noun}"`) by randomly chosing pairs of adjetive, noun.
Recieves `scm` and `baseline_rev` and retries the random choice until a non-existing ref is found.

Closes https://github.com/iterative/dvc/issues/8650
2022-12-09 10:32:38 +01:00
David de la Iglesia Castro 43e281521e exp save: cleanup.
Remove metrics.

Move tests to separate file.

Fix `experiments.get_exact_name` usage.

Remove unused checkpoint logic.
2022-11-28 15:49:34 +01:00
Daniele Trifirò 73b46a5789 exp save: initial implementation 2022-11-28 15:49:34 +01:00
David de la Iglesia Castro 8124396218 exp: Expose baseline and name via run_env.
Required by DVCLive for the experiments onboarding and live metrics features.

Related (not sure if closes): https://github.com/iterative/dvc/issues/6668
2022-11-28 13:13:04 +01:00
Ivan Shcheklein d7587556c9 exp list: cleanup and move logic inside repo api 2022-11-25 16:15:17 -08:00
Ivan Shcheklein 76cc649f5e fix/8550+: cover with test 2022-11-16 15:16:37 +05:45
karajan1001 fa54c1af7c Solve run_all flaky tests
1. Modify run all to include currently running exps.
2. bump dvc-task to 0.1.5
2022-11-11 07:04:43 +05:45
Saugat Pachhai (सौगात) c02bd7f318 migrate some files to pytest 2022-11-03 23:07:28 +05:45
karajan1001 d56376c70a exp show: Use batch call on scm.describe
fix: #8451

1. Change the call of scm.describe from individual revision to a
   collection of them. Accelerate the run speed of `exp show`
2. Bump scmrepo to 0.1.2
2022-10-20 17:22:25 +08:00
David de la Iglesia Castro 91f12395f1 exp run: Support composing and dumping Hydra config.
The feature will be used depending on whether `config.hydra.enabled` is True or False.

Uses `hydra.initialize_config_dir` and `hydra.compose` (from https://hydra.cc/docs/advanced/compose_api/) to build the config and dump it to `params.yaml`. The content of the output file will be overwritten.

`config.hydra.config_dir` and `config.hydra.config_name` can be used to customize the values passed to `hydra.initialize_config_dir` and `hydra.compose`.

Can be combined with `--set-param` overrides.

Closes #8082
2022-08-31 17:24:09 +02:00
David de la Iglesia Castro 31635b8312 exp run: Use hydra for parsing --set-param.
- Overriding a config value : foo.bar=value
- Appending a config value : +foo.bar=value
- Appending or overriding a config value : ++foo.bar=value
- Removing a config value : ~foo.bar, ~foo.bar=value

See https://hydra.cc/docs/advanced/override_grammar/basic/#modifying-the-config-object

---

Breaking changes:

To modify a list, `foo[0]=bar` must now be passed as `foo.0=bar`.

Modifying a nested list inside a dictionary is not supported by omegaconf.

---

Closes #4883
Closes #5868
Closes #6129
2022-08-05 17:53:24 +02:00
karajan1001 9dfe88e2b1 exp run: workspace status wrong
fix: #8045
2022-07-26 15:02:09 +05:45
Peter Rowlands 80272a0770 queue: cleanup flaky tests 2022-07-12 21:39:49 +09:00
karajan1001 b37524603a Queue remove: Can remove successed,failed tasks msgs
Followed from #7592

1. Seperate remove method to a new file.
2. Add queued, failed, processed flags to remove.
3. Add new unit tests for queue status --queue/fail/success
4. Implement methods to remove done tasks.
5. add some new unit and functional test for celery_queue.remove
6. bump into dvc-task version 0.0.13
2022-07-12 21:39:49 +09:00
Peter Rowlands 9e0788aa3a exp run: preserve --temp standalone behavior 2022-07-12 21:39:49 +09:00
Peter Rowlands 570f4345a0 exp queue: implement queue.logs() support 2022-07-12 21:39:49 +09:00
Peter Rowlands 425d0701c9 exp queue: return completed exps in queue status 2022-07-12 21:39:49 +09:00
Peter Rowlands 99a11f7056 exp: save failed queue runs in separate stash/reflog 2022-07-12 21:39:49 +09:00
Peter Rowlands f26664a569 exp: re-enable run --run-all with celery queue 2022-07-12 21:39:49 +09:00
Peter Rowlands 4b0c65093a tests: use pytest-celery for exp tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 1c474e8f30 exp: re-enable most tmpdir/queue tests 2022-07-12 21:39:49 +09:00
Peter Rowlands 46c734eb7e exp: temporarily mark tmpdir/queue tests as xfail 2022-07-12 21:39:49 +09:00
Peter Rowlands 7113568c60 exp: refactor base.py into refs.py and exceptions.py 2022-07-12 21:39:49 +09:00
Peter Rowlands 6143146974 exp: refactor stash handling into separate module 2022-07-12 21:39:49 +09:00
David de la Iglesia Castro 20f69a121e api: Move to sub-package.
Split into sub-modules.

Split tests.
2022-06-21 20:52:19 +03:00
Peter Rowlands 31f67cd3de exp apply: preserve untracked files 2022-06-19 22:41:22 -07:00
Ruslan Kuprieiev 005d56eedf fs: rename RepoFileSystem to DvcFileSystem 2022-05-12 00:39:41 +03:00