Commit Graph
85 Commits
Author SHA1 Message Date
eeshsaxena f74c1c0e70 docs: fix parameter name mismatches in docstrings (#11060)
- repo/experiments/queue/base.py: `scm` -> `repo` in
  BaseStashQueue.__init__ (signature takes a Repo, not an SCM instance)
- repo/experiments/queue/tasks.py: stale `tmp_dir`/`entry_dict` args in
  cleanup_exp replaced with the actual `executor`/`infofile` parameters
2026-07-10 13:12:18 +05:45
Staszek Markowski d086bbbd3a exp: add support for --no-hydra flag (#10866) 2025-12-02 16:05:13 +05:45
pre-commit-ci[bot] 5bec7d1bb9 [pre-commit.ci] pre-commit autoupdate (#10786)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-26 18:49:25 +05:45
pre-commit-ci[bot]andSaugat Pachhai 6cc349edb4 [pre-commit.ci] pre-commit autoupdate (#10717)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.9.9 → v0.11.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.9.9...v0.11.4)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix ruff errors

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2025-04-08 03:15:09 +00:00
pre-commit-ci[bot] 7e75b0616f [pre-commit.ci] pre-commit autoupdate (#10635)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-11-26 07:34:46 +05:45
pre-commit-ci[bot]andSaugat Pachhai 3375a7c39f [pre-commit.ci] pre-commit autoupdate (#10372)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.3.3 → v0.3.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.3.3...v0.3.4)

* fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-03-26 06:37:53 +00:00
skshetry 6f6038888f sort imports for all libraries starting with dvc_ as a first party (#10371)
dvc_studio_client, dvc_ssh, etc. were not properly sorted.
2024-03-23 20:13:13 +05:45
dberenbaum 7df58ce404 Revert "refactor notify_refs_to_studio"
This reverts commit c16d615b55f2bb359b731939ad22712a502d8a8d.
2024-03-11 10:47:01 -04:00
dberenbaum a96a446742 refactor notify_refs_to_studio 2024-03-11 10:47:01 -04:00
skshetry 2bf7bd990a pep584ify: use dict merge operator (#10279)
This was introduced in Python 3.9. The change makes
dict merges 'obvious' and looks cleaner.

I haven't changed `.update()` to `|=` (update operator),
as I find the former to be more readable and "obvious".
2024-02-03 15:58:07 +05:45
skshetry 9a1bcd7f1d nit: fit statement/expr in the same line instead of hanging (#10267)
The code are wrapped due to us using line-length=79 before, but now some
of the code can easily fit in 88 chars.

The formatter did not change this automatically when we changed line-length,
because of magic trailing commas that gets added when they turn into multiple lines.

Black/ruff wrapping them in parens and splitting them into multiple lines
was making it difficult to read.
2024-01-30 18:23:04 +05:45
skshetry b647475a64 drop Python 3.8 support (#10265)
* drop Python 3.8

Note that this PR only makes typing related changes.
Other changes can be done once this gets merged, they
are relatively few in numbers.

* bump build python version

* add typing_extensions to lint group
2024-01-30 11:33:25 +02:00
18a9fea4af hydra: support plugins (#10240)
* hydra: support plugins

* Update dvc/utils/hydra.py

* fix tests

* load plugins after some checks run

---------

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
Co-authored-by: skshetry <[email protected]>
2024-01-23 19:58:14 +05:45
skshetry d678773b53 remove more pylint suppression comments (#10084) 2023-11-10 15:19:52 +05:45
skshetry 519cce3fe4 logger: type-safe logger.trace (#10083)
* use logger.getChild from `dvc.log` instead of `logging.getLogger`

I got tired adding `# type: ignore[attr-defined]` while using `logger.trace`.
So, now you need to import logger from `from dvc.log import logger` and
create a child logger using `logger.getChild(__name__)`. `logger.getChild`
will return a logger type-hinted as a logger having a `trace` method.

So, from now on, instead of:
```python
import logging

logger = logging.getogger(__name__)
```

you'll have to do following:
```python
from dvc.log import logger

logger = logger.getChild(__name__)
```

* remove all dvc imports from dvc.logger

* use Tqdm back
2023-11-10 12:55:49 +05:45
skshetry cbc111b519 remove flake8/bandit (#10082)
ruff already runs most of the flake8/bandit checks
2023-11-09 21:26:37 +05:45
Peter Rowlands 65d412bfa3 stage.run: always set DVC_ROOT env var 2023-09-06 18:43:52 +09:00
Matt Seddon 5b69a16ea7 Add DVC_ROOT env var into queued experiment runs (#9879) 2023-08-28 22:37:03 +00:00
009b238807 Feature/hydra config modules (#9783)
* hydra: enable config modules

Lets the user specify Hydra configs in installed Python modules as well
as inside directories. Adds `hydra.config_module` to `.dvc/config`.

Fixes #9740

old tests green

tests for config modules

* hydra: fix linter errors

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* hydra: fix mypy error

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* hydra: fix tests

* Update dvc/utils/hydra.py

Co-authored-by: David de la Iglesia Castro <[email protected]>

* address comments

* convince mypy not to worry

* split long line

* fix test

* check for raised error in test

---------

Co-authored-by: Dom Miketa <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: David de la Iglesia Castro <[email protected]>
2023-08-06 10:39:09 +02:00
Peter Rowlands 9c9ca39fb4 exp: fix rebased imports 2023-05-23 15:50:58 +09:00
Peter Rowlands 629223cbc2 exp: drop deprecated get_running_exps()
- replaced with queue.collect_active()
2023-05-23 15:43:50 +09:00
Peter Rowlands 8cb2646023 exp: add contextmanager for switching to active tempdir repo 2023-05-23 15:43:50 +09:00
David de la Iglesia Castro 25313f5d82 exp: Drop checkpoints (#9271)
exp: Drop `checkpoints`.

Closes #9221
2023-05-23 11:32:18 +05:45
Dave Berenbaum 7f6890ff25 Live metrics config (#9433)
* exp: live metrics config token and offline

* refactor live metrics config

* use get_studio_config

* revert to using env vars for studio config

* update dvc-studio-client to support live metrics config changes

* upgrade dvc-studio-client

* don't change environ when posting live metrics events
2023-05-17 09:21:01 +00:00
David de la Iglesia Castro c74dc20b97 exp run: Support --message.
Custom commit message to use when committing the experiment.

Part of #8870
2023-05-08 20:08:47 +02:00
David de la Iglesia Castro b6d8c07d4d studio: Ensure studio_repo_url works with exp run --queue.
Closes #9349
2023-04-21 21:04:48 +02:00
Peter Rowlands 58f74ae543 exp: standardize exp data collection 2023-04-20 16:58:32 +09:00
Peter Rowlands ce269ffa58 exp: use explicit scm.add(force=True) when necessary
- add test for dir dep containing gitignored files
2023-04-10 13:17:47 +03:00
David de la Iglesia Castro 3312a90949 exp run: Add --copy-paths arg.
List of paths to copy inside the temp directory. Only used if `--temp` or `--queue` is specified.

Closes #5800
2023-04-05 23:17:07 +02:00
skshetry 7c3c47b4a2 bump ruff to 0.0.260 and enable bandit (#9300)
* bump ruff to 0.0.260 and enable bandit

* install toml extra for bandit

* suppress insecure use of hash md5
2023-04-04 10:01:43 +05:45
David de la Iglesia Castro e73cc9b40a exp: Include STUDIO_REPO_URL in run_env.
Closes #8951.
2023-03-14 13:43:03 +01:00
Ruslan Kuprieiev 81da91a973 exp: run: don't relink during commit
Part of #8809
2023-02-24 08:09:35 +02:00
skshetry 1de37556d5 bump ruff to v0.0.247 (#9050)
* bump ruff to v0.0.247

* ruff: enable pep8-naming check

rename some camelCased APIs, especially in logger
rename G variables to graph
ignore some variable namings and help messages in dvc.commands
2023-02-18 16:50:35 +05:45
David de la Iglesia Castro 320771551b exp: Preserve staged changes after experiments.
- Detach head after stashing workspace.
- Use reinstate_index.
2023-02-14 16:59:10 +01:00
skshetry a86470ccbc update black and change line-length to 88 (#8953) 2023-02-04 05:28:07 +02:00
98bcfad8e7 build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7 (#8821)
* build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7

Bumps [scmrepo](https://github.com/iterative/scmrepo) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/iterative/scmrepo/releases)
- [Commits](https://github.com/iterative/scmrepo/compare/0.1.6...0.1.7)

---
updated-dependencies:
- dependency-name: scmrepo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* mypy: remove scmrepo from ignored modules

* Fix mypy issues

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: skshetry <[email protected]>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-01-28 14:26:00 +05:45
skshetry a137b2a6bc update ruff and enable flake8-logging-format and flake8-type-checking (#8894)
* update ruff and enable flake8-logging-format and flake8-type-checking check

* Update dvc/external_repo.py

* Update dvc/fs/dvc.py

* Update dvc/repo/experiments/__init__.py

* Update dvc/repo/experiments/queue/base.py

* Update pyproject.toml
2023-01-27 19:14:19 +05:45
daniele e1254c9ece Add misc type hints (#8430) 2023-01-15 17:58:34 +05:45
skshetry dba7d3c320 mypy: use typesafe cached_property (#8799) 2023-01-13 09:08:59 +05:45
skshetry 175c578579 ruff: ban relative parent/grandparent imports (#8794) 2023-01-11 16:48:46 +05:45
skshetry 5441964fdb add ruff (#8793)
* add ruff linter

* enable few more checks
2023-01-11 15:20:38 +05:45
karajan1001 eadd0da394 Refactor get_ref_and_entry_by_names
1. Move get_ref_and_entry_by_names from `exp remove` to `celery` for
   other commands usage.
2022-12-30 17:40:18 +08:00
David de la Iglesia Castro bb7512687c exp: Generate a human-readable name beforehand.
- Add `get_random_exp_name` to utils.
Generates a name (`f"{adjective}-{noun}"`) by randomly chosing pairs of adjetive, noun.
Recieves `scm` and `baseline_rev` and retries the random choice until a non-existing ref is found.

Closes https://github.com/iterative/dvc/issues/8650
2022-12-09 10:32:38 +01:00
David de la Iglesia Castro 8124396218 exp: Expose baseline and name via run_env.
Required by DVCLive for the experiments onboarding and live metrics features.

Related (not sure if closes): https://github.com/iterative/dvc/issues/6668
2022-11-28 13:13:04 +01:00
karajan1001 6bfce5a5db Use a more granular lock on exp operations.
For now in `experiment.new`, `init_executor` and `collect_exectuor` we
lock the whole scm repo. We can make a more granular lock in these
operations.
2022-11-28 17:04:16 +08:00
karajan1001 5fa33a06bc Move EXEC_TMP_DIR and EXEC_PID_DIR to experiment.utils 2022-11-28 17:04:16 +08:00
David de la Iglesia Castro 08a643e005 hydra: Raise lazy DvcException for Python >= 3.11 2022-11-03 12:14:18 +01:00
skshetry 6df70d388e update pre-commit; add some more hooks (#8492)
* update pre-commit; add some more hooks

* Apply suggestions from code review
2022-10-31 21:04:11 +05:45
karajan1001 9ef1d1d6e6 Move away tempdir related change from basequeue. 2022-10-29 08:53:02 +08:00
karajan1001 47ece317a5 Refactor: move get_running_exps to queue
Move logic of `get_running_exps` from `repo.experiments` to each queue.
2022-10-29 08:53:02 +08:00