Ruslan Kuprieiev
10184e9370
tests: gc: use timedelta ( #8912 )
...
Bug from https://github.com/iterative/dvc/pull/8880
2023-01-31 05:18:09 +02:00
98bcfad8e7
build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7 ( #8821 )
...
* build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7
Bumps [scmrepo](https://github.com/iterative/scmrepo ) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/iterative/scmrepo/releases )
- [Commits](https://github.com/iterative/scmrepo/compare/0.1.6...0.1.7 )
---
updated-dependencies:
- dependency-name: scmrepo
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <[email protected] [email protected] [email protected] [email protected]
2023-01-28 14:26:00 +05:45
skshetry
7ab4c59d70
test: run pytest-style checks ( #8901 )
2023-01-28 14:10:53 +05:45
skshetry
a137b2a6bc
update ruff and enable flake8-logging-format and flake8-type-checking ( #8894 )
...
* update ruff and enable flake8-logging-format and flake8-type-checking check
* Update dvc/external_repo.py
* Update dvc/fs/dvc.py
* Update dvc/repo/experiments/__init__.py
* Update dvc/repo/experiments/queue/base.py
* Update pyproject.toml
2023-01-27 19:14:19 +05:45
Daniele Trifirò
d294945499
gc: fix usage with --date
2023-01-27 12:52:46 +01:00
5165fb67a1
[pre-commit.ci] pre-commit autoupdate ( #8869 )
...
* [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.223 → v0.0.230](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.223...v0.0.230 )
* Update .pre-commit-config.yaml
* fix lint issues
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: skshetry <[email protected] [email protected]
2023-01-26 14:24:09 +05:45
Ruslan Kuprieiev
b46cfbda5d
deps: bump dvc-data to 0.36.0 ( #8885 )
2023-01-26 00:22:12 +02:00
skshetry
3427e115eb
data status: check if change.old.hash_info is None or not ( #8878 )
...
data:status: check if change.old.hash_info is None or not
2023-01-25 14:24:38 +02:00
David de la Iglesia Castro
80633f4fad
stage.save: Remove merge_versioned option.
...
If we want `add`, `commit` and `move` to work for cloud versioning, option must be always True so it is the same as not having it.
Closes #8828
(We were not passing `merge_versioned=True` in `dvc commit`)
2023-01-25 10:35:37 +01:00
David de la Iglesia Castro
ea2e5dc3e5
output: Don't serialize hash_info or meta in cloud versioning.
...
Closes #8357
2023-01-24 18:40:39 +01:00
Peter Rowlands
38e9b2ba95
deps: bump dvc-data to 0.35.1
2023-01-21 03:44:02 +09:00
David de la Iglesia Castro
5f558d003a
deps: bump dvc-data to 0.33.0
...
- Update `test_gc`
2023-01-18 15:17:07 +02:00
David de la Iglesia Castro
9af6fcad4a
exp run: Fix --dry option.
...
Closes #8831
2023-01-17 13:32:20 +01:00
skshetry
c47c2123fb
cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc ( #8822 )
...
* cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc.
* remove pylint direcitves
* Apply suggestions from code review
* Update pyproject.toml
2023-01-17 08:35:19 +05:45
Peter Rowlands
977a1f9f43
tests: update for dvc-objects batched transfer
2023-01-16 15:58:21 +09:00
skshetry
b0052aa666
pre-commit update: flake8/ruff/pyupgrade/black/isort ( #8813 )
2023-01-15 14:56:54 +05:45
skshetry
da27174e02
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups, etc. ( #8812 )
...
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups etc
2023-01-15 12:45:10 +05:45
karajan1001
fc1a192c90
Bump pathspec to 0.10.3
...
fix : #8217
1. Replace group name to avoid conflict.
2. Disable failed tests.
3. Bump pathspec to 0.10.3
2023-01-13 22:47:36 +08:00
dependabot[bot] and Saugat Pachhai
178244f666
build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10 ( #8807 )
...
* build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10
Bumps [dvc-task](https://github.com/iterative/dvc-task ) from 0.1.9 to 0.1.10.
- [Release notes](https://github.com/iterative/dvc-task/releases )
- [Commits](https://github.com/iterative/dvc-task/compare/0.1.9...0.1.10 )
---
updated-dependencies:
- dependency-name: dvc-task
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <[email protected] [email protected] [email protected]
2023-01-13 17:25:36 +05:45
David de la Iglesia Castro
8b075eb754
stage: Skip run-cache if any out has cache: false.
...
Closes https://github.com/iterative/dvc/issues/4428
2023-01-12 19:10:01 +01:00
Saugat Pachhai (सौगात)
30ec1cf5e9
index/stage: move repo collection logic to index
2023-01-11 19:38:43 +02:00
skshetry
a814f04ee0
tests: xfail test_apply_failed on JSONDecodeError/PermissionError ( #8796 )
2023-01-11 18:04:23 +05:45
skshetry
90a27179eb
ruff: ignore complex functions and remove dvc.utils.fs.makedirs ( #8795 )
...
We may want to prevent any new functions from getting complex.
2023-01-11 17:58:51 +05:45
skshetry
175c578579
ruff: ban relative parent/grandparent imports ( #8794 )
2023-01-11 16:48:46 +05:45
skshetry
5441964fdb
add ruff ( #8793 )
...
* add ruff linter
* enable few more checks
2023-01-11 15:20:38 +05:45
Ivan Shcheklein
d4bfe38a54
perf: remove fs exists check in plots, parallel data collect
2023-01-10 20:27:02 -08:00
skshetry
d87cdba6d8
gha: remove env vars SHELL
2023-01-10 10:19:29 +05:45
dberenbaum
0b192af6f4
params: fix diff bt active branch and workspace
2023-01-09 16:57:18 +02:00
dberenbaum
c6b602a167
metrics: fix diff bt active branch and workspace
2023-01-09 16:57:18 +02:00
David de la Iglesia Castro
1d5de9c5ba
repro: Disable run-cache if --no-commit.
...
As described in https://github.com/iterative/dvc/issues/4428#issuecomment-778263005 .
Add `run_cache` argument to `Stage.save`. Set it to `False` if `no_commit`.
2023-01-09 12:38:57 +01:00
Ruslan Kuprieiev
684fc32e36
dvcfs: ls: don't call info()
...
Cuts down `dvc ls . data/mnist/dataset --recursive` for me from ~37 to ~25
seconds.
Note that this also changes behaviour for ls with a broken symlink, as we
were not able to see it before because of a failing `info()` call, but now
we can follow a normal UNIX ls behaviour more closely.
2023-01-08 00:37:40 +02:00
Saugat Pachhai (सौगात)
3dfdd577e7
tests: xfail experiments-related flakey tests
2023-01-06 18:59:53 +05:45
dberenbaum
6ac7dfc008
exp: fix tests and remove reset checks
2023-01-04 17:29:12 +01:00
dberenbaum
b0372918ea
exp: stop checking for unchanged experiments
2023-01-04 17:29:12 +01:00
Daniele Trifirò
4b0fa4d559
update: prevent re-download on every update
2023-01-03 18:52:48 +01:00
Saugat Pachhai (सौगात)
a24c389674
internal: hoist resolver/top-level metrics/plots/params data into file level
2023-01-03 17:46:06 +05:45
karajan1001
f386f7c082
exp apply can accept exp names and failed exps as target
...
fix : #8182
1. `exp apply` can apply to failed experiments.
2. `exp apply` can accept exp names as target.
3. Add a new functional test for it.
2022-12-30 17:40:18 +08:00
David de la Iglesia Castro
68b24307f9
post_live_metrics: Pass params on start event.
2022-12-28 11:43:37 +01:00
karajan1001
08ae68caf4
Make temp dir executed checkpoint experiment return result to workspace.
...
fix : #8612
For checkpoint experiment in some case users might want to give it a early
stopping to reduce variance. But currently if we interrupt/kill the
experiment it will be marked as failed, and all of the completed
checkpoints will be removed as we cleanup all the running directly after
the process failed.
1. We raise CheckpointKilledError intead of StageCmdFailed error if at
least one checkpoint had been commited.
2. Temp_dir executor will continue collecting data if the Checkpoint
stage was interrupted.
3. Raise warnings if a checkpoint stage was incomplete and the other
stages were not forwarded.
4. Add a new functional test for this
2022-12-26 18:25:43 +05:45
David de la Iglesia Castro
b13184fcec
tests: Fix flaky test_show_only_changed.
...
Some of the random exp names could start with `bar`.
https://github.com/iterative/dvc/actions/runs/3757008396/jobs/6383731723#step:5:79
2022-12-22 13:01:46 +01:00
David de la Iglesia Castro
d7fd68a1ae
Add start and done calls with post_live_metrics.
2022-12-22 11:33:59 +01:00
Daniele Trifirò
59d6d5ea98
exp save: simplify implementation
2022-12-22 11:01:41 +01:00
Saugat Pachhai (सौगात)
23e4d376c9
index: remove unused APIs
...
These are being used by some users, but removing it for easier refactoring.
We are not using it ourselves.
2022-12-20 13:17:28 +05:45
David de la Iglesia Castro
bb7512687c
exp: Generate a human-readable name beforehand.
...
- Add `get_random_exp_name` to utils.
Generates a name (`f"{adjective}-{noun}"`) by randomly chosing pairs of adjetive, noun.
Recieves `scm` and `baseline_rev` and retries the random choice until a non-existing ref is found.
Closes https://github.com/iterative/dvc/issues/8650
2022-12-09 10:32:38 +01:00
Saugat Pachhai (सौगात)
334f297ad3
deps: remove 3.11 checks for hydra; has 3.11 support now
2022-12-09 10:32:12 +05:45
Saugat Pachhai (सौगात)
28402dedf1
add tests
2022-12-01 15:56:26 +01:00
David de la Iglesia Castro
a036eaf817
exp: save: Add include_untracked.
...
Allow to include a list of potentially untracked files.
Covers the DVCLive use case where in the first run the generated files won't be tracked in Git yet.
2022-11-28 15:49:34 +01:00
David de la Iglesia Castro
665185b9e7
exp: save: Handle staged changes.
...
Staged changes were causing a merge error.
Warn and unstage instead of erroring, to match exp run behavior.
2022-11-28 15:49:34 +01:00
David de la Iglesia Castro
43e281521e
exp save: cleanup.
...
Remove metrics.
Move tests to separate file.
Fix `experiments.get_exact_name` usage.
Remove unused checkpoint logic.
2022-11-28 15:49:34 +01:00
Daniele Trifirò
73b46a5789
exp save: initial implementation
2022-11-28 15:49:34 +01:00
Ruslan Kuprieiev
Daniele Trifirò
David de la Iglesia Castro
Peter Rowlands
karajan1001
Ivan Shcheklein
dberenbaum