Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-10 23:10:42 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,354 Commits 7 Branches 579 Tags
73aea2d972750ab4d9562712f2a4f9201ccc9252
Commit Graph
2938 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 10184e9370 tests: gc: use timedelta (#8912) 
Bug from https://github.com/iterative/dvc/pull/8880
 2023-01-31 05:18:09 +02:00
dependabot[bot]skshetrySaugat Pachhai
 		98bcfad8e7 build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7 (#8821) 
* build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7

Bumps [scmrepo](https://github.com/iterative/scmrepo) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/iterative/scmrepo/releases)
- [Commits](https://github.com/iterative/scmrepo/compare/0.1.6...0.1.7)

---
updated-dependencies:
- dependency-name: scmrepo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* mypy: remove scmrepo from ignored modules

* Fix mypy issues

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: skshetry <[email protected]>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-01-28 14:26:00 +05:45
skshetry 7ab4c59d70 test: run pytest-style checks (#8901) 2023-01-28 14:10:53 +05:45
skshetry a137b2a6bc update ruff and enable flake8-logging-format and flake8-type-checking (#8894) 
* update ruff and enable flake8-logging-format and flake8-type-checking check

* Update dvc/external_repo.py

* Update dvc/fs/dvc.py

* Update dvc/repo/experiments/__init__.py

* Update dvc/repo/experiments/queue/base.py

* Update pyproject.toml
 2023-01-27 19:14:19 +05:45
Daniele Trifirò 6f0e8a8db5 tests: fix test_iter_revs mock 2023-01-27 12:52:46 +01:00
Daniele Trifirò d294945499 gc: fix usage with --date 2023-01-27 12:52:46 +01:00
Dave Berenbaum db70d1cc5c analytics: differentiate ci users (#8838) 2023-01-26 21:50:14 +05:45
David de la Iglesia Castro 12dd746165 output: Pass version_aware=True to fs if files is present. 
This will be ignored by filesystems that don't support `version_aware`.

Closes #8853
 2023-01-26 10:48:59 +01:00
pre-commit-ci[bot]skshetrySaugat Pachhai
 		5165fb67a1 [pre-commit.ci] pre-commit autoupdate (#8869) 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.223 → v0.0.230](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.223...v0.0.230)

* Update .pre-commit-config.yaml

* fix lint issues

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: skshetry <[email protected]>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-01-26 14:24:09 +05:45
Ruslan Kuprieiev b46cfbda5d deps: bump dvc-data to 0.36.0 (#8885) 2023-01-26 00:22:12 +02:00
Dave Berenbaum a8d193bf2a queue: update kill ui messages (#8798) 
* queue: update kill ui messages

* queue kill: update ui, kill all valid tasks

* queue kill: update wording of ui
 2023-01-25 20:08:27 +05:45
skshetry 3427e115eb data status: check if change.old.hash_info is None or not (#8878) 
data:status: check if change.old.hash_info is None or not
 2023-01-25 14:24:38 +02:00
David de la Iglesia Castro 80633f4fad stage.save: Remove merge_versioned option. 
If we want `add`, `commit` and `move` to work for cloud versioning, option must be always True so it is the same as not having it.

Closes #8828
(We were not passing `merge_versioned=True` in `dvc commit`)
 2023-01-25 10:35:37 +01:00
David de la Iglesia Castro ea2e5dc3e5 output: Don't serialize hash_info or meta in cloud versioning. 
Closes #8357
 2023-01-24 18:40:39 +01:00
Peter Rowlands 38e9b2ba95 deps: bump dvc-data to 0.35.1 2023-01-21 03:44:02 +09:00
David de la Iglesia CastroandPeter Rowlands dd2d2dce19 param: Fix status check for Tuples. 
Closes #8803

Update dvc/dependency/param.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
 2023-01-19 08:19:16 +01:00
David de la Iglesia Castro 5f558d003a deps: bump dvc-data to 0.33.0 
- Update `test_gc`
 2023-01-18 15:17:07 +02:00
David de la Iglesia Castro 9af6fcad4a exp run: Fix --dry option. 
Closes #8831
 2023-01-17 13:32:20 +01:00
skshetry c47c2123fb cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc (#8822) 
* cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc.

* remove pylint direcitves

* Apply suggestions from code review

* Update pyproject.toml
 2023-01-17 08:35:19 +05:45
Peter Rowlands 977a1f9f43 tests: update for dvc-objects batched transfer 2023-01-16 15:58:21 +09:00
Peter Rowlands e48d0d1d9c tests: remove duplicated dvc-objects/dvc-data tests 2023-01-16 15:58:21 +09:00
skshetry b0052aa666 pre-commit update: flake8/ruff/pyupgrade/black/isort (#8813) 2023-01-15 14:56:54 +05:45
skshetry da27174e02 dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups, etc. (#8812) 
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups etc
 2023-01-15 12:45:10 +05:45
karajan1001 fc1a192c90 Bump pathspec to 0.10.3 
fix: #8217
1. Replace group name to avoid conflict.
2. Disable failed tests.
3. Bump pathspec to 0.10.3
 2023-01-13 22:47:36 +08:00
dependabot[bot]andSaugat Pachhai 178244f666 build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10 (#8807) 
* build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10

Bumps [dvc-task](https://github.com/iterative/dvc-task) from 0.1.9 to 0.1.10.
- [Release notes](https://github.com/iterative/dvc-task/releases)
- [Commits](https://github.com/iterative/dvc-task/compare/0.1.9...0.1.10)

---
updated-dependencies:
- dependency-name: dvc-task
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* remove xfail on json.JSONDecodeError/PermissionError

JSONDecodeError should be fixed by the new release. Next we need to investigate
PermissionError.

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-01-13 17:25:36 +05:45
David de la Iglesia Castro 8b075eb754 stage: Skip run-cache if any out has cache: false. 
Closes https://github.com/iterative/dvc/issues/4428
 2023-01-12 19:10:01 +01:00
David de la Iglesia Castro f507e4d9c0 config: Reraise ConfigError from ConfigObjError. 
Closes #8801
 2023-01-12 18:11:54 +01:00
Saugat Pachhai (सौगात) 30ec1cf5e9 index/stage: move repo collection logic to index 2023-01-11 19:38:43 +02:00
skshetry a814f04ee0 tests: xfail test_apply_failed on JSONDecodeError/PermissionError (#8796) 2023-01-11 18:04:23 +05:45
skshetry 90a27179eb ruff: ignore complex functions and remove dvc.utils.fs.makedirs (#8795) 
We may want to prevent any new functions from getting complex.
 2023-01-11 17:58:51 +05:45
skshetry 175c578579 ruff: ban relative parent/grandparent imports (#8794) 2023-01-11 16:48:46 +05:45
skshetry 5441964fdb add ruff (#8793) 
* add ruff linter

* enable few more checks
 2023-01-11 15:20:38 +05:45
Ivan Shcheklein d4bfe38a54 perf: remove fs exists check in plots, parallel data collect 2023-01-10 20:27:02 -08:00
skshetry d87cdba6d8 gha: remove env vars SHELL 2023-01-10 10:19:29 +05:45
dberenbaum 0b192af6f4 params: fix diff bt active branch and workspace 2023-01-09 16:57:18 +02:00
dberenbaum c6b602a167 metrics: fix diff bt active branch and workspace 2023-01-09 16:57:18 +02:00
David de la Iglesia Castro 1d5de9c5ba repro: Disable run-cache if --no-commit. 
As described in https://github.com/iterative/dvc/issues/4428#issuecomment-778263005.

Add `run_cache` argument to `Stage.save`. Set it to `False` if `no_commit`.
 2023-01-09 12:38:57 +01:00
Ruslan Kuprieiev 684fc32e36 dvcfs: ls: don't call info() 
Cuts down `dvc ls . data/mnist/dataset --recursive` for me from ~37 to ~25
seconds.

Note that this also changes behaviour for ls with a broken symlink, as we
were not able to see it before because of a failing `info()` call, but now
we can follow a normal UNIX ls behaviour more closely.
 2023-01-08 00:37:40 +02:00
Saugat Pachhai (सौगात) 3dfdd577e7 tests: xfail experiments-related flakey tests 2023-01-06 18:59:53 +05:45
Daniele Trifirò b62f17ffaf tests: move tests/unit/remote/test_http.py to dvc-http repo 2023-01-05 19:54:14 +01:00
dberenbaum 6ac7dfc008 exp: fix tests and remove reset checks 2023-01-04 17:29:12 +01:00
dberenbaum b0372918ea exp: stop checking for unchanged experiments 2023-01-04 17:29:12 +01:00
Daniele Trifirò 4b0fa4d559 update: prevent re-download on every update 2023-01-03 18:52:48 +01:00
Saugat Pachhai (सौगात) a24c389674 internal: hoist resolver/top-level metrics/plots/params data into file level 2023-01-03 17:46:06 +05:45
karajan1001 f386f7c082 exp apply can accept exp names and failed exps as target 
fix: #8182

1. `exp apply` can apply to failed experiments.
2. `exp apply` can accept exp names as target.
3. Add a new functional test for it.
 2022-12-30 17:40:18 +08:00
karajan1001 eadd0da394 Refactor get_ref_and_entry_by_names 
1. Move get_ref_and_entry_by_names from `exp remove` to `celery` for
   other commands usage.
 2022-12-30 17:40:18 +08:00
karajan1001 06673dd212 Use SIGINT as the default signal in queue kill 
fix: #8624
1. Add a new flag `--force` for `queue kill`
2. Make `SIGINT` to be the default option and `SIGKILL` to be with
   `--force`
3. Add tests for `queue kill`
4. Bump dvc-task into 0.1.9
 2022-12-30 16:19:39 +08:00
David de la Iglesia Castro 68b24307f9 post_live_metrics: Pass params on start event. 2022-12-28 11:43:37 +01:00
karajan1001 08ae68caf4 Make temp dir executed checkpoint experiment return result to workspace. 
fix: #8612
For checkpoint experiment in some case users might want to give it a early
stopping to reduce variance.  But currently if we interrupt/kill the
experiment it will be marked as failed, and all of the completed
checkpoints will be removed as we cleanup all the running directly after
the process failed.

1. We raise CheckpointKilledError intead of StageCmdFailed error if at
   least one checkpoint had been commited.
2. Temp_dir executor will continue collecting data if the Checkpoint
   stage was interrupted.
3. Raise warnings if a checkpoint stage was incomplete and the other
   stages were not forwarded.
4. Add a new functional test for this
 2022-12-26 18:25:43 +05:45
karajan1001 4400fb4090 queue stop --kill can kill false running status tasks 
fix: #8702

1. replace simple `proc.kill` in `queue stop --kill` with the same logic as in
   `queue kill` to make it possible to set false status `running` tasks
    as faileed.
2. Modified the tests.
 2022-12-26 17:16:46 +09:00