Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,281 Commits 7 Branches 579 Tags
5441964fdbece480e1e8075b549db1f76200a32c
Commit Graph
8281 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
skshetry 5441964fdb add ruff (#8793) 
* add ruff linter

* enable few more checks
 2023-01-11 15:20:38 +05:45
Ivan Shcheklein d4bfe38a54 perf: remove fs exists check in plots, parallel data collect 2023-01-10 20:27:02 -08:00
dependabot[bot] a060049186 build(deps-dev): unpin grandalf 
See https://github.com/bdcht/grandalf/issues/26#issuecomment-1378175273.
 2023-01-11 08:44:49 +05:45
Ruslan Kuprieiev c02285ab49 deps: bump dvc-data to 0.29.0 2.41.1 2023-01-11 03:03:38 +02:00
Jorge Orpinel bccbbc1d94 Update dvc/commands/imp_url.py 2.41.0 2023-01-10 19:05:18 +02:00
Jorge Orpinel d736b31d8b import-url: Update options order 
Per https://github.com/iterative/dvc.org/pull/3934
 2023-01-10 19:05:18 +02:00
Peter Rowlands 68c1cc095c deps: bump dvc-data to 0.28.6 2023-01-11 00:20:22 +09:00
Peter Rowlands c94d28594d worktree push: remove existing version id optimization for file outs 2023-01-10 16:21:31 +09:00
Peter Rowlands d3d429ef7f push/fetch: respect --jobs on worktree push/fetch 2023-01-10 16:21:31 +09:00
dependabot[bot] 6ac88104da build(deps-dev): Bump pylint from 2.15.9 to 2.15.10 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.15.9 to 2.15.10.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.15.9...v2.15.10)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2023-01-10 10:39:48 +05:45
skshetry d87cdba6d8 gha: remove env vars SHELL 2023-01-10 10:19:29 +05:45
Ruslan Kuprieiev b25ef5cdfb metrics/params: show: add hide_workspace flag 2023-01-09 16:57:18 +02:00
dberenbaum 435407185f params/metrics diff: address test failures for missing branch key 2023-01-09 16:57:18 +02:00
dberenbaum 0b192af6f4 params: fix diff bt active branch and workspace 2023-01-09 16:57:18 +02:00
dberenbaum c6b602a167 metrics: fix diff bt active branch and workspace 2023-01-09 16:57:18 +02:00
David de la Iglesia Castro 1d5de9c5ba repro: Disable run-cache if --no-commit. 
As described in https://github.com/iterative/dvc/issues/4428#issuecomment-778263005.

Add `run_cache` argument to `Stage.save`. Set it to `False` if `no_commit`.
 2023-01-09 12:38:57 +01:00
Ruslan Kuprieiev 0e5674ffea ls: compute relparts once for every dir 
Cuts down `dvc ls . data/mnist/dataset --recursive` for me from ~25 to ~12
seconds.

`relparts` is an expensive operation as it calls `relpath` that calls `abspath`,
so we want to avoid calling it as much as possible.
2.40.0 		2023-01-08 03:44:50 +02:00
Ruslan Kuprieiev 684fc32e36 dvcfs: ls: don't call info() 
Cuts down `dvc ls . data/mnist/dataset --recursive` for me from ~37 to ~25
seconds.

Note that this also changes behaviour for ls with a broken symlink, as we
were not able to see it before because of a failing `info()` call, but now
we can follow a normal UNIX ls behaviour more closely.
 2023-01-08 00:37:40 +02:00
Saugat Pachhai (सौगात) 81f37be9d6 ci: timeout tests if it takes >5min 
It most likely means it's frozen and unlikely to resume again. We'd want
to see where it froze, so this will provide us with a traceback.
2.39.0 		2023-01-06 20:16:20 +05:45
Saugat Pachhai (सौगात) 3dfdd577e7 tests: xfail experiments-related flakey tests 2023-01-06 18:59:53 +05:45
dependabot[bot] bfb6a32b51 build(deps-dev): Bump dvc-data from 0.28.4 to 0.28.5 
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.28.4 to 0.28.5.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.28.4...0.28.5)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2023-01-06 16:36:48 +09:00
Daniele Trifirò 33de761560 deps: unpin dvc-http 2023-01-05 21:15:33 +01:00
Daniele Trifirò ff38aed927 config: expose timeouts in http config 
- `read_timeout` timeout for reading new data portions from the peer
- `connect_timeout` timeout for establishing new connections

For more information about individual timeouts meanings:

- `read_timeout` maps to `aiohttp`'s `sock_read` timeout
- `connect_timeout` applies to `aiohttp`'s `connect`, `sock_connect`
  timeouts

https://docs.aiohttp.org/en/stable/client_reference.html?highlight=clienttimeout#clienttimeout
 2023-01-05 21:15:33 +01:00
Daniele Trifirò b62f17ffaf tests: move tests/unit/remote/test_http.py to dvc-http repo 2023-01-05 19:54:14 +01:00
Daniele Trifirò a0c781a676 exp save: fix help for --force 2023-01-04 17:57:29 +01:00
dberenbaum 6ac7dfc008 exp: fix tests and remove reset checks 2023-01-04 17:29:12 +01:00
dberenbaum b0372918ea exp: stop checking for unchanged experiments 2023-01-04 17:29:12 +01:00
Daniele Trifirò 4b0fa4d559 update: prevent re-download on every update 2023-01-03 18:52:48 +01:00
Saugat Pachhai (सौगात) a24c389674 internal: hoist resolver/top-level metrics/plots/params data into file level 2023-01-03 17:46:06 +05:45
karajan1001 f386f7c082 exp apply can accept exp names and failed exps as target 
fix: #8182

1. `exp apply` can apply to failed experiments.
2. `exp apply` can accept exp names as target.
3. Add a new functional test for it.
 2022-12-30 17:40:18 +08:00
karajan1001 eadd0da394 Refactor get_ref_and_entry_by_names 
1. Move get_ref_and_entry_by_names from `exp remove` to `celery` for
   other commands usage.
 2022-12-30 17:40:18 +08:00
karajan1001 06673dd212 Use SIGINT as the default signal in queue kill 
fix: #8624
1. Add a new flag `--force` for `queue kill`
2. Make `SIGINT` to be the default option and `SIGKILL` to be with
   `--force`
3. Add tests for `queue kill`
4. Bump dvc-task into 0.1.9
 2022-12-30 16:19:39 +08:00
dependabot[bot] 5f0b1edb40 build(deps-dev): Bump filelock from 3.8.2 to 3.9.0 
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.8.2 to 3.9.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Changelog](https://github.com/tox-dev/py-filelock/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.8.2...3.9.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-12-30 09:20:26 +05:45
dependabot[bot] 05701e26db build(deps-dev): Bump scmrepo from 0.1.4 to 0.1.5 
Bumps [scmrepo](https://github.com/iterative/scmrepo) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/iterative/scmrepo/releases)
- [Commits](https://github.com/iterative/scmrepo/compare/0.1.4...0.1.5)

---
updated-dependencies:
- dependency-name: scmrepo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-12-30 11:23:42 +09:00
Daniele Trifirò 109288e1cd exp save: update --help 2022-12-29 20:51:52 +01:00
Daniele Trifirò 9e6f44bc1c get/ls/import: improve clone error messages 2022-12-29 20:13:39 +01:00
David de la Iglesia Castro 68b24307f9 post_live_metrics: Pass params on start event. 2022-12-28 11:43:37 +01:00
skshetry ea8a4a3db4 GHA: remove unused env from tests workflow 2022-12-27 10:26:29 +05:45
karajan1001 08ae68caf4 Make temp dir executed checkpoint experiment return result to workspace. 
fix: #8612
For checkpoint experiment in some case users might want to give it a early
stopping to reduce variance.  But currently if we interrupt/kill the
experiment it will be marked as failed, and all of the completed
checkpoints will be removed as we cleanup all the running directly after
the process failed.

1. We raise CheckpointKilledError intead of StageCmdFailed error if at
   least one checkpoint had been commited.
2. Temp_dir executor will continue collecting data if the Checkpoint
   stage was interrupted.
3. Raise warnings if a checkpoint stage was incomplete and the other
   stages were not forwarded.
4. Add a new functional test for this
 2022-12-26 18:25:43 +05:45
karajan1001 4400fb4090 queue stop --kill can kill false running status tasks 
fix: #8702

1. replace simple `proc.kill` in `queue stop --kill` with the same logic as in
   `queue kill` to make it possible to set false status `running` tasks
    as faileed.
2. Modified the tests.
 2022-12-26 17:16:46 +09:00
Fabian Zills 28848c5407 exp run --run-all: re-enable --jobs concurrency (#8608) 
* re-enable --jobs concurrency

* use `start_workers` instead of `spawn_worker`

* make `spawn_worker` a private method.

* update tests: make `spawn_worker` a private method.
 2022-12-26 16:49:46 +09:00
David de la Iglesia Castro b13184fcec tests: Fix flaky test_show_only_changed. 
Some of the random exp names could start with `bar`.

https://github.com/iterative/dvc/actions/runs/3757008396/jobs/6383731723#step:5:79
 2022-12-22 13:01:46 +01:00
dependabot[bot] ee5fbc72ab build(deps-dev): Bump dvc-render from 0.0.15 to 0.0.17 
Bumps [dvc-render](https://github.com/iterative/dvc-render) from 0.0.15 to 0.0.17.
- [Release notes](https://github.com/iterative/dvc-render/releases)
- [Commits](https://github.com/iterative/dvc-render/compare/0.0.15...0.0.17)

---
updated-dependencies:
- dependency-name: dvc-render
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-12-22 12:17:56 +01:00
David de la Iglesia Castro d7fd68a1ae Add start and done calls with post_live_metrics. 2022-12-22 11:33:59 +01:00
Daniele Trifirò 59d6d5ea98 exp save: simplify implementation 2022-12-22 11:01:41 +01:00
dberenbaum 6fc0a6ff98 plots: accept any number of y data sources 2022-12-20 14:32:40 +01:00
dberenbaum b4b5db4b4e plots: accept list of y vals 2022-12-20 14:32:40 +01:00
Saugat Pachhai (सौगात) 23e4d376c9 index: remove unused APIs 
These are being used by some users, but removing it for easier refactoring.
We are not using it ourselves.
 2022-12-20 13:17:28 +05:45
dependabot[bot] 089a90c969 build(deps-dev): Bump pylint from 2.15.8 to 2.15.9 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.15.8 to 2.15.9.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.15.8...v2.15.9)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-12-19 11:28:58 +05:45
dependabot[bot] 198daec421 build(deps): Bump pyinstaller from 5.6.2 to 5.7.0 
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 5.6.2 to 5.7.0.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/develop/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v5.6.2...v5.7.0)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-12-15 19:05:41 +05:45