Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
9,108 Commits 7 Branches 579 Tags
f3fa67440d73ceadcc73abfe3d6aff5d0a1a4467
Commit Graph
9108 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Peter Rowlands 20606d480b exp show: drop internal legacy checkpoint support 2023-09-06 14:58:53 +09:00
dependabot[bot] e2aedb7e97 build(deps): bump actions/checkout from 3 to 4 (#9911) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
3.18.0 		2023-09-05 08:43:43 +05:45
pre-commit-ci[bot] bacae48f96 [pre-commit.ci] pre-commit autoupdate (#9910) 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.286 → v0.0.287](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.286...v0.0.287)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-09-05 08:34:38 +05:45
Peter Rowlands bf119e70a9 exp run: track files for skipped/run-cached stages 2023-09-04 16:23:54 +09:00
BradyJ27 3528c37b66 move: Alias mv to move (#9905) 
* Alias mv to move

* Add alias for mv

* Add aliases for rm, exp rm, exp ls
 2023-09-03 22:11:41 +03:00
Ruslan Kuprieiev f47d21b0c2 [WIP] ls-url/get-url/import-url: introduce --fs-config (#9888) 
ls-url/get-url/import-url: introduce --fs-config

Fixes #9886
 2023-09-01 02:17:17 +03:00
Ruslan Kuprieiev 7a9da0f4a1 cli: utils: DictAction: limit split to 2 elements (#9887) 2023-08-30 03:15:21 +03:00
Ruslan Kuprieiev 8bfe8e3328 site_cache_dir: add salt to token (#9884) 
See comment in the code.
3.17.0 		2023-08-29 17:48:38 +03:00
Ruslan Kuprieiev c290d5c008 repo: don't try to gitignore custom cache locations (#9883) 
If user uses some custom cache location, there is really no reason for us
to try to gitignore it, as we have no idea what he is planning to do with
that and overall it often might be sign of this being misused. Maybe we could
also have some warnings/errors, but that's out of scope for this PR.

Fixes https://github.com/iterative/studio/issues/7405
 2023-08-29 16:31:37 +03:00
Matt Seddon 3c2c1fe72a Remove dead Stage method (#9881) 2023-08-29 13:10:40 +00:00
pre-commit-ci[bot] 0602eca00e [pre-commit.ci] pre-commit autoupdate (#9882) 
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.285 → v0.0.286](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.285...v0.0.286)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-08-29 07:17:32 +05:45
Matt Seddon 5b69a16ea7 Add DVC_ROOT env var into queued experiment runs (#9879) 2023-08-28 22:37:03 +00:00
Dave Berenbaumandpre-commit-ci[bot] ef260eaf2e exp show: show metrics that include separator (#9819) 
* exp show: show metrics that include separator

* sort using partition instead of lpartition

* handle ':' in file and metric names

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix sorting with leading :

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-08-24 10:31:56 -04:00
Peter Rowlands 88431b1c0a update: preserve def_path for version-aware import update 2023-08-24 16:16:43 +09:00
dependabot[bot] bd2dc9dfed build(deps): bump rtCamp/action-slack-notify from 2.2.0 to 2.2.1 (#9874) 
Bumps [rtCamp/action-slack-notify](https://github.com/rtcamp/action-slack-notify) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/rtcamp/action-slack-notify/releases)
- [Commits](https://github.com/rtcamp/action-slack-notify/compare/v2.2.0...v2.2.1)

---
updated-dependencies:
- dependency-name: rtCamp/action-slack-notify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-24 08:30:32 +05:45
Dave Berenbaumandpre-commit-ci[bot] 428c094b8d update msg for multiple overlapping outputs (#9862) 
* update msg for multiple overlapping outputs

* update msg for multiple overlapping outputs

* fix type checking

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* use sets of stages to fix mypy errors

* fix tests/func/test_add.py::test_try_adding_multiple_overlaps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-08-24 00:06:32 +03:00
Dave Berenbaum fc955c1917 Rename partial add/remove benchmarks (#9868) 
* Rename partial add/remove benchmarks

* Update dvc/testing/benchmarks/cli/stories/test_modify_data.py
 2023-08-23 19:04:51 +05:45
skshetry 8cb7e4948d tests: dvcfs: use fspath while testing (#9872) 
get_file does not support PathLike objects. fsspec does not support them in get_file.

See https://github.com/fsspec/filesystem_spec/issues/896.
Fixes test failures in dvc-webhdfs and dvc-hdfs.
 2023-08-23 17:33:22 +05:45
skshetry 8e353f9fce deps: unpin dvc-oss (#9871) 2023-08-23 17:20:41 +05:45
dependabot[bot] a4e2bce5b2 build(deps-dev): bump dvc-oss from 2.19 to 2.20.0 (#9870) 
Bumps [dvc-oss](https://github.com/iterative/dvc-oss) from 2.19 to 2.20.0.
- [Release notes](https://github.com/iterative/dvc-oss/releases)
- [Commits](https://github.com/iterative/dvc-oss/compare/2.19.0...2.20.0)

---
updated-dependencies:
- dependency-name: dvc-oss
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-23 08:45:43 +05:45
Ruslan Kuprieiev 736ca9a5e5 cache: check-missing: add type, oid name and oid 3.16.0 2023-08-23 02:07:32 +03:00
Ruslan Kuprieiev cae1b1b647 ls/ls-url: use tqmd instead of humanize.naturalsize 
Per https://github.com/iterative/dvc/pull/9857#issuecomment-1683185006
 2023-08-22 20:21:08 +03:00
pre-commit-ci[bot]andSaugat Pachhai 3783d65dde [pre-commit.ci] pre-commit autoupdate (#9841) 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.282 → v0.0.285](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.282...v0.0.285)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix ruff issues

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-08-22 15:22:57 +05:45
skshetry 0342e4f1c3 fix type hints (#9863) 2023-08-22 14:56:11 +05:45
skshetry fc9af7a976 assert that the test is getting parametrized correctly (#9859) 
Otherwise this may end up generating 0 or not enough commands, and passing tests
 2023-08-19 13:57:48 +05:45
skshetry fea59cfc52 tests: simplify test parametrization (#9858) 
i.e. remove metafunc/pytest_generate_tests
 2023-08-19 10:02:31 +05:45
Ruslan Kuprieiev 31ffce4fba ls/ls-url: use humanize.naturalsize instead of tqdm 
Currently we get odd formatting like

```
456    .devcontainer.json
139    .dvcignore
61.0   .gitattributes
96.0   .github
7.00   .gitignore
5.76k  README.md
160    data
1.84k  dvc.lock
519    dvc.yaml
4.00   foo
96.0   models
96.0   notebooks
225    params.yaml
46.0   requirements.txt
128    results
160    src
```

(notice stuff like 46.0)

and there doesn't seem to be a simple way to fix that in tqdm, in addition to
the general feeling that we are abusing tqdm's util for this.

So let's just use `humanize`, which gets us a more pleasing result:

```
456B  .devcontainer.json
139B  .dvcignore
61B   .gitattributes
96B   .github
7B    .gitignore
5.8K  README.md
160B  data
1.8K  dvc.lock
519B  dvc.yaml
4B    foo
96B   models
96B   notebooks
225B  params.yaml
46B   requirements.txt
128B  results
160B  src
```
 2023-08-18 02:54:31 +03:00
dependabot[bot] 2da959acd1 build(deps-dev): update pytest-docker requirement from <2,>=1 to >=1,<3 
Updates the requirements on [pytest-docker](https://github.com/avast/pytest-docker) to permit the latest version.
- [Release notes](https://github.com/avast/pytest-docker/releases)
- [Changelog](https://github.com/avast/pytest-docker/blob/master/CHANGELOG.md)
- [Commits](https://github.com/avast/pytest-docker/compare/v1.0.0...v2.0.0)

---
updated-dependencies:
- dependency-name: pytest-docker
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
 2023-08-18 00:41:47 +03:00
Ruslan Kuprieiev 72a42fce20 Update local.py 2023-08-18 00:31:52 +03:00
Dave BerenbaumandRuslan Kuprieiev 1cbe64face Update tests/unit/command/test_experiments.py 
Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2023-08-18 00:31:52 +03:00
dberenbaum 057b0112ea exp: make run/save less noisy 2023-08-18 00:31:52 +03:00
Ruslan Kuprieiev 1805edbd32 import: set core.remote when both --remote and --remote-config are specified 
From https://github.com/iterative/dvc/pull/9853#issuecomment-1682797083
 2023-08-18 00:16:33 +03:00
Ruslan Kuprieiev 1723abcd79 ls/ls-url: introduce --size 
I really need it only for studio right now, but this is really nice to have in a daily life
as well.

E.g.

```
$ dvc ls-url . --size
108    Metadata-and-Results.dvc
97     AudioMP3.dvc
97     AudioWAV.dvc
12181  README.md
54     .gitignore
168    .gitattributes
99     VideoFlash.dvc
312    LICENSE.txt
139    .dvcignore
96     AudioMP3
96     Asset
192    .dvc
64     Metadata-and-Results
64     AudioWAV
192    Scripts
64     VideoFlash
384    .git
96     .vscode
```

```
$ dvc list . --size
139    .dvcignore
168    .gitattributes
54     .gitignore
96     .vscode
96     Asset
96     AudioMP3
97     AudioMP3.dvc
64     AudioWAV
97     AudioWAV.dvc
312    LICENSE.txt
64     Metadata-and-Results
108    Metadata-and-Results.dvc
12181  README.md
192    Scripts
64     VideoFlash
99     VideoFlash.dvc
```
 2023-08-18 00:08:48 +03:00
Ruslan Kuprieiev 3081c60414 dep: repo: handle dict as config (#9853) 2023-08-17 19:37:04 +03:00
dependabot[bot] 152991d878 build(deps-dev): bump mypy from 1.4.1 to 1.5.1 (#9852) 
Bumps [mypy](https://github.com/python/mypy) from 1.4.1 to 1.5.1.
- [Commits](https://github.com/python/mypy/compare/v1.4.1...v1.5.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-17 18:43:31 +05:45
skshetry 8098fd109d plots: fix caching of legacy format (#9849) 
* plots: fix caching of legacy format

* deps: bump dvc-data minversion to 2.15.0
 2023-08-17 07:40:07 +05:45
Ruslan Kuprieiev f0c1ff0219 fs: make _prepare_credentails/fs_args lazy (#9848) 
* tests: unit: move s3 tests to dvc-s3

* deps: bump dvc-data to 2.14.0

* tests: unit: move azure tests to dvc-azure
3.15.3 		2023-08-16 05:51:20 +03:00
Ruslan Kuprieiev 27c548575c pyinstaller: declare dvc_ssh as hidden import (#9840) 
Likely with removal of dvc-machine pyinstaller stopped picking up dvc_ssh automatically, so we need to declare it explicitly just like the rest of plugins.
3.15.2 		2023-08-13 00:26:43 +03:00
Dave Berenbaum 79cc28501b don't unprotect outs if not cached (#9838) 
* don't unprotect outs if not cached

* fix tests
3.15.1 		2023-08-11 18:12:25 +03:00
Peter Rowlands a051b8b60f remove support for dvc machine 3.15.0 2023-08-11 14:42:52 +09:00
Ruslan Kuprieiev bee04c6d5f deps: bump dvc-data to 2.13.1 
Fixes #9733
 2023-08-11 05:53:19 +03:00
Tibor MachandTibor Mach 04e891cef9 exp: Unify dvc exp list and show output (#9808) 
* use _describe in exp list

* fix pre-commit checks

* simplistic fix

* single _describe, too complex

* slightly simpler

* even simpler

* simple, but with regex

* less substitutions

* describe public

* name handling in UI (tests need updating)

* fixed old tests

* lazy imports

---------

Co-authored-by: Tibor Mach <[email protected]>
 2023-08-10 19:23:27 +09:00
Peter Rowlands f5dda2d3cc deps: bump scmrepo minver to 1.2.1 2023-08-10 19:21:06 +09:00
Ruslan Kuprieiev e9545c8b55 fetch: merge config instead of updating 
Update will replace sections instead of merging them together, which
is not desirable here.

Related https://github.com/iterative/studio/pull/6541
3.14.0 		2023-08-10 04:52:08 +03:00
Ruslan Kuprieiev 09cfd911bb config: use local config from local_dvc_dir 
This used to work like so but by accident, when we didn't reload config
at all when collecting other revisions.

Regression from https://github.com/iterative/dvc/pull/9758
 2023-08-10 04:36:42 +03:00
Ruslan Kuprieiev 4f61a63afd config: resolve: handle ~ home dir 
Fixes #9822
 2023-08-10 03:28:32 +03:00
Ruslan Kuprieiev c71b07b996 index: plot_keys: relparts -> parts 
`_plot_sources` is messed up and needs a rewrite hard, but this will
do for now for studio purposes.

Related https://github.com/iterative/studio/pull/6541
3.13.3 		2023-08-08 17:14:28 +03:00
Ruslan Kuprieiev f4c0946f65 repo: cache: support some view options 3.13.2 2023-08-08 01:09:50 +03:00
Ruslan Kuprieiev e35fb76dac repo: add basic cache_check_missing 3.13.1 2023-08-07 22:23:49 +03:00
Ruslan Kuprieiev 722839dda6 fetch: add basic onerror 
Related https://github.com/iterative/studio/pull/6541
3.13.0 		2023-08-07 03:24:39 +03:00