Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,936 Commits 7 Branches 579 Tags
3783d65ddeafe57716d4c5a18ac8f20f333de530
Commit Graph
267 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 31ffce4fba ls/ls-url: use humanize.naturalsize instead of tqdm 
Currently we get odd formatting like

```
456    .devcontainer.json
139    .dvcignore
61.0   .gitattributes
96.0   .github
7.00   .gitignore
5.76k  README.md
160    data
1.84k  dvc.lock
519    dvc.yaml
4.00   foo
96.0   models
96.0   notebooks
225    params.yaml
46.0   requirements.txt
128    results
160    src
```

(notice stuff like 46.0)

and there doesn't seem to be a simple way to fix that in tqdm, in addition to
the general feeling that we are abusing tqdm's util for this.

So let's just use `humanize`, which gets us a more pleasing result:

```
456B  .devcontainer.json
139B  .dvcignore
61B   .gitattributes
96B   .github
7B    .gitignore
5.8K  README.md
160B  data
1.8K  dvc.lock
519B  dvc.yaml
4B    foo
96B   models
96B   notebooks
225B  params.yaml
46B   requirements.txt
128B  results
160B  src
```
 2023-08-18 02:54:31 +03:00
dependabot[bot] 2da959acd1 build(deps-dev): update pytest-docker requirement from <2,>=1 to >=1,<3 
Updates the requirements on [pytest-docker](https://github.com/avast/pytest-docker) to permit the latest version.
- [Release notes](https://github.com/avast/pytest-docker/releases)
- [Changelog](https://github.com/avast/pytest-docker/blob/master/CHANGELOG.md)
- [Commits](https://github.com/avast/pytest-docker/compare/v1.0.0...v2.0.0)

---
updated-dependencies:
- dependency-name: pytest-docker
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
 2023-08-18 00:41:47 +03:00
dependabot[bot] 152991d878 build(deps-dev): bump mypy from 1.4.1 to 1.5.1 (#9852) 
Bumps [mypy](https://github.com/python/mypy) from 1.4.1 to 1.5.1.
- [Commits](https://github.com/python/mypy/compare/v1.4.1...v1.5.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-17 18:43:31 +05:45
skshetry 8098fd109d plots: fix caching of legacy format (#9849) 
* plots: fix caching of legacy format

* deps: bump dvc-data minversion to 2.15.0
 2023-08-17 07:40:07 +05:45
Ruslan Kuprieiev f0c1ff0219 fs: make _prepare_credentails/fs_args lazy (#9848) 
* tests: unit: move s3 tests to dvc-s3

* deps: bump dvc-data to 2.14.0

* tests: unit: move azure tests to dvc-azure
 2023-08-16 05:51:20 +03:00
Peter Rowlands a051b8b60f remove support for dvc machine 2023-08-11 14:42:52 +09:00
Ruslan Kuprieiev bee04c6d5f deps: bump dvc-data to 2.13.1 
Fixes #9733
 2023-08-11 05:53:19 +03:00
Peter Rowlands f5dda2d3cc deps: bump scmrepo minver to 1.2.1 2023-08-10 19:21:06 +09:00
Ruslan Kuprieiev d5533348f7 deps: bump dvc-data to 2.12.0 2023-08-07 02:42:41 +03:00
Ruslan Kuprieiev 43b371c43a deps: bump dvc-data to 2.11.0 2023-08-04 15:42:25 +03:00
Ruslan Kuprieiev 9d1d51caa7 deps: bump dvc-data to 2.10.0 2023-08-04 03:40:36 +03:00
Peter Rowlands aa94bfd66b deps: bump dvc-data to 2.9.1 2023-08-03 18:19:19 +09:00
Ruslan Kuprieiev 8a382f51fd deps: bump dvc-data to 2.9.0 2023-08-03 02:05:14 +03:00
pre-commit-ci[bot]andSaugat Pachhai 502b3df6c9 [pre-commit.ci] pre-commit autoupdate (#9756) 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.9.0 → v3.10.1](https://github.com/asottile/pyupgrade/compare/v3.9.0...v3.10.1)
- [github.com/astral-sh/ruff-pre-commit: v0.0.278 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.278...v0.0.281)
- [github.com/PyCQA/flake8: 6.0.0 → 6.1.0](https://github.com/PyCQA/flake8/compare/6.0.0...6.1.0)

* fix issues reported by ruff

- also remove redundant flake8 plugins
- remove beautysh as we don't have any scripts now
- remove pyupgrade as it is redundant due to ruff

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-08-02 18:08:30 +05:45
dependabot[bot] 5c2fd67e6d build(deps-dev): bump pylint from 2.17.4 to 2.17.5 (#9769) 
Bumps [pylint](https://github.com/pylint-dev/pylint) from 2.17.4 to 2.17.5.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v2.17.4...v2.17.5)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-31 16:35:40 +05:45
Ruslan Kuprieiev 7c60dfd3c5 deps: bump dvc-data to >=2.8.1 2023-07-30 04:18:09 +03:00
Ruslan Kuprieiev ca0ad239f7 deps: bump dvc-data to 0.28.0 2023-07-28 01:37:46 +03:00
Ruslan Kuprieiev f57ac609e9 deps: bump dvc-data to 0.27.1 2023-07-27 20:44:34 +03:00
Dave BerenbaumandRuslan Kuprieiev 41194b7975 dvcfs: handle NotADirectoryError from datafs.ls (#9746) 
* dvc_fs: ignore NotADirectoryError in ls

* deps: bump dvc-data to >= 2.6.0

* add test

---------

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2023-07-19 20:54:55 +03:00
skshetry ea5f7d40c1 ruff: enable a few more rules, and merge tests' config with the main config (#9739) 2023-07-15 14:33:34 +05:45
dependabot[bot] eb06351062 build(deps-dev): bump mypy from 1.3.0 to 1.4.1 (#9664) 
Bumps [mypy](https://github.com/python/mypy) from 1.3.0 to 1.4.1.
- [Commits](https://github.com/python/mypy/compare/v1.3.0...v1.4.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-14 20:30:47 +05:45
pre-commit-ci[bot]andSaugat Pachhai ca3704997b [pre-commit.ci] pre-commit autoupdate (#9632) 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.3.1 → v3.9.0](https://github.com/asottile/pyupgrade/compare/v3.3.1...v3.9.0)
- https://github.com/charliermarsh/ruff-pre-commit → https://github.com/astral-sh/ruff-pre-commit
- [github.com/astral-sh/ruff-pre-commit: v0.0.261 → v0.0.277](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.261...v0.0.277)
- [github.com/codespell-project/codespell: v2.2.4 → v2.2.5](https://github.com/codespell-project/codespell/compare/v2.2.4...v2.2.5)

* fix ruff complaints

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-07-14 20:11:15 +05:45
Ruslan Kuprieiev 2ef2caafd3 deps: bump dvc-data to 2.5.0 2023-07-09 20:30:35 +03:00
Ruslan Kuprieiev adcefbec66 output: pass explicit callback to hashfile.build() 2023-07-09 17:40:49 +03:00
skshetry 1f31a9c696 bandit: add exclusion for dvc/repo/experiments/utils.py back (#9686) 
See https://github.com/PyCQA/bandit/pull/940.

Will not be needed when new version of bandit comes.
 2023-07-03 09:56:45 +05:45
skshetry 097e48fcc9 remove scripts/ related suppresses (#9685) 2023-07-03 09:46:27 +05:45
Ruslan Kuprieiev b1834f34e7 deps: limit flufl-lock to <8 (#9681) 
Looks like flufl-lock switched to pdm in 8.0 which triggered a

```
pkg_resources.DistributionNotFound: The 'flufl.lock>=5' distribution was not found and is required by dvc
```

error in multiple scenarious. Can't put my finger on anything specific
in flufl-lock yet, so will just limit its version for now.

Fixes https://github.com/iterative/dvc/issues/9654
 2023-07-03 03:13:50 +03:00
Ruslan Kuprieiev 536273ab1e build: pyinstaller: use __pyinstaller (#9680) 2023-07-03 01:47:31 +03:00
Ruslan Kuprieiev 5cff4891fd deps: bump dvc-data to >=2.3.2 (#9679) 
Fixes #9462
Fixes #9512
 2023-07-02 23:09:27 +03:00
skshetry 1ca1d78b38 deps: bump gdrive to 2.20 (#9657) 2023-06-23 09:13:35 +05:45
Peter Rowlands 276d499c9b deps: bump scmrepo minver 2023-06-22 22:04:16 +09:00
Ruslan Kuprieiev 77e35145a9 checkout: delete output if we can't create it fully (#9642) 2023-06-22 02:54:53 +03:00
Ruslan Kuprieiev 621fe07a3b checkout: friendlier UI when tryint to remove uncached files 
Related #9618
 2023-06-20 21:09:00 +03:00
Ruslan Kuprieiev 8323923ce9 deps: bump dvc-data to >= 2.1.1 2023-06-20 18:29:52 +03:00
Ruslan Kuprieiev e16c0c8b2f checkout/fetch: finer control over steps 
See https://github.com/iterative/dvc-data/pull/390 and https://github.com/iterative/dvc-data/pull/391
 2023-06-18 18:58:30 +03:00
Ruslan Kuprieiev 97348cfba7 Revert "checkout/fetch: finer control over steps" 
This reverts commit c5e3d42899.
 2023-06-14 03:39:43 +03:00
Ruslan Kuprieiev c5e3d42899 checkout/fetch: finer control over steps 
See https://github.com/iterative/dvc-data/pull/390 and https://github.com/iterative/dvc-data/pull/391
 2023-06-13 23:11:14 +03:00
skshetry be975ad9eb pyproject.toml: reformat project table (#9596) 2023-06-13 20:02:34 +05:45
Peter Rowlands 5fd54679c7 dvc-data: bump to 1.11.0 2023-06-13 21:33:46 +09:00
Ruslan Kuprieiev 5c1487e2ec deps: bump dvc-data to >=1.10.0 2023-06-13 14:45:36 +03:00
skshetry 30517f6676 deps: update dvc-task/dvc-s3/pytest-cov (#9592) 
* remove deprecation warnings

* Update pyproject.toml
 2023-06-13 15:39:34 +05:45
Ruslan Kuprieiev bbb77360e9 deps: bump dvc-data to >=1.9.0 
Fixes #9541
 2023-06-12 22:03:29 +03:00
Ruslan Kuprieiev 1af150a5e9 index: use out.hash_info for complete imports 2023-06-12 16:55:36 +03:00
Peter Rowlands b733dea4c2 dvc-data: bump to 1.7.0 2023-06-12 16:22:25 +09:00
Ruslan Kuprieiev 2ed677057d fetch: cache collected 
This avoids recollecting the cache files over and over again.

For example, `dvc fetch` noop in `dvc-bench` went from ~30sec down to ~10sec
for me.

Next step is to also cache remote index, so we don't have to waste those
~10sec checking files in the remote.

Related https://github.com/iterative/dvc-data/pull/376
 2023-06-12 04:31:37 +03:00
Ruslan Kuprieiev 0597311439 deps: bump dvc-data to >=1.5.0 2023-06-12 01:04:24 +03:00
Ruslan Kuprieiev fc8ec6aa95 fetch: show progress when collecting 2023-06-10 22:48:16 +03:00
Ruslan Kuprieiev 3dca69b678 deps: bump dvc-data to >= 1.2.0 
This brings https://github.com/iterative/dvc-data/pull/373 , which improves UI for `checkout` by not visually hanging during `diff` stage inside.
 2023-06-10 21:33:25 +03:00
Ruslan Kuprieiev 4c0bb8d316 fetch: use index fetch 2023-06-10 21:18:57 +03:00
Ruslan Kuprieiev 8b1c1a31f3 deps: bump dvc-data to >=1.0.3 2023-06-09 18:16:51 +03:00