262 Commits
Author SHA1 Message Date
Peter Rowlands a051b8b60f remove support for dvc machine 2023-08-11 14:42:52 +09:00
Ruslan Kuprieiev bee04c6d5f deps: bump dvc-data to 2.13.1
Fixes #9733
2023-08-11 05:53:19 +03:00
Peter Rowlands f5dda2d3cc deps: bump scmrepo minver to 1.2.1 2023-08-10 19:21:06 +09:00
Ruslan Kuprieiev d5533348f7 deps: bump dvc-data to 2.12.0 2023-08-07 02:42:41 +03:00
Ruslan Kuprieiev 43b371c43a deps: bump dvc-data to 2.11.0 2023-08-04 15:42:25 +03:00
Ruslan Kuprieiev 9d1d51caa7 deps: bump dvc-data to 2.10.0 2023-08-04 03:40:36 +03:00
Peter Rowlands aa94bfd66b deps: bump dvc-data to 2.9.1 2023-08-03 18:19:19 +09:00
Ruslan Kuprieiev 8a382f51fd deps: bump dvc-data to 2.9.0 2023-08-03 02:05:14 +03:00
pre-commit-ci[bot]andSaugat Pachhai 502b3df6c9 [pre-commit.ci] pre-commit autoupdate (#9756)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.9.0 → v3.10.1](https://github.com/asottile/pyupgrade/compare/v3.9.0...v3.10.1)
- [github.com/astral-sh/ruff-pre-commit: v0.0.278 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.278...v0.0.281)
- [github.com/PyCQA/flake8: 6.0.0 → 6.1.0](https://github.com/PyCQA/flake8/compare/6.0.0...6.1.0)

* fix issues reported by ruff

- also remove redundant flake8 plugins
- remove beautysh as we don't have any scripts now
- remove pyupgrade as it is redundant due to ruff

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-08-02 18:08:30 +05:45
dependabot[bot] 5c2fd67e6d build(deps-dev): bump pylint from 2.17.4 to 2.17.5 (#9769)
Bumps [pylint](https://github.com/pylint-dev/pylint) from 2.17.4 to 2.17.5.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v2.17.4...v2.17.5)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-31 16:35:40 +05:45
Ruslan Kuprieiev 7c60dfd3c5 deps: bump dvc-data to >=2.8.1 2023-07-30 04:18:09 +03:00
Ruslan Kuprieiev ca0ad239f7 deps: bump dvc-data to 0.28.0 2023-07-28 01:37:46 +03:00
Ruslan Kuprieiev f57ac609e9 deps: bump dvc-data to 0.27.1 2023-07-27 20:44:34 +03:00
Dave BerenbaumandRuslan Kuprieiev 41194b7975 dvcfs: handle NotADirectoryError from datafs.ls (#9746)
* dvc_fs: ignore NotADirectoryError in ls

* deps: bump dvc-data to >= 2.6.0

* add test

---------

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2023-07-19 20:54:55 +03:00
skshetry ea5f7d40c1 ruff: enable a few more rules, and merge tests' config with the main config (#9739) 2023-07-15 14:33:34 +05:45
dependabot[bot] eb06351062 build(deps-dev): bump mypy from 1.3.0 to 1.4.1 (#9664)
Bumps [mypy](https://github.com/python/mypy) from 1.3.0 to 1.4.1.
- [Commits](https://github.com/python/mypy/compare/v1.3.0...v1.4.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-14 20:30:47 +05:45
pre-commit-ci[bot]andSaugat Pachhai ca3704997b [pre-commit.ci] pre-commit autoupdate (#9632)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.3.1 → v3.9.0](https://github.com/asottile/pyupgrade/compare/v3.3.1...v3.9.0)
- https://github.com/charliermarsh/ruff-pre-commit → https://github.com/astral-sh/ruff-pre-commit
- [github.com/astral-sh/ruff-pre-commit: v0.0.261 → v0.0.277](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.261...v0.0.277)
- [github.com/codespell-project/codespell: v2.2.4 → v2.2.5](https://github.com/codespell-project/codespell/compare/v2.2.4...v2.2.5)

* fix ruff complaints

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-07-14 20:11:15 +05:45
Ruslan Kuprieiev 2ef2caafd3 deps: bump dvc-data to 2.5.0 2023-07-09 20:30:35 +03:00
Ruslan Kuprieiev adcefbec66 output: pass explicit callback to hashfile.build() 2023-07-09 17:40:49 +03:00
skshetry 1f31a9c696 bandit: add exclusion for dvc/repo/experiments/utils.py back (#9686)
See https://github.com/PyCQA/bandit/pull/940.

Will not be needed when new version of bandit comes.
2023-07-03 09:56:45 +05:45
skshetry 097e48fcc9 remove scripts/ related suppresses (#9685) 2023-07-03 09:46:27 +05:45
Ruslan Kuprieiev b1834f34e7 deps: limit flufl-lock to <8 (#9681)
Looks like flufl-lock switched to pdm in 8.0 which triggered a

```
pkg_resources.DistributionNotFound: The 'flufl.lock>=5' distribution was not found and is required by dvc
```

error in multiple scenarious. Can't put my finger on anything specific
in flufl-lock yet, so will just limit its version for now.

Fixes https://github.com/iterative/dvc/issues/9654
2023-07-03 03:13:50 +03:00
Ruslan Kuprieiev 536273ab1e build: pyinstaller: use __pyinstaller (#9680) 2023-07-03 01:47:31 +03:00
Ruslan Kuprieiev 5cff4891fd deps: bump dvc-data to >=2.3.2 (#9679)
Fixes #9462
Fixes #9512
2023-07-02 23:09:27 +03:00
skshetry 1ca1d78b38 deps: bump gdrive to 2.20 (#9657) 2023-06-23 09:13:35 +05:45
Peter Rowlands 276d499c9b deps: bump scmrepo minver 2023-06-22 22:04:16 +09:00
Ruslan Kuprieiev 77e35145a9 checkout: delete output if we can't create it fully (#9642) 2023-06-22 02:54:53 +03:00
Ruslan Kuprieiev 621fe07a3b checkout: friendlier UI when tryint to remove uncached files
Related #9618
2023-06-20 21:09:00 +03:00
Ruslan Kuprieiev 8323923ce9 deps: bump dvc-data to >= 2.1.1 2023-06-20 18:29:52 +03:00
Ruslan Kuprieiev e16c0c8b2f checkout/fetch: finer control over steps
See https://github.com/iterative/dvc-data/pull/390 and https://github.com/iterative/dvc-data/pull/391
2023-06-18 18:58:30 +03:00
Ruslan Kuprieiev 97348cfba7 Revert "checkout/fetch: finer control over steps"
This reverts commit c5e3d42899.
2023-06-14 03:39:43 +03:00
Ruslan Kuprieiev c5e3d42899 checkout/fetch: finer control over steps
See https://github.com/iterative/dvc-data/pull/390 and https://github.com/iterative/dvc-data/pull/391
2023-06-13 23:11:14 +03:00
skshetry be975ad9eb pyproject.toml: reformat project table (#9596) 2023-06-13 20:02:34 +05:45
Peter Rowlands 5fd54679c7 dvc-data: bump to 1.11.0 2023-06-13 21:33:46 +09:00
Ruslan Kuprieiev 5c1487e2ec deps: bump dvc-data to >=1.10.0 2023-06-13 14:45:36 +03:00
skshetry 30517f6676 deps: update dvc-task/dvc-s3/pytest-cov (#9592)
* remove deprecation warnings

* Update pyproject.toml
2023-06-13 15:39:34 +05:45
Ruslan Kuprieiev bbb77360e9 deps: bump dvc-data to >=1.9.0
Fixes #9541
2023-06-12 22:03:29 +03:00
Ruslan Kuprieiev 1af150a5e9 index: use out.hash_info for complete imports 2023-06-12 16:55:36 +03:00
Peter Rowlands b733dea4c2 dvc-data: bump to 1.7.0 2023-06-12 16:22:25 +09:00
Ruslan Kuprieiev 2ed677057d fetch: cache collected
This avoids recollecting the cache files over and over again.

For example, `dvc fetch` noop in `dvc-bench` went from ~30sec down to ~10sec
for me.

Next step is to also cache remote index, so we don't have to waste those
~10sec checking files in the remote.

Related https://github.com/iterative/dvc-data/pull/376
2023-06-12 04:31:37 +03:00
Ruslan Kuprieiev 0597311439 deps: bump dvc-data to >=1.5.0 2023-06-12 01:04:24 +03:00
Ruslan Kuprieiev fc8ec6aa95 fetch: show progress when collecting 2023-06-10 22:48:16 +03:00
Ruslan Kuprieiev 3dca69b678 deps: bump dvc-data to >= 1.2.0
This brings https://github.com/iterative/dvc-data/pull/373 , which improves UI for `checkout` by not visually hanging during `diff` stage inside.
2023-06-10 21:33:25 +03:00
Ruslan Kuprieiev 4c0bb8d316 fetch: use index fetch 2023-06-10 21:18:57 +03:00
Ruslan Kuprieiev 8b1c1a31f3 deps: bump dvc-data to >=1.0.3 2023-06-09 18:16:51 +03:00
Ruslan Kuprieiev 262419ef35 deps: bump dvc-data to >=1.0.2
Related https://github.com/iterative/dvc/issues/9566
2023-06-09 16:26:31 +03:00
Peter Rowlands 265fc581c0 dvc-data: bump to 1.0.0 2023-06-09 15:49:44 +03:00
Ruslan Kuprieiev d0291e8039 checkout: let index checkout error-out on missing hashes
This ensures we don't assume that everything needs to have hash_info to be
checked out (e.g. we can totally checkout directly from FileStorage).

Related https://github.com/iterative/dvc/pull/9424
2023-06-07 14:56:19 +03:00
Ruslan Kuprieiev 78637713cf checkout: use index checkout
Needed for https://github.com/iterative/dvc/pull/9424
2023-06-04 23:27:01 +03:00
Ruslan Kuprieiev f9104275f3 deps: install tzdata for testing with celery (#9537)
Temporary fix for CI on windows with py >=3.9.
2023-06-04 22:18:23 +03:00