Commit Graph
8875 Commits
Author SHA1 Message Date
Ruslan Kuprieiev cf976f17ae data: status: use ui.progress instead of ui.status (#9662)
With all fetch and checkout related things that we've done in the recent past,
we now have all the mechanisms necessary to provide actual progress (no total,
but still better than spinner) during `dvc data status` operations.
2023-06-26 00:10:44 +03:00
Ruslan Kuprieiev 7041fa5f42 checkout: use ui.progress (#9661)
* fs: callbacks: create our own TqdmCallback

We have a near identical implementation in dvc-objects, but this one is
meant to work with our Tqdm progress instance.

dvc-objects one is useful on other levels, but dvc should have itsown
that inherits from dvc-objects' Callback.

* progress: add as_callback

* checkout: use ui.progress
2023-06-25 23:14:49 +03:00
Ruslan Kuprieiev d898877175 checkout: use single tics instead of curly braces (#9659)
Typo during recent changes 🤦 

Related https://github.com/iterative/dvc/issues/9658
2023-06-24 00:06:05 +03:00
skshetry 1ca1d78b38 deps: bump gdrive to 2.20 (#9657) 3.2.1 2023-06-23 09:13:35 +05:45
David de la Iglesia Castro da6ebdbf8c Update schema.py 3.2.0 2023-06-23 00:10:11 +09:00
Peter Rowlands e995492095 output: check legacy hash name in output.add() 2023-06-22 22:21:34 +09:00
Peter Rowlands 276d499c9b deps: bump scmrepo minver 2023-06-22 22:04:16 +09:00
skshetry fd4696cc50 Revert "drop stage-level vars" (#9647)
Revert "drop stage-level vars (#9556)"

This reverts commit 358c5e0c39.
2023-06-22 18:44:51 +05:45
Peter Rowlands 90b9193250 output: preserve legacy hash_name on save() for unchanged data 2023-06-22 16:20:56 +09:00
Ruslan Kuprieiev 77e35145a9 checkout: delete output if we can't create it fully (#9642) 2023-06-22 02:54:53 +03:00
Dave Berenbaum 49c7c55ba3 exp: push/pull --rev help text shows default val (#9636) 2023-06-21 02:05:50 +03:00
Dave BerenbaumandRuslan Kuprieiev f28d5ee9e0 output: set remove to recursive=false (#9574)
* output: set remove to recursive=false

* try to make test pass on windows

* Update dvc/output.py

---------

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2023-06-21 01:33:04 +03:00
Ruslan Kuprieiev 621fe07a3b checkout: friendlier UI when tryint to remove uncached files
Related #9618
2023-06-20 21:09:00 +03:00
Ruslan Kuprieiev 8323923ce9 deps: bump dvc-data to >= 2.1.1 2023-06-20 18:29:52 +03:00
dberenbaum 04b50c5cd0 exp push/pull: default to head rev 2023-06-20 15:53:46 +09:00
dberenbaum b20a59d748 exp push/pull: default to all exps on head 2023-06-20 15:53:46 +09:00
dberenbaum bf2f08da09 exp list: fix ui for tags/branches 2023-06-20 15:52:18 +09:00
Ruslan Kuprieiev 1835c8c6e9 tests: unit: explicitly ask for dvc repo
Currently these rely on the root of the repo being in the dvc repo.
2023-06-19 03:44:10 +03:00
Ruslan Kuprieiev 7e1bb07f10 gha: don't build binary packages
The build process is much more stable these days and there is no reason
to keep building these packages here. Instead every package has itsown
building process (dvc-osxpkg, dvc-exe, dvc-s3-repo) and knows best how
to build and sign correctly.

pyinstaller hooks need to go into a `__pyinstaller` later on.
2023-06-19 03:04:04 +03:00
Ruslan Kuprieiev 92dc1f31f5 outputs: is_in_repo: don't try to realpath fs_path
Fixes #9621
3.1.0
2023-06-19 01:19:46 +03:00
Ruslan Kuprieiev e16c0c8b2f checkout/fetch: finer control over steps
See https://github.com/iterative/dvc-data/pull/390 and https://github.com/iterative/dvc-data/pull/391
2023-06-18 18:58:30 +03:00
Ruslan Kuprieiev 33e7645603 tests: odb: index: simplify test
For some unknown reason the mock in this test leaks into the outside tests.

I wasn't able to determine why in the reasonable amount of time, so instead just
simplifying it to not use mocker at all.

That whole file should really be migrated to dvc-data, but that needs some time.
2023-06-18 18:58:30 +03:00
Ruslan Kuprieiev c827654fe7 tests: use mocker instead of unittest.mock where possible
Had the "pleasure" of finding a mocker.patch leak into other tests and one
of the suspicions is that mixing unittest.mock and mocker might be somehow
related.

In any case, we should use mocker where possible anyway.
2023-06-18 17:10:45 +03:00
dberenbaum b8c666b1b7 exp show: quick fix for handling defaultdict in py<3.11.0 2023-06-16 22:54:40 +03:00
Ruslan Kuprieiev fd73b389c6 api: show/exp: add config option
Fixes #9613
2023-06-16 17:09:38 +03:00
Ruslan Kuprieiev 0433565f1f api: add config support for open/read
Fixes #9610
2023-06-15 18:30:14 +03:00
Peter Rowlands b5449503f0 benchmarks: update min dvc version for test_exp_show 2023-06-15 12:58:21 +09:00
Peter Rowlands fa14507bc8 benchmarks: use bench_dvc for commands that generate dvcfiles 2023-06-15 10:44:58 +09:00
Peter Rowlands 21458796ed benchmarks: update test_exp_show to use example-get-started 2023-06-15 10:44:58 +09:00
Ruslan Kuprieiev a6e49a72de diff: handle revisions pointing to the same sha
Fixes #9604
2023-06-14 18:48:18 +03:00
Ruslan Kuprieiev 97348cfba7 Revert "checkout/fetch: finer control over steps"
This reverts commit c5e3d42899.
2023-06-14 03:39:43 +03:00
Ruslan Kuprieiev c5e3d42899 checkout/fetch: finer control over steps
See https://github.com/iterative/dvc-data/pull/390 and https://github.com/iterative/dvc-data/pull/391
2023-06-13 23:11:14 +03:00
skshetry c6c2ce130a remove dvc._build (#9597) 3.0.0 2023-06-13 20:31:22 +05:45
skshetry be975ad9eb pyproject.toml: reformat project table (#9596) 2023-06-13 20:02:34 +05:45
daavoo d0aa1ce776 repro: Check for hash mismatch bewteen deleted dependencies and upstream outputs.
Only if `--allow-missing` is passed.

- Create `tests/func/repro` and extract `pull` and `allow_missing` to separate test files.

Closes #9530
2023-06-13 15:46:34 +03:00
Peter Rowlands 5ec4fa473c cache migrate: add utility for migrating local cache files to 3.x 2023-06-13 21:33:46 +09:00
Peter Rowlands 5fd54679c7 dvc-data: bump to 1.11.0 2023-06-13 21:33:46 +09:00
Ruslan Kuprieiev 5c1487e2ec deps: bump dvc-data to >=1.10.0 2023-06-13 14:45:36 +03:00
skshetry 30517f6676 deps: update dvc-task/dvc-s3/pytest-cov (#9592)
* remove deprecation warnings

* Update pyproject.toml
2023-06-13 15:39:34 +05:45
Ruslan Kuprieiev bbb77360e9 deps: bump dvc-data to >=1.9.0
Fixes #9541
2023-06-12 22:03:29 +03:00
Ruslan Kuprieiev 1af150a5e9 index: use out.hash_info for complete imports 2023-06-12 16:55:36 +03:00
Peter Rowlands cf59d801ce repo.index: use correct hash when rebuilding index entries 2023-06-12 16:22:25 +09:00
Peter Rowlands b733dea4c2 dvc-data: bump to 1.7.0 2023-06-12 16:22:25 +09:00
Ruslan Kuprieiev 2ed677057d fetch: cache collected
This avoids recollecting the cache files over and over again.

For example, `dvc fetch` noop in `dvc-bench` went from ~30sec down to ~10sec
for me.

Next step is to also cache remote index, so we don't have to waste those
~10sec checking files in the remote.

Related https://github.com/iterative/dvc-data/pull/376
2023-06-12 04:31:37 +03:00
Ruslan Kuprieiev 0597311439 deps: bump dvc-data to >=1.5.0 2023-06-12 01:04:24 +03:00
Ruslan Kuprieiev fc8ec6aa95 fetch: show progress when collecting 2023-06-10 22:48:16 +03:00
Ruslan Kuprieiev 3dca69b678 deps: bump dvc-data to >= 1.2.0
This brings https://github.com/iterative/dvc-data/pull/373 , which improves UI for `checkout` by not visually hanging during `diff` stage inside.
2023-06-10 21:33:25 +03:00
Ruslan Kuprieiev 4c0bb8d316 fetch: use index fetch 2023-06-10 21:18:57 +03:00
Ruslan Kuprieiev 1cb371d4d7 repo: commit: don't require target
Same as CLI.
2023-06-10 19:19:40 +03:00
Ruslan Kuprieiev c96795dcfb tests: checkout: fix test_checkout_relink_protected on windows 2023-06-09 22:36:33 +03:00