Commit Graph
8861 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 2102ba8465 output: fix formatting (#9716)
Regression from https://github.com/iterative/dvc/pull/9715
2023-07-09 19:50:00 +05:45
skshetry d736d0a3ef output: checkout: use f-string (#9715)
Regression from https://github.com/iterative/dvc/pull/9673.
2023-07-09 11:49:22 +05:45
dberenbaum 0d1f1f6436 update link for external data sources 2023-07-05 19:42:07 +03:00
Ruslan Kuprieiev 63de16ced1 readme: point to dvc.org for packages
We no longer attach bin packages to github releases, so old link is completely misleading.
2023-07-04 18:47:32 +03:00
Ruslan Kuprieiev 2f6c308c84 repo: site_cache_dir: take major version into account
So that different major dvc versions don't use the same cache path.
2023-07-04 18:24:20 +03:00
skshetry c8d180d2b0 repro: move code around (#9701)
emphasizing on building the graph correctly,
and preparing for being able to work with graph directly.
2023-07-04 21:02:53 +05:45
skshetry d69972d199 repro: ignore --downstream if used together with --pipeline (#9692)
* repro: no need to build pipelines for --all-pipeline

* repro: ignore --downstream when using --pipeline
2023-07-04 19:03:09 +05:45
Peter Rowlands eb1ed40228 repo.open_repo: use abspath for local fs URL 2023-07-04 12:28:09 +03:00
Peter Rowlands 120289128e dvcfs: cleanup subrepo contexts on close() 2023-07-04 12:28:09 +03:00
Peter Rowlands a607e7b3d7 dvcfs: return md5-dos2unix in info 2023-07-04 12:28:09 +03:00
Peter Rowlands 86e199c6f2 api.get_url: use index remote storage for getting remote URL 2023-07-04 12:28:09 +03:00
Ruslan Kuprieiev 8661abf224 fetch: introduce --max-size (#9696)
Needed for iterative/studio#4782
3.4.0
2023-07-04 02:17:53 +03:00
Jon Crallandskshetry 541def388b environment variable to show debug options (#9693)
* environment variable to show debug options

* lint

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add message to debug opts

* spelling

* Add argument group and move cprofile-dump

line length

linelen

lint

* use DVC_DEBUG instead of DVC_SHOW_DEBUG_OPTIONS

* use -v instead of environ

* Update dvc/_debug.py

Co-authored-by: skshetry <[email protected]>

* fix

---------

Co-authored-by: skshetry <[email protected]>
2023-07-04 01:35:33 +03:00
Ruslan Kuprieiev 58eb145b6a Delete hook-google_compute_engine.logger.py 2023-07-03 22:13:22 +03:00
Ruslan Kuprieiev 16465e0b63 Delete hook-dvc.system.py 2023-07-03 22:13:22 +03:00
Ruslan Kuprieiev 6614efd35a Delete hook-dvc.tree.gs.py 2023-07-03 22:13:22 +03:00
skshetry 2becfd9344 refactor repro: split logic for planning stages to run for repro (#9690)
The planning logic has now been moved to `plan_repro` api, that allows us
to test without creating stages.
2023-07-03 17:53:48 +05:45
skshetry b7b6642428 repro: do not autostage stage files (#9689)
The logic is now moved to experiments, where it tracks all the stage related files
itself. The `_get_stage_files()` has been modified to also return lockfile path,
so that it can be staged, and has been moved to `stage/utils.py`, and out of `reproduce.py`.

Fixes #9688.
2023-07-03 14:42:22 +05:45
skshetry fea2e6bb30 repro: simplify _reproduce_stages (#9687) 2023-07-03 12:25:25 +05:45
skshetry bfa5b1a0c3 repro: make stages run in order of the graph (#9675)
* repro: make stages run in order of the graph.

In case of --single-item, this means it'll run in the order of arguments passed.
For --downstream, this fixes running of stage out of order when multiple arguments are passed.
For normal case, this enforces the same logic, so that they are never run out-of-order.

* add tests

* tests: repro: add test for --single-item with multiple targets

* graph: support when no nodes are provided
2023-07-03 11:44:51 +05:45
Peter Rowlands 24d9a05d69 fetch: warn if cloud-versioned file has no version ID 2023-07-03 14:05:15 +09:00
skshetry 1f31a9c696 bandit: add exclusion for dvc/repo/experiments/utils.py back (#9686)
See https://github.com/PyCQA/bandit/pull/940.

Will not be needed when new version of bandit comes.
2023-07-03 09:56:45 +05:45
skshetry 097e48fcc9 remove scripts/ related suppresses (#9685) 2023-07-03 09:46:27 +05:45
skshetry 453a8f9516 ci: use kebab-case (#9684) 2023-07-03 09:16:53 +05:45
skshetry 9f59d8652e remove build scripts and scripts directory (#9683) 2023-07-03 09:05:06 +05:45
Ruslan Kuprieiev b1834f34e7 deps: limit flufl-lock to <8 (#9681)
Looks like flufl-lock switched to pdm in 8.0 which triggered a

```
pkg_resources.DistributionNotFound: The 'flufl.lock>=5' distribution was not found and is required by dvc
```

error in multiple scenarious. Can't put my finger on anything specific
in flufl-lock yet, so will just limit its version for now.

Fixes https://github.com/iterative/dvc/issues/9654
3.3.1
2023-07-03 03:13:50 +03:00
Ruslan Kuprieiev 536273ab1e build: pyinstaller: use __pyinstaller (#9680) 3.3.0 2023-07-03 01:47:31 +03:00
Ruslan Kuprieiev 5cff4891fd deps: bump dvc-data to >=2.3.2 (#9679)
Fixes #9462
Fixes #9512
2023-07-02 23:09:27 +03:00
Jon Crall e29da6eae1 Add doctest for dagascii.draw (#9678) 2023-07-01 20:36:14 +03:00
Ruslan Kuprieiev 046ebb05dd stage: cache: catch CollectCacheError (#9677) 3.2.4 2023-07-01 02:19:52 +03:00
skshetry 36ec25ffd7 repro: remove on_unchanged kwarg and _repro_callback (#9674)
repro: remove on_unchaged kwarg and _repro_callback
2023-06-30 10:05:44 +05:45
Ruslan Kuprieiev 6e6d9253d6 output: use path instead of relpath (#9673)
Relpath might not exist (e.g. on windows between different drives),
we should just use path as is and not get too fancy with the ui,
unless we want to constantly keep ValueError in mind when using
relpath for ui needs all over the place.

These operations should use index, where we always have current
directory and relative paths, so the ui will change accordingly
anyway.
3.2.3
2023-06-29 14:52:49 +03:00
Ruslan Kuprieievandpre-commit-ci[bot] 910c232077 output: save: error out on cached external outputs (#9672)
* output: save: error out on cached external outputs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-06-29 13:05:17 +03:00
skshetry c90947f33c make it a dvc repo (#9670) 2023-06-29 08:50:03 +05:45
Peter Rowlands 4ee4f137f2 exp pull: update for index fetch 3.2.2 2023-06-26 14:09:24 +09:00
Oded Messer a750a75016 Stable experiment range sorting (#9660)
* secondary sorting by rev alphanumerically. to introduce determinism

* add UT
2023-06-26 10:53:39 +09:00
Ruslan Kuprieiev cf976f17ae data: status: use ui.progress instead of ui.status (#9662)
With all fetch and checkout related things that we've done in the recent past,
we now have all the mechanisms necessary to provide actual progress (no total,
but still better than spinner) during `dvc data status` operations.
2023-06-26 00:10:44 +03:00
Ruslan Kuprieiev 7041fa5f42 checkout: use ui.progress (#9661)
* fs: callbacks: create our own TqdmCallback

We have a near identical implementation in dvc-objects, but this one is
meant to work with our Tqdm progress instance.

dvc-objects one is useful on other levels, but dvc should have itsown
that inherits from dvc-objects' Callback.

* progress: add as_callback

* checkout: use ui.progress
2023-06-25 23:14:49 +03:00
Ruslan Kuprieiev d898877175 checkout: use single tics instead of curly braces (#9659)
Typo during recent changes 🤦 

Related https://github.com/iterative/dvc/issues/9658
2023-06-24 00:06:05 +03:00
skshetry 1ca1d78b38 deps: bump gdrive to 2.20 (#9657) 3.2.1 2023-06-23 09:13:35 +05:45
David de la Iglesia Castro da6ebdbf8c Update schema.py 3.2.0 2023-06-23 00:10:11 +09:00
Peter Rowlands e995492095 output: check legacy hash name in output.add() 2023-06-22 22:21:34 +09:00
Peter Rowlands 276d499c9b deps: bump scmrepo minver 2023-06-22 22:04:16 +09:00
skshetry fd4696cc50 Revert "drop stage-level vars" (#9647)
Revert "drop stage-level vars (#9556)"

This reverts commit 358c5e0c39.
2023-06-22 18:44:51 +05:45
Peter Rowlands 90b9193250 output: preserve legacy hash_name on save() for unchanged data 2023-06-22 16:20:56 +09:00
Ruslan Kuprieiev 77e35145a9 checkout: delete output if we can't create it fully (#9642) 2023-06-22 02:54:53 +03:00
Dave Berenbaum 49c7c55ba3 exp: push/pull --rev help text shows default val (#9636) 2023-06-21 02:05:50 +03:00
Dave BerenbaumandRuslan Kuprieiev f28d5ee9e0 output: set remove to recursive=false (#9574)
* output: set remove to recursive=false

* try to make test pass on windows

* Update dvc/output.py

---------

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2023-06-21 01:33:04 +03:00
Ruslan Kuprieiev 621fe07a3b checkout: friendlier UI when tryint to remove uncached files
Related #9618
2023-06-20 21:09:00 +03:00
Ruslan Kuprieiev 8323923ce9 deps: bump dvc-data to >= 2.1.1 2023-06-20 18:29:52 +03:00