3378 Commits
Author SHA1 Message Date
skshetry e04f1499a3 fix(dvcfs): rename argument url to repo (#10754)
See #10700 for the motivation.
2025-06-05 08:30:22 +05:45
skshetry 0eebeb9900 tests: fix flaky test in test_get_used_objs (#10739)
Clear logs before assertions so that logs from other tests do not interfere with the test.
Some destructors may also run in between due to GC, so hopefully this will fix that issue too,
or reduce the chances of it happening.

See
https://github.com/iterative/dvc/actions/runs/14873217814/job/41775314725#step:6:97.

```console
>       assert first(caplog.messages) == expected_message
E       assert 'RC: discarding all clients' == "Output 'path... 'stage.dvc'."
E
E         + RC: discarding all clients
E         - Output 'path'(stage: 'stage.dvc') is missing version info. Cache for it will not be collected. Use `dvc repro` to get your pipeline up to date.
E         - You can also use `dvc commit stage.dvc` to associate existing 'path' with stage: 'stage.dvc'.
```
2025-05-07 12:40:51 +05:45
skshetry 415d4bc980 repo: try to support opening uninitialized/broken repositories (#10736)
This `_get_remote_config()` should use `uninitialized=True` so that it supports
more broader kinds of broken/partially-initialized repositories, including some that have
`.dvc` directory missing, or `.git` directory missing.

This partially reverts #10719. #10608 is also fixed, and no longer requires `core.no_scm` to be passed.
This was already supported by `dvc.api.get_url` as it uses `uninitialized=True`, but this was not respected
in `_get_remote_config()` where it would fail before.

That said, this whole `open_repo`/`_get_remote_config` is terribly broken. For one, it is opening
a local repository, and forcing it's remote config to a repository opened with `Repo(rev=...)`,
where the config may be different.
2025-05-06 08:39:23 +00:00
Rodrigo Goyaandskshetry 4bde0e754e Forward args to _get_remote_config() and honour core/no_scm if present (#10719)
* Forward args to _get_remote_config and honour core/no_scm if present

* Pass full config, except cache, to Repo()

* Avoid modifying original kwargs contents

* Simplify if config

* Add test

* Relax error matching.. .. Avoid windows vs unix paths

* Update dvc/repo/open_repo.py

---------

Co-authored-by: skshetry <[email protected]>
2025-04-30 12:20:09 +05:45
skshetry 88f1ada4bf tests: func test for #4344 (#10732)
See https://github.com/iterative/dvc-data/pull/601 that fixed the issue.
The test is based on reproduction steps from:
https://github.com/iterative/dvc/issues/4344#issuecomment-2822000925
2025-04-29 13:25:28 +05:45
pre-commit-ci[bot]andSaugat Pachhai 6cc349edb4 [pre-commit.ci] pre-commit autoupdate (#10717)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.9.9 → v0.11.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.9.9...v0.11.4)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix ruff errors

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2025-04-08 03:15:09 +00:00
Rory CreanandRMCrean e455ee69aa remote: show default when run "dvc remote list" (#10711)
* remote: show default when run "dvc remote list"

Fixes #10708

* Indent other remotes if there is a default remote

Now regardless of whether there is a default remote or not, all remotes are equally indented

* stricter test for listing remotes shows default

* Rewrite dvc remote list implemetation

- Default remote is indicated at end of the output and colored green
- Remotes no longer reordered to show default first
- Test updated to reflect changes and remove need for function escape_ansi.

---------

Co-authored-by: RMCrean <[email protected]>
2025-04-07 09:02:09 +05:45
skshetry 7d14acb5a7 3.13 support for Windows (#10667)
python/cpython#113829 changed `os.path.isabs` on Windows to not consider a path starting a single backslash to be an absolute path.

We do a lot of naive `posixpath` to `ntpath` conversions, and it broke DVC in multiple places. DVC is likely broken in many other places, and these bugs are difficult to identify.

In this commit, I have tried to fix in the places where the tests failed. And mostly by trying to imitate pre-3.13 behaviour.
Some of the changes may not be correct but keeps the old behaviour in place.

Also re-enabled the CI for all Python versions supported for Windows.
2025-01-10 18:39:38 +05:45
skshetry dd501554fe tests: try to make tests pass on Python 3.13 on unix platforms (#10665) 2025-01-09 22:36:35 +05:45
skshetry 54acfda687 fix implicit concatenated strings (#10666) 2025-01-09 22:05:33 +05:45
pre-commit-ci[bot] e70a1e9bfc [pre-commit.ci] pre-commit autoupdate (#10663)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-01-09 21:49:48 +05:45
skshetry 2c6b917fc5 ls-url: add support for --tree/--level (#10664) 2025-01-09 21:12:34 +05:45
Raphaël Michelandpre-commit-ci[bot] 198c8f2ffd Fix #10638 : makes remove return correct list when used with both --queue and -A (#10641)
* fixed a bug where using both `--queue` and `-A` would yield a "removed" list without the queued experiments

* added a test to validate the fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* clearer name for test

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-12-01 08:21:46 -08:00
Raphaël Michelandpre-commit-ci[bot] 368c785410 Fix #10593 -- add --keep option for dvc experiments remove (#10633)
* Add keep_selected parameter, and corresponding code to keep only the selected exps (and remove all the other ones)

* test keep_selected_by_name

* test keep_selected_by_rev

* test keep_selected multiple, by name

* test keep all by name

* test keep by rev, with num=2

* added option to cli

* refactoring to meet pr needs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed test_experiments to add keep_selected=False to remove tests

* rename parameter to match cli option

* follow the normal path, then invert the selection before removing

* fixed tests for list ordering + fixed test with non existent name, it didn't make sense to delete everything if an exp name did not exist

* changed cli option comment

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed typing issue

* updated parameter name

* removed handling queued experiments (since --queue would remove them all)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* code simplification, added __eq__ and __hash__ to be able to compare ExpRefs, updated and parametrized tests.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fixed linting issues

* - --keep and --queue together raise an InvalidArgumentError
    - added a test to check if the error is raised
    - fixed CLI message

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* re-run gh tests. Some tests which did not involve my changes started failing while they were passing fine before.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-11-30 10:43:13 -08:00
pre-commit-ci[bot] 7e75b0616f [pre-commit.ci] pre-commit autoupdate (#10635)
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2024-11-26 07:34:46 +05:45
Ivan Shcheklein 2431ec6189 Rename dvcx to DataChain in DVC <> DataChain integration (#10611) 2024-11-05 12:27:45 +05:45
dependabot[bot] 70fc4e6cda build(deps): update platformdirs requirement from <4,>=3.1.1 to >=3.1.1,<5 (#10603)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-05 12:27:30 +05:45
skshetry 746f028f6c ls: implement --tree to show as a tree and --level to limit depth for recursion (#10598) 2024-10-23 07:56:24 +05:45
Pete Bachant d236667048 Add option to show MD5 checksums in dvc ls (#10573) 2024-10-21 16:32:03 +05:45
skshetry ee1864af8e fix tests with the new pydot release (#10567) 2024-09-26 08:13:06 +05:45
skshetry 4f3fb15f2e dvcfs: optimize get() by reducing index.info calls() (#10540) 2024-08-27 18:08:40 +05:45
skshetry 705b7bbea6 simplify running benchmarks (#10536) 2024-08-21 21:09:59 +05:45
Dave BerenbaumandSaugat Pachhai 81db9b4572 save import hash info to state (#10531)
* save import hash info to state

* clean up path handling

* avoid walking

* tests: do not use as_posix to check for output

---------

Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-08-20 13:41:37 +05:45
skshetry 38ee0a01f7 update ruff to 0.6 (#10526) 2024-08-15 20:02:09 +05:45
skshetry fd3b43df05 fix test_daemon.py::test_analytics flakiness (#10523)
Turns out `iterative-telemetry` spends a long time trying to
generate an id. Sometimes it gets frozen for a long time (>30s).
2024-08-14 16:55:18 +05:45
skshetry 52405a0e10 ci: use uv to install dependencies (#10507)
* ci: use uv to install dependencies

* split Windows tests into four for each python versions

* split tests and check into two jobs

* increase timeout
2024-08-14 13:24:56 +05:45
skshetry 5e4daf6cfb deps: bump dvc-data to >=3.16.0 (#10513) 2024-08-13 07:15:05 +05:45
dberenbaum 248ba3b6b6 limit reimport behavior to checkout 2024-07-24 09:37:32 -04:00
George KussumotoandSaugat Pachhai 0e38ccc031 fix(pull): normalize dependency path (#10446)
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-06-03 16:43:43 +05:45
Ivan Shcheklein 8f8d49d023 fix(push/pull): don't fail on unsupported run cache (#10441) 2024-05-27 14:37:44 +05:45
Dave Berenbaum 83ec952da1 improve repro --pull --run-cache test 2024-05-23 14:26:42 -04:00
Dave Berenbaum c0d7e2a8de test repro --pull failure 2024-05-23 14:26:42 -04:00
Dave Berenbaum 1f7733fec1 refactor 2024-05-23 14:26:42 -04:00
Dave Berenbaum 856c902b65 add tests 2024-05-23 14:26:42 -04:00
Dave Berenbaum ef507efa04 artifacts: get studio info from local config 2024-05-23 13:45:48 -04:00
Sigurd Spieckermann 5bfacec974 import/get: escape non-pattern LFS path parts (#10416) 2024-05-13 10:24:06 -07:00
skshetry 8517cccff4 artifacts: add support for monorepo (#10386) 2024-04-16 21:13:02 +05:45
dberenbaum c52a25e011 fix tests 2024-04-16 11:15:52 -04:00
Dave Berenbaum e6048287cb fix test for dvc-render updates to plots title anchor (#10387) 2024-04-11 19:23:03 +05:45
skshetry c223cdaecd params/metrics: do not fail during dir expansion (#10375)
params/metrics: assume file when dir expansion raises error

Closes https://github.com/iterative/dvc/issues/10018.

We have to be careful when expanding paths. This PR fall backs to assume it
as a file when either of `isdir` and `find` raises error due to some mishaps.
The consequence of this assumption happens when we try to read the file.
In that case, if the directory exists, and we happen to try to open it
as a file, dvc will raise `IsADirectoryError`, but that is handled correctly
and won't fail catastrophically like this did.
2024-03-28 07:30:12 +05:45
pre-commit-ci[bot]andSaugat Pachhai 3375a7c39f [pre-commit.ci] pre-commit autoupdate (#10372)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.3.3 → v0.3.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.3.3...v0.3.4)

* fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-03-26 06:37:53 +00:00
dberenbaum 519c47f5fc sync: skip if remote doesn't match output remote 2024-03-25 15:12:33 -04:00
skshetry 6f6038888f sort imports for all libraries starting with dvc_ as a first party (#10371)
dvc_studio_client, dvc_ssh, etc. were not properly sorted.
2024-03-23 20:13:13 +05:45
skshetry fb628bf9ea add a regression test for #10309 (#10369)
https://github.com/iterative/dvc/issues/10309
2024-03-23 11:37:36 +05:45
skshetry c62a0380e9 templating: support on top-level params/metrics/plots/artifacts/datasets (#10359)
* templating: support on top-level params/metrics/plots/artifacts/datasets

* make resolving top-level ordered

* prevent updating parametrized dataset

* fix dataset tests

* add tests

* fix comment

* support parametrizing artifacts name
2024-03-22 12:22:16 +00:00
pre-commit-ci[bot]andSaugat Pachhai 49709a4810 [pre-commit.ci] pre-commit autoupdate (#10350)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.2.2 → v0.3.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.2.2...v0.3.2)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix ruff errors

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-03-13 16:43:10 +05:45
dberenbaum 837c8e8de4 exp: use exp git remote for studio repo url 2024-03-11 10:47:01 -04:00
dberenbaum 603da1ca1e api: update remote config tests 2024-03-11 09:47:05 -04:00
dberenbaum 377d0c9d58 get: update show-url to take remote args 2024-03-11 09:47:05 -04:00
dberenbaum 5aae934379 test auto push with tmp dir 2024-03-07 07:39:53 -05:00