Commit Graph
1833 Commits
Author SHA1 Message Date
George KussumotoandSaugat Pachhai 0e38ccc031 fix(pull): normalize dependency path (#10446)
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-06-03 16:43:43 +05:45
Ivan Shcheklein 8f8d49d023 fix(push/pull): don't fail on unsupported run cache (#10441) 2024-05-27 14:37:44 +05:45
Dave Berenbaum 83ec952da1 improve repro --pull --run-cache test 2024-05-23 14:26:42 -04:00
Dave Berenbaum c0d7e2a8de test repro --pull failure 2024-05-23 14:26:42 -04:00
Dave Berenbaum 1f7733fec1 refactor 2024-05-23 14:26:42 -04:00
Dave Berenbaum 856c902b65 add tests 2024-05-23 14:26:42 -04:00
Dave Berenbaum ef507efa04 artifacts: get studio info from local config 2024-05-23 13:45:48 -04:00
Sigurd Spieckermann 5bfacec974 import/get: escape non-pattern LFS path parts (#10416) 2024-05-13 10:24:06 -07:00
skshetry 8517cccff4 artifacts: add support for monorepo (#10386) 2024-04-16 21:13:02 +05:45
skshetry c223cdaecd params/metrics: do not fail during dir expansion (#10375)
params/metrics: assume file when dir expansion raises error

Closes https://github.com/iterative/dvc/issues/10018.

We have to be careful when expanding paths. This PR fall backs to assume it
as a file when either of `isdir` and `find` raises error due to some mishaps.
The consequence of this assumption happens when we try to read the file.
In that case, if the directory exists, and we happen to try to open it
as a file, dvc will raise `IsADirectoryError`, but that is handled correctly
and won't fail catastrophically like this did.
2024-03-28 07:30:12 +05:45
pre-commit-ci[bot]andSaugat Pachhai 3375a7c39f [pre-commit.ci] pre-commit autoupdate (#10372)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.3.3 → v0.3.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.3.3...v0.3.4)

* fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-03-26 06:37:53 +00:00
dberenbaum 519c47f5fc sync: skip if remote doesn't match output remote 2024-03-25 15:12:33 -04:00
skshetry fb628bf9ea add a regression test for #10309 (#10369)
https://github.com/iterative/dvc/issues/10309
2024-03-23 11:37:36 +05:45
skshetry c62a0380e9 templating: support on top-level params/metrics/plots/artifacts/datasets (#10359)
* templating: support on top-level params/metrics/plots/artifacts/datasets

* make resolving top-level ordered

* prevent updating parametrized dataset

* fix dataset tests

* add tests

* fix comment

* support parametrizing artifacts name
2024-03-22 12:22:16 +00:00
pre-commit-ci[bot]andSaugat Pachhai 49709a4810 [pre-commit.ci] pre-commit autoupdate (#10350)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.2.2 → v0.3.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.2.2...v0.3.2)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix ruff errors

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-03-13 16:43:10 +05:45
dberenbaum 603da1ca1e api: update remote config tests 2024-03-11 09:47:05 -04:00
dberenbaum 5aae934379 test auto push with tmp dir 2024-03-07 07:39:53 -05:00
dberenbaum cf9855af45 handle auto push errors 2024-03-05 09:44:48 -05:00
cd1e228e55 experiments: auto push experiments (#10323)
* add auto_push for experiment on run and on save.
add config.exp.auto_push and config.exp.git_remote

* add func tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* separate auto_push behaviour out of commit

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-03-04 14:20:56 +05:45
skshetry 3b2e031e60 replace flaky with pytest-rerunfailures (#10335)
* Revert "limit pytest version to <8.1 (#10334)"

This reverts commit bc95b650a4.

* replace flaky with pytest-rerunfailures

`flaky` project seems to be unmaintained (see box/flaky#192).
`pytest-rerunfailures` seems to be maintained under `pytest-dev`
and tests 5 minor releases of pytest.

Note that this plugin is not compatible with `flaky` plugin.
You will have to uninstall it if it exists.
2024-03-04 11:12:45 +05:45
AlexandreKempf 0ee6d6eba8 add more extensive tests 2024-02-29 13:50:30 -05:00
pre-commit-ci[bot] 7c34370464 [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
2024-02-29 13:50:30 -05:00
AlexandreKempf b67b254287 add test 2024-02-29 13:50:30 -05:00
skshetry 9bb4501bf0 ds add: use --dvc/--dvcx/--url flags to accept url (#10326)
* ds add: use --dvc/--dvcx/--url flags to accept url

.. instead of relying on `scheme://`

I think this is more simpler from user's perspective, easier to
teach and easier to maintain as we won't have to do url parsing.

I am taking inspiration from `cargo install` has `--git`/`--rev`, etc.
to install from alternate sources.

```console
Options:
      --version <VERSION>     Specify a version to install
      --index <INDEX>         Registry index to install from
      --registry <REGISTRY>   Registry to use
      --git <URL>             Git URL to install the specified crate from
      --branch <BRANCH>       Branch to use when installing from git
      --tag <TAG>             Tag to use when installing from git
      --rev <SHA>             Specific commit to use when installing from git
      --path <PATH>           Filesystem path to local crate to install
      --root <DIR>            Directory to install packages into```
```

The `name` is also changed from an option to an argument.

### Examples

```console
dvc ds add example-get-started --dvc [email protected]:iterative/example-get-started.git
dvc ds add dogs --dvcx dogs
dvc ds add dogs --url s3://bucket/key/path
```

* add --version alias
2024-02-27 20:06:38 +05:45
Matt Seddon 9f66897bf7 plots: support x-dict in nested dvc.yaml (#10318) 2024-02-26 19:54:20 +00:00
skshetry 6cb37ac27b tests: add tests for datasets (#10321) 2024-02-26 21:45:09 +05:45
skshetry b243ccab1f tests: remove pytest-lazy-fixture (#10294)
Since it's no longer being maintained. See https://github.com/TvoroG/pytest-lazy-fixture/issues/65.
2024-02-08 17:10:58 +05:45
Dave Berenbaum 594716ec79 status: add option to skip imports (#10277) 2024-02-01 21:49:50 +02:00
skshetry 9a1bcd7f1d nit: fit statement/expr in the same line instead of hanging (#10267)
The code are wrapped due to us using line-length=79 before, but now some
of the code can easily fit in 88 chars.

The formatter did not change this automatically when we changed line-length,
because of magic trailing commas that gets added when they turn into multiple lines.

Black/ruff wrapping them in parens and splitting them into multiple lines
was making it difficult to read.
2024-01-30 18:23:04 +05:45
skshetry b647475a64 drop Python 3.8 support (#10265)
* drop Python 3.8

Note that this PR only makes typing related changes.
Other changes can be done once this gets merged, they
are relatively few in numbers.

* bump build python version

* add typing_extensions to lint group
2024-01-30 11:33:25 +02:00
skshetry 4eb073868c update ruff to 0.1.15 (#10263)
update ruff
2024-01-30 13:09:55 +05:45
Ruslan Kuprieievanddberenbaum df5b582c3f fetch: handle missing/modified imports (#10220)
tests: pull after imp-url no-download

Co-authored-by: dberenbaum <[email protected]>
2024-01-25 18:34:06 +02:00
18a9fea4af hydra: support plugins (#10240)
* hydra: support plugins

* Update dvc/utils/hydra.py

* fix tests

* load plugins after some checks run

---------

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
Co-authored-by: skshetry <[email protected]>
2024-01-23 19:58:14 +05:45
dberenbaum 48ec86de28 exp save: add target arg 2024-01-19 12:57:27 +09:00
Dave Berenbaum 5c31189a91 repro --pull: always pull deps (#10187)
* repro --pull: always pull deps

* drop unused allow_missing args
2024-01-18 05:11:03 +02:00
skshetry 426a7149a4 database: drop dbt implementation (#10222)
* remove dbt implementation in the database

* lazy load client

* fixup

* fixup

* fix server side cursors

* use exec_driver_sql for testing connection
2024-01-05 19:16:12 +05:45
skshetry 0100c14d6a import-db: support importing a table (#10219)
* import-db: support importing a table

* unhide the import-db command
2024-01-05 09:26:07 +05:45
Ruslan Kuprieiev f1de712710 index: data_tree: handle partial imports 2024-01-04 18:17:45 +02:00
dberenbaum bf5fe95c30 imp_url: gitignore during to_remote op 2024-01-03 16:12:45 +02:00
Ruslan Kuprieiev 041ea17233 index: fix storage for imports 2024-01-02 13:38:39 +02:00
Ruslan Kuprieiev f2e479422f index: mark remote as read_only for imports
Fixes #10206
2023-12-28 08:08:45 +02:00
skshetry e7b695dbef fix partial fetch (#10205)
Closes #10199. Bumps dvc-data to >=3.4.
2023-12-27 19:05:33 +05:45
dberenbaum 457d615a51 gc: fix test_gc_no_dir_cache 2023-12-20 00:21:23 +02:00
dberenbaum 827f9a7f03 gc: add --skip-failed 2023-12-20 00:21:23 +02:00
Ruslan Kuprieiev 3e4e143bee deps: update dvc-data to >=3 2023-12-15 08:02:40 +02:00
Ruslan Kuprieiev 92823f1d84 tests: repro: don't depend on particular order 2023-12-14 21:43:24 +02:00
Ruslan Kuprieiev 8ccae0a086 dvcfs: implement basic du 2023-12-12 23:56:19 +02:00
skshetry 77b451d118 database: rename Adapter to Client (#10151) 2023-12-09 11:28:47 +05:45
Dave Berenbaum 7c898cfad7 exp: ignore workspace errors during push/pull (#10128)
* exp: ignore workspace errors during push/pull

* refactor fetch revs warning

* mention failed revs are skipped

* add workspace arg to brancher

* pass workspace=False only for dvc exp push/pull
2023-12-07 13:46:18 +02:00
Ruslan Kuprieiev 6ff4342f02 deps: bump dvc-data to >=2.22.4 (#10139)
* deps: bump dvc-data to >=2.22.4

* Update test_odb.py
2023-12-06 11:51:05 +02:00