Commit Graph
194 Commits
Author SHA1 Message Date
skshetry cb96cf1487 fix name (#10920) 2025-12-01 15:52:37 +05:45
skshetry 5e4daf6cfb deps: bump dvc-data to >=3.16.0 (#10513) 2024-08-13 07:15:05 +05:45
skshetry 9a1bcd7f1d nit: fit statement/expr in the same line instead of hanging (#10267)
The code are wrapped due to us using line-length=79 before, but now some
of the code can easily fit in 88 chars.

The formatter did not change this automatically when we changed line-length,
because of magic trailing commas that gets added when they turn into multiple lines.

Black/ruff wrapping them in parens and splitting them into multiple lines
was making it difficult to read.
2024-01-30 18:23:04 +05:45
Dave Berenbaumandpre-commit-ci[bot] 428c094b8d update msg for multiple overlapping outputs (#9862)
* update msg for multiple overlapping outputs

* update msg for multiple overlapping outputs

* fix type checking

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* use sets of stages to fix mypy errors

* fix tests/func/test_add.py::test_try_adding_multiple_overlaps

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-08-24 00:06:32 +03:00
pre-commit-ci[bot]andSaugat Pachhai 502b3df6c9 [pre-commit.ci] pre-commit autoupdate (#9756)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.9.0 → v3.10.1](https://github.com/asottile/pyupgrade/compare/v3.9.0...v3.10.1)
- [github.com/astral-sh/ruff-pre-commit: v0.0.278 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.278...v0.0.281)
- [github.com/PyCQA/flake8: 6.0.0 → 6.1.0](https://github.com/PyCQA/flake8/compare/6.0.0...6.1.0)

* fix issues reported by ruff

- also remove redundant flake8 plugins
- remove beautysh as we don't have any scripts now
- remove pyupgrade as it is redundant due to ruff

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-08-02 18:08:30 +05:45
Peter Rowlands e995492095 output: check legacy hash name in output.add() 2023-06-22 22:21:34 +09:00
Ruslan Kuprieiev c827654fe7 tests: use mocker instead of unittest.mock where possible
Had the "pleasure" of finding a mocker.patch leak into other tests and one
of the suspicions is that mixing unittest.mock and mocker might be somehow
related.

In any case, we should use mocker where possible anyway.
2023-06-18 17:10:45 +03:00
Ruslan Kuprieiev af1fb722c8 dvc: drop cached external outputs
Fixes #9531

Docs https://github.com/iterative/dvc.org/pull/4574 Kudos @dberenbaum
2023-06-09 18:34:36 +03:00
Peter Rowlands 2cd33e10ac output: serialize 3.x hash name in 'hash' field 2023-06-09 15:49:44 +03:00
Peter Rowlands 603f5fdba1 cache: move 3.x data objects into files/md5/ prefix 2023-06-09 15:49:44 +03:00
skshetry 46f3157f43 add: rename jobs to remote_jobs to mirror CLI; move validation to CLI (#9550)
- Follow up on https://github.com/iterative/dvc/pull/9522, renames jobs kwarg to remote_jobs
  to mirror CLI.
- Also moves validation logic to CLI.
2023-06-06 19:30:39 +05:45
skshetry 36fd69be21 add: remove file (#9548)
* add: remove `file`

Follow up to #9547.

* Update add.py

* fix tests
2023-06-06 18:44:19 +05:45
dberenbaum 2c4365513e add: deprecate --file 2023-06-06 14:57:07 +03:00
Ruslan Kuprieiev 78637713cf checkout: use index checkout
Needed for https://github.com/iterative/dvc/pull/9424
2023-06-04 23:27:01 +03:00
Dave Berenbaum 6ace5ed380 add: tweak stage overlap error message (#9514)
* add: tweak stage overlap error message

* add: instructions for how to fix overlap error
2023-06-02 00:46:55 +03:00
skshetry 355ebc72eb add: remove --recursive, rename fname to file (#9513)
add: remove --recursive
2023-05-26 12:07:13 +05:45
skshetry 0e38525d8d cli: remove dvc run (#9508)
Note that `Repo.run()` API still exists, which will be removed in
successive PRs. A lot of tests depend on it that it'll take time
before removing it.
2023-05-25 16:27:14 +05:45
skshetry e089ee0365 remove data ls and support for --meta/--desc/--label/--type cli flags (#9481) 2023-05-19 20:10:34 +05:45
skshetry 2342099bd8 add: support virtual operation (#9389) 2023-05-11 08:31:25 +05:45
dberenbaum 1f03c40dbe add: msg for overlapping parent 2023-05-02 19:03:19 +03:00
Oded Messer d203bf9024 Fix: Prettifying unexpected error on FileExists for get/get-url/imp-url/add and introducing --force (#9365) 2023-04-27 00:41:09 +00:00
Ruslan Kuprieiev 59a4c4f67c remote: set hash_name as md5
Fixes #7977
2023-03-01 13:26:24 +02:00
Karan Jakhar 73497dae9e Handle filenames which has ":" (#9000)
* Fixes #8773 issue

* added test for #8773 issue

* skip test for windows
2023-02-14 11:21:32 +00:00
Ruslan Kuprieiev 58295ca0ab repo: rename odb -> cache
We used to have it named `cache` before, but these days it is clear that
you can have three odbs: just odb (e.g. in-memory), cache (typically local)
or remote (typically cloud). Fixing naming here before proceeding with
introducing `out.odb` and `out.cache`.

In preparation for migration to https://github.com/iterative/dvc-data/pull/304
2023-02-08 23:05:45 +02:00
skshetry a86470ccbc update black and change line-length to 88 (#8953) 2023-02-04 05:28:07 +02:00
skshetry 7ab4c59d70 test: run pytest-style checks (#8901) 2023-01-28 14:10:53 +05:45
David de la Iglesia Castro ea2e5dc3e5 output: Don't serialize hash_info or meta in cloud versioning.
Closes #8357
2023-01-24 18:40:39 +01:00
skshetry c47c2123fb cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc (#8822)
* cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc.

* remove pylint direcitves

* Apply suggestions from code review

* Update pyproject.toml
2023-01-17 08:35:19 +05:45
skshetry da27174e02 dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups, etc. (#8812)
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups etc
2023-01-15 12:45:10 +05:45
skshetry 5441964fdb add ruff (#8793)
* add ruff linter

* enable few more checks
2023-01-11 15:20:38 +05:45
Saugat Pachhai (सौगात) 1cad8ea7fd migrate test_{add,analytics,init,unprotect,local,process}.py to pytest 2022-11-03 23:07:28 +05:45
Ronan Lamy 3a9d0244b5 test: rename dvc.testing.test_*.py so pytest doesn't handle them as test files 2022-10-01 01:03:09 +03:00
Ruslan Kuprieiev 4531fec2b7 deps: bump dvc-data to 0.10.0 2022-09-16 22:56:56 +03:00
Saugat Pachhai (सौगात) c73764156e metadata: support custom metadata in .dvc files
This adds support for custom metadata in outputs under `meta` key.
Also adds `--meta` flag to `add`/`import`/`import-url` that supports
`key=value` values.
2022-09-08 14:10:35 +03:00
skshetry df60df5242 add metadata fields: label, type to data (#8232)
gto like metadata fields: labels and type
2022-09-08 16:33:47 +05:45
Ruslan Kuprieiev 628be2ae49 deps: bump dvc-data to 0.6.1 2022-09-06 20:30:20 +03:00
Ruslan Kuprieiev fcc7240617 odb: use odb.path instead of odb.cache_dir 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev 8b3020aaf3 deps: bump dvc-data to 0.0.20 2022-07-08 21:37:38 +03:00
Ruslan Kuprieiev 2e21c92c4f dvc: use dvc-data 0.0.8 2022-06-17 22:08:33 +03:00
Saugat Pachhai (सौगात) af649af462 deps: bump dvc-data to 0.0.6; fix imports 2022-06-14 17:50:57 +05:45
Ruslan Kuprieiev ef689c5593 dvc: upgrade to dvc-objects 0.0.4 2022-06-13 18:03:53 +05:45
Daniele Trifirò 793f97bdd5 fix dvc-objects/data imports 2022-06-13 18:03:53 +05:45
Ruslan Kuprieiev 9ac2f49b57 Revert "fix dvc-objects/data imports"
This reverts commit 677da2b22c.
2022-06-10 12:01:07 +03:00
Daniele Trifirò 677da2b22c fix dvc-objects/data imports 2022-06-10 14:07:33 +05:45
Saugat Pachhai (सौगात) 9a2feaa4ed dvc: remove dvc.data, fix imports 2022-05-20 15:13:50 +05:45
Saugat Pachhai (सौगात) 37921d618d dvc: remove dvc.objects, fix imports 2022-05-20 14:02:08 +05:45
Saugat Pachhai (सौगात) f23d31af64 fs/data: move repo related utils to dvc
This moves get_fs_config/get_fs_cloud and ODBManager to DVC
as they require DVC related utilities, and we don't need them
strictly in dvc-objects/dvc-data.
This also adds support to ReferenceHashFile to load from
any filesystem, using it's module name and filename.
Now we can get rid of DVCFileSystem dependency in
ReferenceHashFile.
This is a temporary solution, we really need a registry.
(I am not quite sure if this works in pyinstaller mode,
but I think it should, it's how fsspec does it if I understand
correctly.)
2022-05-18 19:39:09 +03:00
Saugat Pachhai (सौगात) 570e3a5359 dvc.fs: move fs to dvc.objects.fs
Some of the APIs/FileSystems are reimported inside dvc.fs module.
2022-05-16 13:15:19 +03:00
Saugat Pachhai (सौगात) 3f5757a0a5 dvc.objects: move hashing functions to dvc.objects
This commit moves `HashedStreamReader`/`get_hash_file`/`file_md5`
into `dvc.objects.hash`.

The `get_hash_file` has been renamed to `hash_file`.
This also copies `relpath` and `is_exec` to the `dvc.fs.utils`.
2022-05-12 20:33:18 +03:00
Saugat Pachhai (सौगात) b4faa5616f system: change System class to a dvc.fs.system module 2022-05-12 18:02:42 +03:00