Commit Graph
8388 Commits
Author SHA1 Message Date
Peter Rowlands b060cadbf1 deps: bump dvc-data to 0.36.2 2023-01-26 17:21:47 +09:00
Peter Rowlands 7c55a9c7cd testing: add initial versioned remote tests 2023-01-26 17:03:50 +09:00
Peter Rowlands 037b5b2e0a testing: add cloud versioning remote fixtures 2023-01-26 17:03:50 +09:00
dependabot[bot] 99acb8e335 build(deps-dev): bump dvc-data from 0.36.0 to 0.36.1 (#8887)
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.36.0 to 0.36.1.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.36.0...0.36.1)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-26 08:32:02 +05:45
Ruslan Kuprieiev b46cfbda5d deps: bump dvc-data to 0.36.0 (#8885) 2023-01-26 00:22:12 +02:00
Dave Berenbaum a8d193bf2a queue: update kill ui messages (#8798)
* queue: update kill ui messages

* queue kill: update ui, kill all valid tasks

* queue kill: update wording of ui
2023-01-25 20:08:27 +05:45
dependabot[bot] 0b1cf6c771 build(deps-dev): bump dvc-gs from 2.20.0 to 2.21.0 (#8841)
Bumps [dvc-gs](https://github.com/iterative/dvc-gs) from 2.20.0 to 2.21.0.
- [Release notes](https://github.com/iterative/dvc-gs/releases)
- [Commits](https://github.com/iterative/dvc-gs/compare/2.20.0...2.21.0)

---
updated-dependencies:
- dependency-name: dvc-gs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-25 19:29:21 +05:45
skshetry b3aa698cd4 dumpd: serialize files at the end in .dvc files (#8879) 2023-01-25 18:52:15 +05:45
skshetry bb9a3a0403 import/import-url: try not to delete dvcfile before import/import-url completes (#8876)
* import/import-url: try not to delete dvcfile before import/import-url completes

* use repo.stage.create instead of internal create_stage utility
2023-01-25 18:16:45 +05:45
skshetry 3427e115eb data status: check if change.old.hash_info is None or not (#8878)
data:status: check if change.old.hash_info is None or not
2023-01-25 14:24:38 +02:00
skshetry 3d5f4dc66c add support for output-specific remotes (#8862)
Also splits metadata in `files` to cloud vs local. But this happens
during dump/load, so actual behaviour is changed. During runtime,
the metadata is merged and loaded. During dump, the metadata
are split if there is a `remote` set.
2023-01-25 18:06:12 +05:45
David de la Iglesia Castro 80633f4fad stage.save: Remove merge_versioned option.
If we want `add`, `commit` and `move` to work for cloud versioning, option must be always True so it is the same as not having it.

Closes #8828
(We were not passing `merge_versioned=True` in `dvc commit`)
2023-01-25 10:35:37 +01:00
Ruslan Kuprieiev 8bda3650c5 deps: bump dvc-data to 0.35.3 2023-01-24 23:06:34 +02:00
David de la Iglesia Castro ea2e5dc3e5 output: Don't serialize hash_info or meta in cloud versioning.
Closes #8357
2023-01-24 18:40:39 +01:00
dependabot[bot] 61afdca3a6 build(deps-dev): bump dvc-data from 0.35.1 to 0.35.2 (#8871)
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.35.1 to 0.35.2.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.35.1...0.35.2)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-24 20:47:16 +05:45
Ruslan Kuprieiev 6a6bc7035a data: status: use data index (#8846)
Pre-requisite for #8761
2023-01-23 21:29:32 +02:00
Ruslan Kuprieiev ccc97622fa deps: bump dvc-ssh to 2.21.0 (#8865)
* deps: bump dvc-ssh to 2.21.0

* Update pyproject.toml
2.43.1
2023-01-23 01:36:52 +02:00
Ruslan Kuprieiev 916b5d169f deps: bump dvc-azure to 2.21.0 2023-01-23 00:46:58 +02:00
Peter Rowlands 38e9b2ba95 deps: bump dvc-data to 0.35.1 2.43.0 2023-01-21 03:44:02 +09:00
Dave Berenbaumandskshetry 18f039d7c7 add viztracer async option (#8856)
* add viztracer async option

* Update dvc/_debug.py

Co-authored-by: skshetry <[email protected]>
2023-01-20 22:29:11 +05:45
skshetry f1c9e9365b remove setup.py file (#8851) 2023-01-20 20:26:13 +05:45
Ruslan Kuprieiev b50d1ee674 deps: bump dvc-data to 0.35.0 2023-01-20 04:16:12 +02:00
Peter Rowlands 1c33024b2c deps: bump dvc-data to 0.34.0 2.42.0 2023-01-19 19:09:45 +09:00
David de la Iglesia CastroandPeter Rowlands dd2d2dce19 param: Fix status check for Tuples.
Closes #8803

Update dvc/dependency/param.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2023-01-19 08:19:16 +01:00
David de la Iglesia Castro 5f558d003a deps: bump dvc-data to 0.33.0
- Update `test_gc`
2023-01-18 15:17:07 +02:00
David de la Iglesia Castro 3676f35b7c output.remove: Pass explicit recursive flag.
Per changes in https://github.com/iterative/dvc-objects/pull/176
2023-01-18 15:17:07 +02:00
David de la Iglesia Castro 9af6fcad4a exp run: Fix --dry option.
Closes #8831
2023-01-17 13:32:20 +01:00
skshetry 6d80b72434 lint: run lint checks on all platforms (#8832) 2023-01-17 10:51:57 +05:45
pre-commit-ci[bot] 46d9290fc2 [pre-commit.ci] pre-commit autoupdate (#8830)
updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.222 → v0.0.223](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.222...v0.0.223)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-01-17 08:55:33 +05:45
skshetry c47c2123fb cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc (#8822)
* cleanup linters: get rid of pylint-pytest, run pylint only on dvc/ directory, etc.

* remove pylint direcitves

* Apply suggestions from code review

* Update pyproject.toml
2023-01-17 08:35:19 +05:45
Dave Berenbaum d25d26ff74 exp: drop some words from exp names (#8811) 2023-01-16 17:11:04 +01:00
Peter Rowlands c2e4f51479 serialize: fix mypy error 2023-01-16 15:58:21 +09:00
Peter Rowlands 977a1f9f43 tests: update for dvc-objects batched transfer 2023-01-16 15:58:21 +09:00
Peter Rowlands 608269dc88 status: remove deprecated/unused dvc-data.hashfile.status flag 2023-01-16 15:58:21 +09:00
Peter Rowlands e48d0d1d9c tests: remove duplicated dvc-objects/dvc-data tests 2023-01-16 15:58:21 +09:00
Peter Rowlands bf08bd3f12 deps: bump dvc-data to 0.32.0 2023-01-16 15:58:21 +09:00
dependabot[bot] db5dfdd6bc build(deps-dev): bump pytest from 7.2.0 to 7.2.1 (#8819)
Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.2.0 to 7.2.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.2.0...7.2.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 08:37:15 +05:45
dependabot[bot] 3b10f40ca3 build(deps-dev): bump dvc-task from 0.1.10 to 0.1.11 (#8820)
Bumps [dvc-task](https://github.com/iterative/dvc-task) from 0.1.10 to 0.1.11.
- [Release notes](https://github.com/iterative/dvc-task/releases)
- [Commits](https://github.com/iterative/dvc-task/compare/0.1.10...0.1.11)

---
updated-dependencies:
- dependency-name: dvc-task
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 08:36:39 +05:45
Ruslan Kuprieiev 4660443d0e deps: bump dvc-data to 0.31.4 2023-01-16 04:09:00 +02:00
Ruslan Kuprieiev a92ace0054 deps: bump dvc-data to 0.31.3 2023-01-16 01:16:35 +02:00
skshetry 309ae4d995 mypy: enable check-untyped-defs (#8815) 2023-01-15 21:40:40 +05:45
daniele e1254c9ece Add misc type hints (#8430) 2023-01-15 17:58:34 +05:45
skshetry 7a65ac06b2 mypy: don't ignore all imports, also disallow implicit optional (#8814)
* mypy: don't ignore all imports

* mypy: disallow implicit_optional
2023-01-15 16:25:03 +05:45
skshetry b0052aa666 pre-commit update: flake8/ruff/pyupgrade/black/isort (#8813) 2023-01-15 14:56:54 +05:45
skshetry da27174e02 dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups, etc. (#8812)
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups etc
2023-01-15 12:45:10 +05:45
Ruslan Kuprieiev aabdddc2b7 index: introduce build_data_index
This is a generally useful function that builds a data index from repo index.

This is currently only used in cloud versioning, but it is going to be used in
`status`, `diff` and other data-centric operations
2023-01-15 01:45:09 +02:00
karajan1001 fc1a192c90 Bump pathspec to 0.10.3
fix: #8217
1. Replace group name to avoid conflict.
2. Disable failed tests.
3. Bump pathspec to 0.10.3
2023-01-13 22:47:36 +08:00
dependabot[bot]andSaugat Pachhai 178244f666 build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10 (#8807)
* build(deps-dev): bump dvc-task from 0.1.9 to 0.1.10

Bumps [dvc-task](https://github.com/iterative/dvc-task) from 0.1.9 to 0.1.10.
- [Release notes](https://github.com/iterative/dvc-task/releases)
- [Commits](https://github.com/iterative/dvc-task/compare/0.1.9...0.1.10)

---
updated-dependencies:
- dependency-name: dvc-task
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* remove xfail on json.JSONDecodeError/PermissionError

JSONDecodeError should be fixed by the new release. Next we need to investigate
PermissionError.

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-01-13 17:25:36 +05:45
skshetry dba7d3c320 mypy: use typesafe cached_property (#8799) 2023-01-13 09:08:59 +05:45
David de la Iglesia Castro 8b075eb754 stage: Skip run-cache if any out has cache: false.
Closes https://github.com/iterative/dvc/issues/4428
2023-01-12 19:10:01 +01:00