8378 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 8e1b75feb5 index: build_data_index: handle empty dirs (#8959)
Fixes https://github.com/iterative/dvc/issues/8958
2.43.3
2023-02-06 01:44:15 +02:00
Saugat Pachhai (सौगात) 4ec6d64629 add .git-blame-ignore-revs 2023-02-04 10:02:31 +05:45
skshetry a86470ccbc update black and change line-length to 88 (#8953) 2023-02-04 05:28:07 +02:00
dependabot[bot] 5d2701341b build(deps-dev): bump pylint from 2.16.0 to 2.16.1 (#8944)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-03 14:06:36 +05:45
Peter Rowlands 97589a1358 testing: add tests for pull after import-url --version-aware 2023-02-03 15:50:57 +09:00
Peter Rowlands 9e72ce53cd deps: bump dvc-data to 0.37.5 2023-02-03 15:50:57 +09:00
Peter Rowlands e34a6d3b30 update: fix meta & version ID merge on worktree update 2023-02-03 13:23:04 +09:00
Peter Rowlands 25dcc33580 output: build from files in get_obj() regardless is hash_info set 2023-02-03 13:23:04 +09:00
frederik-sandfort 18fa1a0272 Custom header for dvc webdav (#8933)Fix https://github.com/iterative/dvc/issues/8931
* include custom auth header for webdav

* Revert "include custom auth header for webdav"

This reverts commit d2f2b2889b6bcf0429e4a28076f709a04642ae3a.

* include custom auth header for dvc webdav, ran unit tests

* update dvc-webdav version
2.43.2
2023-02-02 16:36:11 +05:45
dependabot[bot] 49a27f4140 build(deps-dev): bump dvc-render from 0.0.17 to 0.1.0 (#8935)
Bumps [dvc-render](https://github.com/iterative/dvc-render) from 0.0.17 to 0.1.0.
- [Release notes](https://github.com/iterative/dvc-render/releases)
- [Commits](https://github.com/iterative/dvc-render/compare/0.0.17...0.1.0)

---
updated-dependencies:
- dependency-name: dvc-render
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-02 14:13:27 +05:45
dependabot[bot] 9b2aa79c18 build(deps-dev): bump pylint from 2.15.10 to 2.16.0 (#8936)
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.15.10 to 2.16.0.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.15.10...v2.16.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-02 14:12:53 +05:45
Ruslan Kuprieiev 24023c6977 index: introduce data_keys (#8938)
`data_keys` are similar to `outs`, but they provide more granularity, so we
can use them in `build_data_index` to rebuild index views with filtered
outputs. E.g. when running `dvc diff dir/file`, workspace index should only
have `dir/file` in it.

This also could be used for standalone (non-output) data in the future.

Pre-requisite for #8930
2023-02-02 05:50:16 +02:00
Ruslan Kuprieiev 5310bcdcec deps: bump dvc-data to 0.37.3 (#8923) 2023-02-02 04:18:18 +02:00
dependabot[bot] ae92154cec build(deps-dev): bump beautifulsoup4 from 4.11.1 to 4.11.2 (#8928)
Bumps [beautifulsoup4](https://www.crummy.com/software/BeautifulSoup/bs4/) from 4.11.1 to 4.11.2.

---
updated-dependencies:
- dependency-name: beautifulsoup4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-01 09:10:50 +05:45
Ruslan Kuprieiev f661cee8f6 stage: use repo.fs directly without copying (#8929)
Otherwise when using brancher we end up in a strange situation where even
though we are collecting targets from `gitfs`, we are still using `localfs`
as `self.fs`, which make operations like `self.fs.abspath` in
`collect_granular` return invalid paths.
2023-02-01 05:21:53 +02:00
Ruslan Kuprieiev e07c85a993 index: collect_targets: introduce onerror (#8925) 2023-02-01 03:49:27 +02:00
Ruslan Kuprieiev 58d9e379fe tests: gc: use timedelta again (#8926)
Leftover from #8912
2023-02-01 03:14:22 +02:00
skshetry 524162dc3f version/info: match dashes' length below DVC version (#8921) 2023-01-31 18:32:04 +05:45
skshetry aa15b21dce logging: log pkg version/type, platform info and cli arguments (#8918)
logging: log pkg version and type, platform info and cli arguments
2023-01-31 15:46:14 +05:45
Peter Rowlands 0404967591 deps: bump dvc-gs to 2.21.1 2023-01-31 17:26:44 +09:00
Peter Rowlands 465d886c29 testing: add tests for version-aware import-url 2023-01-31 15:55:44 +09:00
Peter Rowlands 459adfb40f import-url: disable push on version aware imports before saving 2023-01-31 15:55:44 +09:00
Peter Rowlands fe9f8654a3 testing: add test for worktree update 2023-01-31 15:55:44 +09:00
skshetry 8208846003 bump ruff to 0.0.238 (#8915) 2023-01-31 11:24:03 +05:45
Peter Rowlands 73aea2d972 ci: run dvc-s3 tests on DVC pull requests 2023-01-31 14:13:55 +09:00
Ruslan Kuprieiev 10184e9370 tests: gc: use timedelta (#8912)
Bug from https://github.com/iterative/dvc/pull/8880
2023-01-31 05:18:09 +02:00
Ruslan Kuprieiev 2690f05cee pre-commit: bump isort to 5.12.0
https://github.com/PyCQA/isort/issues/2077
2023-01-31 04:02:24 +02:00
Ruslan Kuprieiev 34c2c95115 index: expose rev property (#8909)
index: add rev property

This is very useful to identify indexes that are built from git revisions
and so are describing an immutable state and can be used for caching.

We were already using this for `__repr__`.
2023-01-30 20:18:54 +02:00
skshetry 5c6c1dc719 mypy: unignore module override (#8902) 2023-01-28 15:08:52 +05:45
98bcfad8e7 build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7 (#8821)
* build(deps-dev): bump scmrepo from 0.1.6 to 0.1.7

Bumps [scmrepo](https://github.com/iterative/scmrepo) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/iterative/scmrepo/releases)
- [Commits](https://github.com/iterative/scmrepo/compare/0.1.6...0.1.7)

---
updated-dependencies:
- dependency-name: scmrepo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* mypy: remove scmrepo from ignored modules

* Fix mypy issues

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: skshetry <[email protected]>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-01-28 14:26:00 +05:45
skshetry 7ab4c59d70 test: run pytest-style checks (#8901) 2023-01-28 14:10:53 +05:45
skshetry a137b2a6bc update ruff and enable flake8-logging-format and flake8-type-checking (#8894)
* update ruff and enable flake8-logging-format and flake8-type-checking check

* Update dvc/external_repo.py

* Update dvc/fs/dvc.py

* Update dvc/repo/experiments/__init__.py

* Update dvc/repo/experiments/queue/base.py

* Update pyproject.toml
2023-01-27 19:14:19 +05:45
Daniele Trifirò 6f0e8a8db5 tests: fix test_iter_revs mock 2023-01-27 12:52:46 +01:00
Daniele Trifirò d294945499 gc: fix usage with --date 2023-01-27 12:52:46 +01:00
Dave Berenbaum db70d1cc5c analytics: differentiate ci users (#8838) 2023-01-26 21:50:14 +05:45
Andrii Korzhuk ed6e11e856 Fixed typo in _format_field function 2023-01-26 13:50:40 +01:00
skshetry c1cd13c34f serialize: fix loading of cloud metadata from .dvc/dvc.lock files (#8889) 2023-01-26 16:18:03 +05:45
David de la Iglesia Castro 12dd746165 output: Pass version_aware=True to fs if files is present.
This will be ignored by filesystems that don't support `version_aware`.

Closes #8853
2023-01-26 10:48:59 +01:00
5165fb67a1 [pre-commit.ci] pre-commit autoupdate (#8869)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.223 → v0.0.230](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.223...v0.0.230)

* Update .pre-commit-config.yaml

* fix lint issues

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: skshetry <[email protected]>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-01-26 14:24:09 +05:45
Peter Rowlands b2c1c4dccd data_cloud: move version_aware config check to get_remote 2023-01-26 17:21:47 +09:00
Peter Rowlands b060cadbf1 deps: bump dvc-data to 0.36.2 2023-01-26 17:21:47 +09:00
Peter Rowlands 7c55a9c7cd testing: add initial versioned remote tests 2023-01-26 17:03:50 +09:00
Peter Rowlands 037b5b2e0a testing: add cloud versioning remote fixtures 2023-01-26 17:03:50 +09:00
dependabot[bot] 99acb8e335 build(deps-dev): bump dvc-data from 0.36.0 to 0.36.1 (#8887)
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.36.0 to 0.36.1.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.36.0...0.36.1)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-26 08:32:02 +05:45
Ruslan Kuprieiev b46cfbda5d deps: bump dvc-data to 0.36.0 (#8885) 2023-01-26 00:22:12 +02:00
Dave Berenbaum a8d193bf2a queue: update kill ui messages (#8798)
* queue: update kill ui messages

* queue kill: update ui, kill all valid tasks

* queue kill: update wording of ui
2023-01-25 20:08:27 +05:45
dependabot[bot] 0b1cf6c771 build(deps-dev): bump dvc-gs from 2.20.0 to 2.21.0 (#8841)
Bumps [dvc-gs](https://github.com/iterative/dvc-gs) from 2.20.0 to 2.21.0.
- [Release notes](https://github.com/iterative/dvc-gs/releases)
- [Commits](https://github.com/iterative/dvc-gs/compare/2.20.0...2.21.0)

---
updated-dependencies:
- dependency-name: dvc-gs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-25 19:29:21 +05:45
skshetry b3aa698cd4 dumpd: serialize files at the end in .dvc files (#8879) 2023-01-25 18:52:15 +05:45
skshetry bb9a3a0403 import/import-url: try not to delete dvcfile before import/import-url completes (#8876)
* import/import-url: try not to delete dvcfile before import/import-url completes

* use repo.stage.create instead of internal create_stage utility
2023-01-25 18:16:45 +05:45
skshetry 3427e115eb data status: check if change.old.hash_info is None or not (#8878)
data:status: check if change.old.hash_info is None or not
2023-01-25 14:24:38 +02:00