Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,944 Commits 7 Branches 579 Tags
bd2dc9dfed2f405aec5ec4be79250658ea557f7a
Commit Graph
8944 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
pre-commit-ci[bot]andSaugat Pachhai 502b3df6c9 [pre-commit.ci] pre-commit autoupdate (#9756) 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.9.0 → v3.10.1](https://github.com/asottile/pyupgrade/compare/v3.9.0...v3.10.1)
- [github.com/astral-sh/ruff-pre-commit: v0.0.278 → v0.0.281](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.278...v0.0.281)
- [github.com/PyCQA/flake8: 6.0.0 → 6.1.0](https://github.com/PyCQA/flake8/compare/6.0.0...6.1.0)

* fix issues reported by ruff

- also remove redundant flake8 plugins
- remove beautysh as we don't have any scripts now
- remove pyupgrade as it is redundant due to ruff

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-08-02 18:08:30 +05:45
dependabot[bot] 5c2fd67e6d build(deps-dev): bump pylint from 2.17.4 to 2.17.5 (#9769) 
Bumps [pylint](https://github.com/pylint-dev/pylint) from 2.17.4 to 2.17.5.
- [Release notes](https://github.com/pylint-dev/pylint/releases)
- [Commits](https://github.com/pylint-dev/pylint/compare/v2.17.4...v2.17.5)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-31 16:35:40 +05:45
Fabian Zills ed34c0f0bd dvc commit update description (#9760) 
* Update commit.py

* fix formatting
 2023-07-31 08:45:52 +00:00
Ruslan Kuprieiev 7c60dfd3c5 deps: bump dvc-data to >=2.8.1 3.10.1 2023-07-30 04:18:09 +03:00
Ruslan Kuprieiev 225814fdde dvc: don't use realpath where not needed 3.10.0 2023-07-30 02:24:00 +03:00
Ruslan Kuprieiev ca0ad239f7 deps: bump dvc-data to 0.28.0 3.9.1 2023-07-28 01:37:46 +03:00
Ruslan Kuprieiev f57ac609e9 deps: bump dvc-data to 0.27.1 3.9.0 2023-07-27 20:44:34 +03:00
Ruslan Kuprieiev cc4f505f99 api: open/read: introduce remote_config 2023-07-26 16:29:14 +03:00
Ruslan Kuprieiev 4d011718a6 out: str: don't try to relpath paths outside of the repo 
This is not very useful overall, but also results in a ValueError from
relpath on windows if your repo and output are on different drives.
3.8.1 		2023-07-25 23:49:27 +03:00
Ruslan Kuprieiev 9c89841637 fetch: support passing config 
For now not exposing it in CLI since there is no need/use yet.

Requirement for https://github.com/iterative/studio/pull/6541
3.8.0 		2023-07-25 20:03:38 +03:00
Peter Rowlands 92cae74512 exp.collect: strip metrics from queued experiments 2023-07-25 20:57:15 +09:00
Ruslan Kuprieiev 73dad4780f repo: brancher: reset parsed fields 
Related #9754 and fixes `dvc fetch`.
 2023-07-25 14:00:33 +03:00
Ruslan Kuprieiev 65ba9ba527 fs: get_cloud_fs: accept config instead of repo 
Config is a more immutable thing and using `repo` is confusing and might be error
prone in the bigger picture.

Related to #9754
 2023-07-25 00:58:47 +03:00
Peter Rowlands 7806afd4ad fs: prefer generic.copy over fsspec fs.get() 2023-07-24 17:01:34 +09:00
Ivan Shcheklein a9d0ac2ba9 fix(exceptions): add parent ctr init in InvalidArgumentError 3.7.0 2023-07-21 08:27:53 +02:00
Ruslan Kuprieiev 41df96fcdb cli: ls/import/get: fix help 
Leftover from https://github.com/iterative/dvc/pull/9747
 2023-07-20 21:53:14 +03:00
Ruslan Kuprieiev 53ec0fa41d ls/import/get: introduce --config 
Adds support for `--config` to all three commands. For `ls` and `get`, it is
a purely runtime option that will merge the config you provide with target
repo's configs (similar to how .dvc/config.local works). E.g.

```
$ cat myconfig
[remote "myremote"]
    access_key_id = 12345
    secret_access_key = 98765

$ dvc get https://github.com/efiop/mydataregistry recent-grads.csv --config myconfig
```

In case of `dvc import`, the semantics are the same, but the value is also
recorded in the resulting dvcfile. E.g.

```
$ dvc import https://github.com/efiop/mydataregistry recent-grads.csv --config myconfig
...
$ cat recent-grads.csv.dvc
md5: 85c86eeb3afa6d1f7d11beedc644d28e
frozen: true
deps:
- path: recent-grads.csv
  repo:
    url: https://github.com/efiop/mydataregistry
    config: myconfig
    rev_lock: af22e06bbfe34d62a140437abf32f48b535944a7
outs:
- md5: f447e23a170a9f25a7799f069a1ba934
  size: 26872
  hash: md5
  path: recent-grads.csv
```

This is the most general and powerful mechanism that we can give, that works
for everything from reconfiguring your default remote to completely changing
your remote layout and beyond (will be handy in the future).

Fixes #2466
 2023-07-20 18:41:10 +03:00
Dave BerenbaumandRuslan Kuprieiev 41194b7975 dvcfs: handle NotADirectoryError from datafs.ls (#9746) 
* dvc_fs: ignore NotADirectoryError in ls

* deps: bump dvc-data to >= 2.6.0

* add test

---------

Co-authored-by: Ruslan Kuprieiev <[email protected]>
3.6.0 		2023-07-19 20:54:55 +03:00
skshetry ea5f7d40c1 ruff: enable a few more rules, and merge tests' config with the main config (#9739) 2023-07-15 14:33:34 +05:45
dependabot[bot] eb06351062 build(deps-dev): bump mypy from 1.3.0 to 1.4.1 (#9664) 
Bumps [mypy](https://github.com/python/mypy) from 1.3.0 to 1.4.1.
- [Commits](https://github.com/python/mypy/compare/v1.3.0...v1.4.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-07-14 20:30:47 +05:45
pre-commit-ci[bot]andSaugat Pachhai ca3704997b [pre-commit.ci] pre-commit autoupdate (#9632) 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/asottile/pyupgrade: v3.3.1 → v3.9.0](https://github.com/asottile/pyupgrade/compare/v3.3.1...v3.9.0)
- https://github.com/charliermarsh/ruff-pre-commit → https://github.com/astral-sh/ruff-pre-commit
- [github.com/astral-sh/ruff-pre-commit: v0.0.261 → v0.0.277](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.261...v0.0.277)
- [github.com/codespell-project/codespell: v2.2.4 → v2.2.5](https://github.com/codespell-project/codespell/compare/v2.2.4...v2.2.5)

* fix ruff complaints

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
 2023-07-14 20:11:15 +05:45
Saugat Pachhai (सौगात) f1764bdc77 stage add: remove --external 2023-07-14 15:38:57 +03:00
skshetry eb212a7ba3 repro: introduce --keep-going/--ignore to continue on error (#9712) 
* repro: introduce --keep-going/--ignore to continue on error

`-k`/`--keep-going` will skip all dependents of a stage if it fails.
It'll still try to reproduce the other dependencies of the pending
targets, before it exits with a non-zero status.

`--ignore-errors` will treat any error during stage execution as a
success, and will continue executing other dependencies as if it
succeeded. Exits with 0 even on failure.

Also fixes:
* --force-downstream used to force everything even if one upstream nodes changed.
  This used to work when we supported a single targets, but there could be independent
  pipelines, so  we should not force everything if any node changes.
  Now we'll force reproduce the stage only if one of its dependency changed upstream
  or it's upstream node was also forced.

* Makes `--downstream` more correct, in relation to `--allow-missing` and `--pull`.
  Now we use an active graph for downstream/upstream logic instead of a subgraph.
  Note that this does not affect the downstream logic though, as dvc always goes
  downstream, so subgraph was no different than active graph.
  The subgraph only did not have the upstream nodes.

* add tests

* add spacing in the message
 2023-07-12 21:52:41 +05:45
skshetry 4c8f539c56 plots: log failures during datapoint conversions (#9726) 
Fixes #9441.

Note that we do merge props, and use that for all revision. So you may
see warning for multiple revisions.
 2023-07-12 19:14:31 +05:45
skshetry 45aefc8423 parsing: pass config instead of loading every time (#9729) 2023-07-12 17:22:42 +05:45
Peter Rowlands 67a747a356 stage.cache: override both legacy+local cache link types on save/commit 2023-07-12 17:17:49 +09:00
Peter Rowlands a5dda1b8ad open_repo: return without entering context for both local and erepo 2023-07-12 14:45:11 +09:00
Peter Rowlands dd4035bc05 repo: add get_data_index_entry to handle subrepo lookup 2023-07-12 14:45:11 +09:00
Peter Rowlands f69b3533e4 erepo: drop dvcx for_write support 2023-07-12 14:45:11 +09:00
Ruslan Kuprieiev 1b2291ca83 fetch: catch collection errors in revisions 
https://github.com/iterative/studio/pull/6541
3.5.1 		2023-07-11 04:03:26 +03:00
Ruslan Kuprieiev 4de4f0fd65 fetch: introduce --type metrics/plots 
Needed for https://github.com/iterative/studio/pull/6541 but is also generally useful.
3.5.0 		2023-07-10 15:24:30 +03:00
Ruslan Kuprieiev 2ef2caafd3 deps: bump dvc-data to 2.5.0 2023-07-09 20:30:35 +03:00
Ruslan Kuprieiev adcefbec66 output: pass explicit callback to hashfile.build() 2023-07-09 17:40:49 +03:00
Ruslan Kuprieiev 2102ba8465 output: fix formatting (#9716) 
Regression from https://github.com/iterative/dvc/pull/9715
 2023-07-09 19:50:00 +05:45
skshetry d736d0a3ef output: checkout: use f-string (#9715) 
Regression from https://github.com/iterative/dvc/pull/9673.
 2023-07-09 11:49:22 +05:45
dberenbaum 0d1f1f6436 update link for external data sources 2023-07-05 19:42:07 +03:00
Ruslan Kuprieiev 63de16ced1 readme: point to dvc.org for packages 
We no longer attach bin packages to github releases, so old link is completely misleading.
 2023-07-04 18:47:32 +03:00
Ruslan Kuprieiev 2f6c308c84 repo: site_cache_dir: take major version into account 
So that different major dvc versions don't use the same cache path.
 2023-07-04 18:24:20 +03:00
skshetry c8d180d2b0 repro: move code around (#9701) 
emphasizing on building the graph correctly,
and preparing for being able to work with graph directly.
 2023-07-04 21:02:53 +05:45
skshetry d69972d199 repro: ignore --downstream if used together with --pipeline (#9692) 
* repro: no need to build pipelines for --all-pipeline

* repro: ignore --downstream when using --pipeline
 2023-07-04 19:03:09 +05:45
Peter Rowlands eb1ed40228 repo.open_repo: use abspath for local fs URL 2023-07-04 12:28:09 +03:00
Peter Rowlands 120289128e dvcfs: cleanup subrepo contexts on close() 2023-07-04 12:28:09 +03:00
Peter Rowlands a607e7b3d7 dvcfs: return md5-dos2unix in info 2023-07-04 12:28:09 +03:00
Peter Rowlands 86e199c6f2 api.get_url: use index remote storage for getting remote URL 2023-07-04 12:28:09 +03:00
Ruslan Kuprieiev 8661abf224 fetch: introduce --max-size (#9696) 
Needed for iterative/studio#4782
3.4.0 		2023-07-04 02:17:53 +03:00
Jon Crallandskshetry 541def388b environment variable to show debug options (#9693) 
* environment variable to show debug options

* lint

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add message to debug opts

* spelling

* Add argument group and move cprofile-dump

line length

linelen

lint

* use DVC_DEBUG instead of DVC_SHOW_DEBUG_OPTIONS

* use -v instead of environ

* Update dvc/_debug.py

Co-authored-by: skshetry <[email protected]>

* fix

---------

Co-authored-by: skshetry <[email protected]>
 2023-07-04 01:35:33 +03:00
Ruslan Kuprieiev 58eb145b6a Delete hook-google_compute_engine.logger.py 2023-07-03 22:13:22 +03:00
Ruslan Kuprieiev 16465e0b63 Delete hook-dvc.system.py 2023-07-03 22:13:22 +03:00
Ruslan Kuprieiev 6614efd35a Delete hook-dvc.tree.gs.py 2023-07-03 22:13:22 +03:00
skshetry 2becfd9344 refactor repro: split logic for planning stages to run for repro (#9690) 
The planning logic has now been moved to `plan_repro` api, that allows us
to test without creating stages.
 2023-07-03 17:53:48 +05:45