Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
7,935 Commits 7 Branches 579 Tags
c29e721823f5cd5bdedcfd3c164e1996a33e594b
Commit Graph
7935 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Peter Rowlands c29e721823 output: support loading/dumping dir trees from metadata 2022-09-15 13:13:31 +03:00
Peter Rowlands a9c7032d50 dvc-data: bump to 0.8.0 2022-09-15 13:13:31 +03:00
Peter Rowlands ecb767d622 index: add workspace/key generation helper 2022-09-15 13:13:31 +03:00
Saugat Pachhai (सौगात) fa45a9bbba dvcfs: default open to binary mode 2022-09-15 13:11:52 +03:00
daniele f50ae589ba deps: remove unused mock dep (#8290) 2022-09-15 07:34:21 +05:45
Daniele Trifirò 611c114f01 import-url: pass fs_config down from imp_url to get_cloud_fs 2022-09-14 22:26:19 +03:00
skshetry fdc18d5483 expose dvcfs in dvc.api and add to fsspec's registry (#8287) 
* expose dvcfs in dvc.api and add to fsspec's registry

* readd __all__ to avoid exposing modules
 2022-09-14 20:02:56 +05:45
David de la Iglesia Castro 3b532b7d59 exp run: Support hydra basic sweeper. 
Requires `--queue` to be used alongside `--set-param`.

Doesn't strictly require `hydra.config.enabled`.

Add `get_hydra_sweeps` to `dvc/utils/hydra.py`.  Uses hydra `BasicSweeper` to generate the different sweeps.

`glob` syntax for sweeps is not supported in this P.R.

Closes #8083

---

Using setup from https://hydra.cc/docs/tutorials/basic/your_first_app/composition/ :

```console
$ dvc exp run -S "db=mysql,postgresql" -S "schema=warehouse,school" --queue
Queueing with '{'params.yaml': ['db=mysql', 'schema=warehouse']}'.
Queued experiment '5ab98b8' for future execution.
Queueing with '{'params.yaml': ['db=mysql', 'schema=school']}'.
Queued experiment '57c2fb6' for future execution.
Queueing with '{'params.yaml': ['db=postgresql', 'schema=warehouse']}'.
Queued experiment 'b9d6391' for future execution.
Queueing with '{'params.yaml': ['db=postgresql', 'schema=school']}'.
Queued experiment '145cd55' for future execution.
```
2.25.0 		2022-09-13 13:43:30 +02:00
skshetry e78c4455de dvcfs: remove unused config code for opening a repo, change signature (#8277) 
dvcfs: remove unused code for opening a repo
 2022-09-13 09:26:33 +05:45
skshetry 8e5f4a8f35 data ls: new command to show metadata with outputs (#8252) 
data ls: add new command to show metadata for the outputs
 2022-09-09 17:36:24 +05:45
Saugat Pachhai (सौगात) cb3159e586 dvcfs: remove config 2022-09-09 13:16:27 +03:00
skshetry 7b301b09a1 data status: fix type hints (#8275) 2022-09-09 13:38:41 +05:45
skshetry 55b16166c8 add/import/import-url: update help message for labels/type/desc (#8271) 2022-09-08 23:09:10 +05:45
Ronan Lamy c70d0bd14d fs: cleanup and rename _get_fs_pair_2() to _get_subrepo_info() 2022-09-08 17:39:32 +03:00
Ronan Lamy b99a0cd6ca fs: _get_dvc_path() cannot be called with dvc_fs=None 2022-09-08 17:39:32 +03:00
Ronan Lamy d4170b2ec0 fs: refactor _get_fs_pair_2() 2022-09-08 17:39:32 +03:00
Ronan Lamy a3181bb509 fs: remove _get_fs_pair() 2022-09-08 17:39:32 +03:00
Ronan Lamy 4d6b90502b fs: improve performance of _DvcFileSystem._update() 
Avoid back and forth conversions between key tuples and path strings.
 2022-09-08 17:39:32 +03:00
Ronan Lamy 947427f4d9 fs: remove dead code 2022-09-08 17:39:32 +03:00
Ronan Lamy 0482a1a88b fs: simplify data flow 
This uses the fact that repo.fs is always self.repo.fs and that
self.repo.fs always exists.
 2022-09-08 17:39:32 +03:00
Ronan Lamy b470459987 fs: simplify return values of _get_fs_pair_2() 2022-09-08 17:39:32 +03:00
Ronan Lamy 81a010674e fs: avoid costly redundant dvcignore check when calling ._info() from .ls() 2022-09-08 17:39:32 +03:00
Ronan Lamy d946a1328e fs: Extract ._info() from .info() 
This avoids unnecessarily recomputing dvc_parts when calling .info()
from .ls()
 2022-09-08 17:39:32 +03:00
Ronan Lamy ad7d39da20 fs: chop up ._get_fs_pair() 2022-09-08 17:39:32 +03:00
Saugat Pachhai (सौगात) 8432de5980 metadata: add support for dvc.yaml file 2022-09-08 15:10:33 +03:00
Saugat Pachhai (सौगात) c73764156e metadata: support custom metadata in .dvc files 
This adds support for custom metadata in outputs under `meta` key.
Also adds `--meta` flag to `add`/`import`/`import-url` that supports
`key=value` values.
 2022-09-08 14:10:35 +03:00
skshetry df60df5242 add metadata fields: label, type to data (#8232) 
gto like metadata fields: labels and type
 2022-09-08 16:33:47 +05:45
dependabot[bot] 504485ed6e build(deps-dev): Bump pylint from 2.15.0 to 2.15.2 (#8268) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.15.0 to 2.15.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.15.0...v2.15.2)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-09-08 10:57:12 +05:45
dependabot[bot] 1e989bde2c build(deps): Bump scmrepo from 0.1.0 to 0.1.1 (#8269) 
Bumps [scmrepo](https://github.com/iterative/scmrepo) from 0.1.0 to 0.1.1.
- [Release notes](https://github.com/iterative/scmrepo/releases)
- [Commits](https://github.com/iterative/scmrepo/compare/0.1.0...0.1.1)

---
updated-dependencies:
- dependency-name: scmrepo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-09-08 10:56:53 +05:45
Ruslan Kuprieiev 7e2248bbe5 deps: bump dvc-data to 0.7.1 2022-09-07 23:07:35 +03:00
Daniele Trifirò a2d80b2403 silence flake8 errors (abstract class with no abstract method) 2022-09-07 21:51:33 +03:00
Daniele Trifirò 76e7c76ae5 data status: fix for scmrepo 0.1.0 2022-09-07 21:48:53 +03:00
Ruslan Kuprieiev 6cada6dc12 dvc: cloud versioning POC 2022-09-07 21:43:34 +03:00
Ruslan Kuprieiev 89d4461908 deps: temporarily limit dulwich to <0.20.46 
Looks like the new release broke some data status logic.
 2022-09-07 02:56:14 +03:00
Ruslan Kuprieiev a124d7e866 deps: bump dvc-data to 0.6.3 2022-09-07 02:23:29 +03:00
Ruslan Kuprieiev 628be2ae49 deps: bump dvc-data to 0.6.1 2022-09-06 20:30:20 +03:00
dependabot[bot] 7a71e06946 build(deps): Bump dvc-render from 0.0.9 to 0.0.10 
Bumps [dvc-render](https://github.com/iterative/dvc-render) from 0.0.9 to 0.0.10.
- [Release notes](https://github.com/iterative/dvc-render/releases)
- [Commits](https://github.com/iterative/dvc-render/compare/0.0.9...0.0.10)

---
updated-dependencies:
- dependency-name: dvc-render
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-09-06 17:12:36 +02:00
dependabot[bot] 91e54d8c27 build(deps): Bump dvc-data from 0.4.0 to 0.5.3 2022-09-06 17:17:21 +03:00
skshetry 071b2b8f81 add support for foreach target (#8210) 
add general support for foreach target
 2022-09-06 06:39:37 +05:45
skshetry 0b2b59a601 dvc.yaml: preserve outputs' desc on rewrites/updates to the stage (#8247) 
preserve outputs' desc on rewrites/updates to the stage

We were not loading `desc` of the output, due to which we were not preserving the desc field over rewrites. Note that the rewrite only happens in select commands for `dvc.yaml`: `freeze`/`unfreeze`/`run`/`stage-add`, so we did not notice this before.
 2022-09-05 18:49:39 +05:45
dependabot[bot] 65dc1cd08f build(deps-dev): Bump pytest from 7.1.2 to 7.1.3 
Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.1.2 to 7.1.3.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.1.2...7.1.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-09-03 02:04:01 +03:00
dependabot[bot] 1b103297b7 build(deps-dev): Bump dvc-azure from 2.20.0 to 2.20.2 
Bumps [dvc-azure](https://github.com/iterative/dvc-azure) from 2.20.0 to 2.20.2.
- [Release notes](https://github.com/iterative/dvc-azure/releases)
- [Commits](https://github.com/iterative/dvc-azure/compare/2.20.0...2.20.2)

---
updated-dependencies:
- dependency-name: dvc-azure
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-09-03 02:03:46 +03:00
John Henning 887c73b3c6 adjusted formatting for dvc-hdfs 2022-09-02 23:04:59 +03:00
Peter Rowlands 660c17f654 output: save version_id meta 2022-09-01 19:36:53 +03:00
skshetry 49da2eef02 tests: set celery ping_task_timeout 3x the default (#8221) 
tests: set celery ping_timeout 3x the default
 2022-09-01 16:48:19 +05:45
Ruslan Kuprieiev aff65d42bb deps: bump dvc-azure and dvc-s3 to 2.20.0 2022-09-01 13:00:47 +03:00
skshetry 8968405770 data status: fix path for committed changes in Windows (#8220) 
The path we get from GitFileSystem is posixpath, even on Windows. We need to
convert that posixpath to os.path here.
2.24.0 		2022-09-01 10:02:08 +05:45
dependabot[bot] b6e8ba6758 build(deps-dev): Bump pylint from 2.14.5 to 2.15.0 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.14.5 to 2.15.0.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.14.5...v2.15.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...
 2022-09-01 10:42:16 +08:00
Ruslan Kuprieiev e307a62dc0 deps: bump dvc-data to 0.4.0 2022-09-01 02:59:24 +03:00
Saugat Pachhai (सौगात) 639eeed43b skip tests that depend on hydra.compose in 3.11 2022-08-31 17:24:09 +02:00