Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
7,977 Commits 7 Branches 579 Tags
7fb9dcec12f3b324b197bcc2080610e483598877
Commit Graph
7977 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
karajan1001 7fb9dcec12 Add tests for executor status 
1. Move some basic test script from function tests to unit test.
2. Add success/failed tests for the status change of `tempdir`, `celery`, `workspace` running case.
 2022-09-26 16:25:47 +08:00
karajan1001 f23bc71eea Use task status to replace collected. 
fix: #8088

1. Use status to replace collected.
2. Make status show more accurate.

Fix lint
 2022-09-26 16:25:47 +08:00
karajan1001 8b326137c6 Add a new attribute status to ExecutorInfo file 
1. Add a new attribute status to ExecutorInfo file
2. Update running status to the executor infofile.
 2022-09-26 16:25:47 +08:00
karajan1001 90109bccd7 Refactor seperate the initialization of executor and setup environment 
Reason:
1. Need some more granular control over initialization progress of
   executor.

What done:
1. Seperate `init_git` and `init_cache` progress out from setup_executor
2. Move `set_ref` into `init_git`
 2022-09-26 16:25:47 +08:00
Ruslan Kuprieiev 8c484645fa cloud-versioning: better handling for directories 2022-09-26 02:54:17 +03:00
dberenbaum c039e9a6e7 merge-driver: support removes and changes 2022-09-23 23:22:08 +03:00
Ruslan Kuprieiev 10cfa0d5d7 fetch: use md5() before save() 2022-09-23 15:12:19 +03:00
dependabot[bot] ecab4a22fe build(deps): Bump dvc-data from 0.10.1 to 0.12.0 
Bumps [dvc-data](https://github.com/iterative/dvc-data) from 0.10.1 to 0.12.0.
- [Release notes](https://github.com/iterative/dvc-data/releases)
- [Commits](https://github.com/iterative/dvc-data/compare/0.10.1...0.12.0)

---
updated-dependencies:
- dependency-name: dvc-data
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-09-23 15:12:19 +03:00
dberenbaum 4f354f5333 pull: hide glob option 2022-09-23 14:13:25 +03:00
skshetry f6b1f01a5c aiohttp: remove AIOHTTP_NO_EXTENSIONS envvar (#8353) 
* aiohttp: remove AIOHTTP_NO_EXTENSIONS envvar

aiohttp now provides wheel for 3.11

* advertise 3.11 support
 2022-09-23 16:23:37 +05:45
Saugat Pachhai (सौगात) 6d17bcce93 cli: add help for foreach 2022-09-23 13:36:34 +03:00
karajan1001 8da04c7c81 Solve the on_diverged function not executed error. 
fix: #8348
 2022-09-23 17:20:31 +08:00
Paweł Redzyńskiandpre-commit-ci[bot] c617769a1d deps: bump codespell (#8199) 
[pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/codespell-project/codespell: v2.1.0 → v2.2.1](https://github.com/codespell-project/codespell/compare/v2.1.0...v2.2.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2022-09-22 19:31:24 +05:45
Daniele Trifirò 69c3c83833 import/import-url: ignore outs when using --no-download 2022-09-22 00:29:46 +03:00
David de la Iglesia Castro 122a68398e hydra: Fix sweeps on Defaults List. 
Defaults List overrides (`defaults/foo=1,2`) were not working because of the inclusion of the `/` character.
Fixed it by updating how we construct the `merged_overrides` and added a regression test.
 2022-09-21 15:48:19 +02:00
Martino Mensio 748c015638 fixed link to "get started: pipelines" docs 2022-09-21 13:45:39 +03:00
skshetry ba1ab86078 document dvcfs (#8332) 2022-09-21 11:30:15 +05:45
skshetry aaa467a96b fs.callback.RichCallback: add transient support (#8335) 2022-09-20 17:03:17 +05:45
Daniele Trifirò ed15e59115 deps: bump dvc-http to 2.27.2 2022-09-20 13:18:18 +03:00
skshetry e63cb31a66 dvcfs: remove get-file implementation, make dvcfs non-cachable (#8331) 
dvcfs: remove get-file implementation
 2022-09-20 15:24:44 +05:45
Peter Rowlands 4c1f172132 deps: bump dvc-data to 0.10.1 2022-09-20 18:30:35 +09:00
Ruslan Kuprieiev d2a31eb660 push: use update_pipeline=False for cloud versioning 2.27.2 2022-09-19 17:21:43 +03:00
Ruslan Kuprieiev 625b735502 deps: bump dvc-s3 2.27.1 2022-09-19 14:57:35 +03:00
skshetry 7e93fc8c4a lint: pin astroid version (#8323) 
See https://github.com/PyCQA/pylint/issues/7488. Pylint is failing on our codebase.
2.27.0 		2022-09-19 12:22:21 +05:45
skshetry 853a918d5c cli: rename cli util, hide data ls from the help (#8321) 2022-09-19 10:29:52 +05:45
skshetry b5f20f8467 use native concurrency (#8320) 2022-09-19 10:23:55 +05:45
skshetry 2f6bc70b45 remove mergify (#8319) 2022-09-19 09:50:56 +05:45
Daniele Trifirò 197f522175 deps: add testing group for dvc.testing requirements 2022-09-17 00:54:35 +03:00
Ruslan Kuprieiev 35e6f082a4 analytics: use iterative-telemetry for user_id lookup 2022-09-17 00:45:45 +03:00
Ruslan Kuprieiev 4531fec2b7 deps: bump dvc-data to 0.10.0 2022-09-16 22:56:56 +03:00
skshetry 391bb29be2 dvcfs: rename DvcFileSystem to DVCFileSystem (#8307) 2022-09-16 18:18:55 +05:45
Peter Rowlands 146bdbbe9b deps: bump dvc-azure to 2.20.4 2022-09-16 17:47:38 +09:00
Saugat Pachhai (सौगात) c1f659bdd7 dvcfs: prevent opening file object in write mode 2022-09-16 11:31:24 +03:00
dependabot[bot] 0fe8f4e368 build(deps): Bump dvc-render from 0.0.10 to 0.0.11 
Bumps [dvc-render](https://github.com/iterative/dvc-render) from 0.0.10 to 0.0.11.
- [Release notes](https://github.com/iterative/dvc-render/releases)
- [Commits](https://github.com/iterative/dvc-render/compare/0.0.10...0.0.11)

---
updated-dependencies:
- dependency-name: dvc-render
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2022-09-16 09:58:16 +02:00
Ruslan Kuprieiev 3cbf319e0d loader: don't forget to set meta.isdir 
Leftover from #8273

Fixes #8300
2.26.2 		2022-09-15 22:53:48 +03:00
Ruslan Kuprieiev 477a6e177f index: use tree obj if using inline files 
Leftover from #8273
2.26.1 		2022-09-15 21:25:28 +03:00
skshetry f854fc6365 try to split testsuites into two (#8297) 
try to split tests
 2022-09-15 22:11:02 +05:45
skshetry 1b90683e02 dvcfs: add fsspec-compliance tests (#8296) 
dvcfs: add tests
 2022-09-15 18:42:50 +05:45
skshetry d762372d55 schema: add strict schema validation for top-level plots (#8289) 
schema: add strict schema validation for plots
2.26.0 		2022-09-15 16:30:10 +05:45
Peter Rowlands f3d304580a schema: update lockfile schema for versioning and dir files 2022-09-15 13:13:31 +03:00
Peter Rowlands 0e0d46e232 output: lazy load tree from files list in get_obj 2022-09-15 13:13:31 +03:00
Peter Rowlands 95ae018ed8 push/fetch: support dirs in worktree remotes 2022-09-15 13:13:31 +03:00
Peter Rowlands c29e721823 output: support loading/dumping dir trees from metadata 2022-09-15 13:13:31 +03:00
Peter Rowlands a9c7032d50 dvc-data: bump to 0.8.0 2022-09-15 13:13:31 +03:00
Peter Rowlands ecb767d622 index: add workspace/key generation helper 2022-09-15 13:13:31 +03:00
Saugat Pachhai (सौगात) fa45a9bbba dvcfs: default open to binary mode 2022-09-15 13:11:52 +03:00
daniele f50ae589ba deps: remove unused mock dep (#8290) 2022-09-15 07:34:21 +05:45
Daniele Trifirò 611c114f01 import-url: pass fs_config down from imp_url to get_cloud_fs 2022-09-14 22:26:19 +03:00
skshetry fdc18d5483 expose dvcfs in dvc.api and add to fsspec's registry (#8287) 
* expose dvcfs in dvc.api and add to fsspec's registry

* readd __all__ to avoid exposing modules
 2022-09-14 20:02:56 +05:45
David de la Iglesia Castro 3b532b7d59 exp run: Support hydra basic sweeper. 
Requires `--queue` to be used alongside `--set-param`.

Doesn't strictly require `hydra.config.enabled`.

Add `get_hydra_sweeps` to `dvc/utils/hydra.py`.  Uses hydra `BasicSweeper` to generate the different sweeps.

`glob` syntax for sweeps is not supported in this P.R.

Closes #8083

---

Using setup from https://hydra.cc/docs/tutorials/basic/your_first_app/composition/ :

```console
$ dvc exp run -S "db=mysql,postgresql" -S "schema=warehouse,school" --queue
Queueing with '{'params.yaml': ['db=mysql', 'schema=warehouse']}'.
Queued experiment '5ab98b8' for future execution.
Queueing with '{'params.yaml': ['db=mysql', 'schema=school']}'.
Queued experiment '57c2fb6' for future execution.
Queueing with '{'params.yaml': ['db=postgresql', 'schema=warehouse']}'.
Queued experiment 'b9d6391' for future execution.
Queueing with '{'params.yaml': ['db=postgresql', 'schema=school']}'.
Queued experiment '145cd55' for future execution.
```
2.25.0 		2022-09-13 13:43:30 +02:00