Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,574 Commits 7 Branches 579 Tags
9b329d01bf6bcfdfe372a85ccf20235b7fb33372
Commit Graph
8574 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
David de la Iglesia Castro 9b329d01bf api: Add exp_save. 2023-03-23 10:33:55 +01:00
dependabot[bot] 5c442cb867 build(deps-dev): bump pylint from 2.17.0 to 2.17.1 (#9229) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.17.0 to 2.17.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.17.0...v2.17.1)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-23 09:24:52 +01:00
Ruslan Kuprieiev 2d5c15bee1 erepo: move legacy exceptions to api (#9228) 
* erepo: get rid of NoRemoteInExternalRepoError

This exception hasn't been making any sense at all since we've introduced
get and import, as they can be chained in repos.

* erepo: get rid of NoOutputInExternalRepoError

This is dead code these days.

* erepo: get rid of FileMissingError and PathMissingError

* api: preserve legacy exc wrappers
 2023-03-23 01:40:42 +02:00
Peter Rowlands 7868c1301c exp: enforce waiting for task completion in CI 2023-03-22 18:58:42 +09:00
Ruslan Kuprieiev dfe76c54d7 erepo: set subrepos/uninitialized explicitly 2023-03-22 02:26:44 +02:00
Ruslan Kuprieiev 392c15e1d4 erepo: pass rev instead of fs/scm 
Towards unifying `Repo.open` and `external_repo` behaviour to get rid
of the latter.
 2023-03-22 00:21:25 +02:00
David de la Iglesia Castro 121d56b739 hydra: resolve OmegaConf interpolations. (#9212) 
Closes #9196
2.51.0 		2023-03-20 21:38:22 +05:45
skshetry 66406e6c99 scm: fetch_all_exps pass kwargs to scm.fetch_refspecs (#9214) 2023-03-20 21:26:24 +05:45
skshetry 8399bf612a sconfig: add support for studio token (#9213) 
config: add support for studio token
 2023-03-20 20:59:52 +05:45
daniele 6c1b786564 parsing: fix foreach resolution with int keys (#8904) 2023-03-20 18:40:30 +05:45
Oded Messerandpre-commit-ci[bot] d1c5fcc1b8 Add template for epic/story issues (#9208) 
* Add template

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-03-20 09:56:55 +01:00
skshetry 4ee3271b0c dvc.repo.exp: import some modules lazily (#9207) 2023-03-18 17:24:01 +05:45
skshetry dae8515933 plots: small refactorings (#9206) 2023-03-17 19:42:27 +05:45
Saugat Pachhai (सौगात) f02dd83e53 close all streams in child 2023-03-17 14:43:44 +02:00
skshetry a2066d75f9 add a note why we are flushing (#9205) 2023-03-17 17:52:04 +05:45
skshetry 283d8245e1 flush streams before fork (#9203) 2023-03-17 17:14:34 +05:45
skshetry c52b12ac2a dvcfs: support caching remote streams; use that in plots (#9183) 
* dvcfs: support caching remote streams; use that in plots

* plots: show progress bar, cap max_workers to 16

* bump dvc-data to 0.44.1
 2023-03-17 16:04:16 +05:45
Peter Rowlands 440cb9d2bc ui: flush stdout/stderr before colorama re-init 2023-03-17 14:52:30 +09:00
David de la Iglesia Castro 2d820cccad deps: bump dvc-render>=0.3.1<0.4.0. 
For https://github.com/iterative/dvc-render/pull/124
 2023-03-16 21:36:02 +01:00
skshetry 30c3e3e4d4 exp push: set default endpoint (#9198) 2023-03-16 17:50:38 +05:45
skshetry 58029e9719 exp push: push experiments revs to studio (#9160) 
* exp push: push experiments revs to studio

* log json data that is sent
 2023-03-16 11:24:50 +05:45
Ruslan Kuprieiev 990e834f1d repo: open: handle bare git repos 
If we are asked to open a potentially unitialized repo, we need to make sure
it is not a bare git repo, so that we don't accidentally start putting dvc
files inside git control dir.

Discovered while working on #8789
 2023-03-16 02:03:31 +02:00
Ruslan Kuprieiev 33a2449c91 deps: repo: use _get_rev when creating dvcfs 2023-03-16 01:05:04 +02:00
Ruslan Kuprieiev e2ce95e17e deps: repo: pass url explicitly to DVCFileSystem 
Otherwise our wrapper ignores it.
 2023-03-16 00:35:15 +02:00
Ruslan Kuprieiev 53f9953cd1 index: data: support imports 
Stepping stone to simplifying `dvc fetch/pull` by using index.

Fetch handles regular imports through index already, but not repo
imports because their processing is much more involved (e.g. chained
imports) in the current arch.

With `FileStorage` support introduced into `DataIndex` and `datafs`
supporting imports overall, `dvcfs` can now handle repo imports (even
chained ones). This will soon allow us to handle repo imports the same
way we handle regular ones, improve performance and get rid of a lot of
messy code (e.g. DependencyRepo).

Related https://github.com/iterative/scmrepo/issues/207
Related https://github.com/iterative/dvc-data/pull/315
Related https://github.com/iterative/studio/issues/5261
 2023-03-15 20:32:13 +02:00
skshetry e6419b939d do not disable colorama in tests (#9187) 
skip disabling colorama in tests

It might affect pytest as it wraps sys.stderr/sys.stdout for capsys fixture.
This is a temporary hack, as we might get rid of colorama soon.
2.50.0 		2023-03-15 18:23:26 +05:45
Ruslan Kuprieiev 37ccc5e8d8 repo: create site_cache_dir with proper perms 
Fixes #9184
 2023-03-15 14:26:17 +02:00
skshetry 8312eb9b42 ui: disable colorama on write_json when highlighting is disabled (#9186) 
On a large json output (say ~300MB), this saves about ~6 seconds of time.
 2023-03-15 17:42:45 +05:45
Ruslan Kuprieiev c8745dd580 deps: bump dvc-data to 0.44.0 2023-03-15 03:41:19 +02:00
Ruslan Kuprieiev 1cf8b5e582 install: pre-commit: install all deps by default 
Fixes #9178
2.49.0 		2023-03-15 01:19:42 +02:00
Dave Berenbaum 12955da3c3 exp: fail fast in empty git repo (#8840) 2023-03-14 17:16:17 +00:00
David de la Iglesia Castro e73cc9b40a exp: Include STUDIO_REPO_URL in run_env. 
Closes #8951.
 2023-03-14 13:43:03 +01:00
Ruslan Kuprieiev b56f6b69dd exp: push/pull: don't include baseline commit 
Fixes #6592
 2023-03-14 14:00:14 +02:00
Ruslan Kuprieiev 54a6709d99 Update pyproject.toml 2.48.0 2023-03-13 21:03:33 +02:00
Ruslan Kuprieiev e4a5a2aec5 deps: bump dvc-data to >=0.43.0 2023-03-13 21:03:33 +02:00
skshetry 16ecd53b71 tests: remove gc.collect() from windows (#9159) 2023-03-13 13:37:57 +05:45
Ruslan Kuprieiev 612f3039e0 repo: move state/index/data_index to site_cache_dir 
We use all of these for optimization and should not depend on a potentially slow or broken
filesystem on which repository resides (could be HDD or CIFS/NFS/overlayfs/etc).

This PR means a few things:
    * all odd filesystems that dvc repo might be on will work quickly out of the box
    * when we start caching cloned repositories for `dvc get/import`, we will be
      able to cache indexes for them as well (note that bare clones don't have a
      worktree so there is no place for .dvc/tmp with the old approach).
    * `index.dir` and `state.dir` are now obsolete.
    * `dvc init` will cleanup site_cache_dir.

Related #5562
 2023-03-13 02:58:23 +02:00
Ruslan Kuprieiev 17f523bf86 deps: bump dvc-data to >=0.42.3 2.47.2 2023-03-11 02:01:31 +02:00
Ruslan Kuprieiev a0bf2d997a repo: data_index: fallback to in-memory 
Makes for a much simpler resource management elsewhere.
 2023-03-11 01:23:12 +02:00
Ruslan Kuprieiev 8ed9b911db deps: bump dvc-data to >= 0.42.2 2.47.1 2023-03-10 22:40:59 +02:00
skshetry ef22b1d77e plots: disable syntax highlighting on plots json output (#9155) 
I don't think it's worth it to highlight given the size of json output.
 2023-03-11 00:50:06 +05:45
skshetry aaa825a9a8 config: ignore unspecified keys from feature section (#9153) 
A warning is shown to the user for the unsupported/ignored keys, and then ignored.
The warning is raised only once per interpreter session.
 2023-03-10 21:52:05 +05:45
Ivan Shcheklein 2a40a55bb4 fix(exp): cleanup help messages 2023-03-10 11:13:06 +09:00
Ruslan Kuprieiev 3aa9f51e36 output: suppress cloud.pull errors on cache collection 
Fixes https://github.com/iterative/studio/issues/5261
2.47.0 		2023-03-09 22:44:32 +02:00
dependabot[bot] 1cfdc22549 build(deps-dev): bump pylint from 2.16.4 to 2.17.0 (#9141) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.16.4 to 2.17.0.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.16.4...v2.17.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-09 16:23:47 +05:45
skshetry 2b889baa5e plots: avoid printing anything on --json output (#9144) 2023-03-09 16:10:46 +05:45
Peter Rowlands 8589aa4001 tests: mark test_queue_remove_done xfail 2023-03-09 16:25:09 +09:00
Peter Rowlands 8cbc4c43f1 tests: use session-scoped exp celery fixtures when possible 2023-03-09 16:25:09 +09:00
Peter Rowlands 417c327143 exp: add session-scoped celery test fixtures 2023-03-09 16:25:09 +09:00
Peter Rowlands 204eebaf15 tests: use brancher.switch 2023-03-09 15:02:26 +09:00