Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,579 Commits 7 Branches 579 Tags
da3bbccf66b34b915c189f0c2e3a49fed92210f0
Commit Graph
8579 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Dave Berenbaum da3bbccf66 Update duplicate output error msg (#9138) 
* update dup output error msg

* update dup output error msg

* stage add: add test for duplicated output error msg

* cleanup OutputDuplicationError code
 2023-03-23 15:54:44 +00:00
Ruslan Kuprieiev 40484e8d57 dvcfs: handle broken symlink/TreeError (#9231) 
* plots: handle broken symlink/TreeError

* dvcfs: ls: raise FileNotFoundError on broken symlink/TreeError

* deps: bump dvc-data to 0.45.1
 2023-03-23 15:38:21 +02:00
pre-commit-ci[bot]andDavid de la Iglesia Castro 48b56c83dc [pre-commit.ci] pre-commit autoupdate (#9168) 
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/charliermarsh/ruff-pre-commit: v0.0.254 → v0.0.257](https://github.com/charliermarsh/ruff-pre-commit/compare/v0.0.254...v0.0.257)
- [github.com/codespell-project/codespell: v2.2.2 → v2.2.4](https://github.com/codespell-project/codespell/compare/v2.2.2...v2.2.4)

* pre-commit: fix ruff errors

* pre-commit: fix flake8

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: David de la Iglesia Castro <[email protected]>
 2023-03-23 13:05:31 +00:00
Peter Rowlands f8bbf20b77 exp: use individual q for timeout dependent tests 2023-03-23 19:18:52 +09:00
Peter Rowlands cd2c4d0caf tests: remark flaky queue tests as xfail on windows 2023-03-23 19:18:52 +09:00
David de la Iglesia Castro 9b329d01bf api: Add exp_save. 2023-03-23 10:33:55 +01:00
dependabot[bot] 5c442cb867 build(deps-dev): bump pylint from 2.17.0 to 2.17.1 (#9229) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.17.0 to 2.17.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.17.0...v2.17.1)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-23 09:24:52 +01:00
Ruslan Kuprieiev 2d5c15bee1 erepo: move legacy exceptions to api (#9228) 
* erepo: get rid of NoRemoteInExternalRepoError

This exception hasn't been making any sense at all since we've introduced
get and import, as they can be chained in repos.

* erepo: get rid of NoOutputInExternalRepoError

This is dead code these days.

* erepo: get rid of FileMissingError and PathMissingError

* api: preserve legacy exc wrappers
 2023-03-23 01:40:42 +02:00
Peter Rowlands 7868c1301c exp: enforce waiting for task completion in CI 2023-03-22 18:58:42 +09:00
Ruslan Kuprieiev dfe76c54d7 erepo: set subrepos/uninitialized explicitly 2023-03-22 02:26:44 +02:00
Ruslan Kuprieiev 392c15e1d4 erepo: pass rev instead of fs/scm 
Towards unifying `Repo.open` and `external_repo` behaviour to get rid
of the latter.
 2023-03-22 00:21:25 +02:00
David de la Iglesia Castro 121d56b739 hydra: resolve OmegaConf interpolations. (#9212) 
Closes #9196
2.51.0 		2023-03-20 21:38:22 +05:45
skshetry 66406e6c99 scm: fetch_all_exps pass kwargs to scm.fetch_refspecs (#9214) 2023-03-20 21:26:24 +05:45
skshetry 8399bf612a sconfig: add support for studio token (#9213) 
config: add support for studio token
 2023-03-20 20:59:52 +05:45
daniele 6c1b786564 parsing: fix foreach resolution with int keys (#8904) 2023-03-20 18:40:30 +05:45
Oded Messerandpre-commit-ci[bot] d1c5fcc1b8 Add template for epic/story issues (#9208) 
* Add template

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2023-03-20 09:56:55 +01:00
skshetry 4ee3271b0c dvc.repo.exp: import some modules lazily (#9207) 2023-03-18 17:24:01 +05:45
skshetry dae8515933 plots: small refactorings (#9206) 2023-03-17 19:42:27 +05:45
Saugat Pachhai (सौगात) f02dd83e53 close all streams in child 2023-03-17 14:43:44 +02:00
skshetry a2066d75f9 add a note why we are flushing (#9205) 2023-03-17 17:52:04 +05:45
skshetry 283d8245e1 flush streams before fork (#9203) 2023-03-17 17:14:34 +05:45
skshetry c52b12ac2a dvcfs: support caching remote streams; use that in plots (#9183) 
* dvcfs: support caching remote streams; use that in plots

* plots: show progress bar, cap max_workers to 16

* bump dvc-data to 0.44.1
 2023-03-17 16:04:16 +05:45
Peter Rowlands 440cb9d2bc ui: flush stdout/stderr before colorama re-init 2023-03-17 14:52:30 +09:00
David de la Iglesia Castro 2d820cccad deps: bump dvc-render>=0.3.1<0.4.0. 
For https://github.com/iterative/dvc-render/pull/124
 2023-03-16 21:36:02 +01:00
skshetry 30c3e3e4d4 exp push: set default endpoint (#9198) 2023-03-16 17:50:38 +05:45
skshetry 58029e9719 exp push: push experiments revs to studio (#9160) 
* exp push: push experiments revs to studio

* log json data that is sent
 2023-03-16 11:24:50 +05:45
Ruslan Kuprieiev 990e834f1d repo: open: handle bare git repos 
If we are asked to open a potentially unitialized repo, we need to make sure
it is not a bare git repo, so that we don't accidentally start putting dvc
files inside git control dir.

Discovered while working on #8789
 2023-03-16 02:03:31 +02:00
Ruslan Kuprieiev 33a2449c91 deps: repo: use _get_rev when creating dvcfs 2023-03-16 01:05:04 +02:00
Ruslan Kuprieiev e2ce95e17e deps: repo: pass url explicitly to DVCFileSystem 
Otherwise our wrapper ignores it.
 2023-03-16 00:35:15 +02:00
Ruslan Kuprieiev 53f9953cd1 index: data: support imports 
Stepping stone to simplifying `dvc fetch/pull` by using index.

Fetch handles regular imports through index already, but not repo
imports because their processing is much more involved (e.g. chained
imports) in the current arch.

With `FileStorage` support introduced into `DataIndex` and `datafs`
supporting imports overall, `dvcfs` can now handle repo imports (even
chained ones). This will soon allow us to handle repo imports the same
way we handle regular ones, improve performance and get rid of a lot of
messy code (e.g. DependencyRepo).

Related https://github.com/iterative/scmrepo/issues/207
Related https://github.com/iterative/dvc-data/pull/315
Related https://github.com/iterative/studio/issues/5261
 2023-03-15 20:32:13 +02:00
skshetry e6419b939d do not disable colorama in tests (#9187) 
skip disabling colorama in tests

It might affect pytest as it wraps sys.stderr/sys.stdout for capsys fixture.
This is a temporary hack, as we might get rid of colorama soon.
2.50.0 		2023-03-15 18:23:26 +05:45
Ruslan Kuprieiev 37ccc5e8d8 repo: create site_cache_dir with proper perms 
Fixes #9184
 2023-03-15 14:26:17 +02:00
skshetry 8312eb9b42 ui: disable colorama on write_json when highlighting is disabled (#9186) 
On a large json output (say ~300MB), this saves about ~6 seconds of time.
 2023-03-15 17:42:45 +05:45
Ruslan Kuprieiev c8745dd580 deps: bump dvc-data to 0.44.0 2023-03-15 03:41:19 +02:00
Ruslan Kuprieiev 1cf8b5e582 install: pre-commit: install all deps by default 
Fixes #9178
2.49.0 		2023-03-15 01:19:42 +02:00
Dave Berenbaum 12955da3c3 exp: fail fast in empty git repo (#8840) 2023-03-14 17:16:17 +00:00
David de la Iglesia Castro e73cc9b40a exp: Include STUDIO_REPO_URL in run_env. 
Closes #8951.
 2023-03-14 13:43:03 +01:00
Ruslan Kuprieiev b56f6b69dd exp: push/pull: don't include baseline commit 
Fixes #6592
 2023-03-14 14:00:14 +02:00
Ruslan Kuprieiev 54a6709d99 Update pyproject.toml 2.48.0 2023-03-13 21:03:33 +02:00
Ruslan Kuprieiev e4a5a2aec5 deps: bump dvc-data to >=0.43.0 2023-03-13 21:03:33 +02:00
skshetry 16ecd53b71 tests: remove gc.collect() from windows (#9159) 2023-03-13 13:37:57 +05:45
Ruslan Kuprieiev 612f3039e0 repo: move state/index/data_index to site_cache_dir 
We use all of these for optimization and should not depend on a potentially slow or broken
filesystem on which repository resides (could be HDD or CIFS/NFS/overlayfs/etc).

This PR means a few things:
    * all odd filesystems that dvc repo might be on will work quickly out of the box
    * when we start caching cloned repositories for `dvc get/import`, we will be
      able to cache indexes for them as well (note that bare clones don't have a
      worktree so there is no place for .dvc/tmp with the old approach).
    * `index.dir` and `state.dir` are now obsolete.
    * `dvc init` will cleanup site_cache_dir.

Related #5562
 2023-03-13 02:58:23 +02:00
Ruslan Kuprieiev 17f523bf86 deps: bump dvc-data to >=0.42.3 2.47.2 2023-03-11 02:01:31 +02:00
Ruslan Kuprieiev a0bf2d997a repo: data_index: fallback to in-memory 
Makes for a much simpler resource management elsewhere.
 2023-03-11 01:23:12 +02:00
Ruslan Kuprieiev 8ed9b911db deps: bump dvc-data to >= 0.42.2 2.47.1 2023-03-10 22:40:59 +02:00
skshetry ef22b1d77e plots: disable syntax highlighting on plots json output (#9155) 
I don't think it's worth it to highlight given the size of json output.
 2023-03-11 00:50:06 +05:45
skshetry aaa825a9a8 config: ignore unspecified keys from feature section (#9153) 
A warning is shown to the user for the unsupported/ignored keys, and then ignored.
The warning is raised only once per interpreter session.
 2023-03-10 21:52:05 +05:45
Ivan Shcheklein 2a40a55bb4 fix(exp): cleanup help messages 2023-03-10 11:13:06 +09:00
Ruslan Kuprieiev 3aa9f51e36 output: suppress cloud.pull errors on cache collection 
Fixes https://github.com/iterative/studio/issues/5261
2.47.0 		2023-03-09 22:44:32 +02:00
dependabot[bot] 1cfdc22549 build(deps-dev): bump pylint from 2.16.4 to 2.17.0 (#9141) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.16.4 to 2.17.0.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.16.4...v2.17.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-09 16:23:47 +05:45