Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,554 Commits 7 Branches 579 Tags
58029e97191b0a2c6c041e4d6bc929b5f17c079b
Commit Graph
8554 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
skshetry 58029e9719 exp push: push experiments revs to studio (#9160) 
* exp push: push experiments revs to studio

* log json data that is sent
 2023-03-16 11:24:50 +05:45
Ruslan Kuprieiev 990e834f1d repo: open: handle bare git repos 
If we are asked to open a potentially unitialized repo, we need to make sure
it is not a bare git repo, so that we don't accidentally start putting dvc
files inside git control dir.

Discovered while working on #8789
 2023-03-16 02:03:31 +02:00
Ruslan Kuprieiev 33a2449c91 deps: repo: use _get_rev when creating dvcfs 2023-03-16 01:05:04 +02:00
Ruslan Kuprieiev e2ce95e17e deps: repo: pass url explicitly to DVCFileSystem 
Otherwise our wrapper ignores it.
 2023-03-16 00:35:15 +02:00
Ruslan Kuprieiev 53f9953cd1 index: data: support imports 
Stepping stone to simplifying `dvc fetch/pull` by using index.

Fetch handles regular imports through index already, but not repo
imports because their processing is much more involved (e.g. chained
imports) in the current arch.

With `FileStorage` support introduced into `DataIndex` and `datafs`
supporting imports overall, `dvcfs` can now handle repo imports (even
chained ones). This will soon allow us to handle repo imports the same
way we handle regular ones, improve performance and get rid of a lot of
messy code (e.g. DependencyRepo).

Related https://github.com/iterative/scmrepo/issues/207
Related https://github.com/iterative/dvc-data/pull/315
Related https://github.com/iterative/studio/issues/5261
 2023-03-15 20:32:13 +02:00
skshetry e6419b939d do not disable colorama in tests (#9187) 
skip disabling colorama in tests

It might affect pytest as it wraps sys.stderr/sys.stdout for capsys fixture.
This is a temporary hack, as we might get rid of colorama soon.
2.50.0 		2023-03-15 18:23:26 +05:45
Ruslan Kuprieiev 37ccc5e8d8 repo: create site_cache_dir with proper perms 
Fixes #9184
 2023-03-15 14:26:17 +02:00
skshetry 8312eb9b42 ui: disable colorama on write_json when highlighting is disabled (#9186) 
On a large json output (say ~300MB), this saves about ~6 seconds of time.
 2023-03-15 17:42:45 +05:45
Ruslan Kuprieiev c8745dd580 deps: bump dvc-data to 0.44.0 2023-03-15 03:41:19 +02:00
Ruslan Kuprieiev 1cf8b5e582 install: pre-commit: install all deps by default 
Fixes #9178
2.49.0 		2023-03-15 01:19:42 +02:00
Dave Berenbaum 12955da3c3 exp: fail fast in empty git repo (#8840) 2023-03-14 17:16:17 +00:00
David de la Iglesia Castro e73cc9b40a exp: Include STUDIO_REPO_URL in run_env. 
Closes #8951.
 2023-03-14 13:43:03 +01:00
Ruslan Kuprieiev b56f6b69dd exp: push/pull: don't include baseline commit 
Fixes #6592
 2023-03-14 14:00:14 +02:00
Ruslan Kuprieiev 54a6709d99 Update pyproject.toml 2.48.0 2023-03-13 21:03:33 +02:00
Ruslan Kuprieiev e4a5a2aec5 deps: bump dvc-data to >=0.43.0 2023-03-13 21:03:33 +02:00
skshetry 16ecd53b71 tests: remove gc.collect() from windows (#9159) 2023-03-13 13:37:57 +05:45
Ruslan Kuprieiev 612f3039e0 repo: move state/index/data_index to site_cache_dir 
We use all of these for optimization and should not depend on a potentially slow or broken
filesystem on which repository resides (could be HDD or CIFS/NFS/overlayfs/etc).

This PR means a few things:
    * all odd filesystems that dvc repo might be on will work quickly out of the box
    * when we start caching cloned repositories for `dvc get/import`, we will be
      able to cache indexes for them as well (note that bare clones don't have a
      worktree so there is no place for .dvc/tmp with the old approach).
    * `index.dir` and `state.dir` are now obsolete.
    * `dvc init` will cleanup site_cache_dir.

Related #5562
 2023-03-13 02:58:23 +02:00
Ruslan Kuprieiev 17f523bf86 deps: bump dvc-data to >=0.42.3 2.47.2 2023-03-11 02:01:31 +02:00
Ruslan Kuprieiev a0bf2d997a repo: data_index: fallback to in-memory 
Makes for a much simpler resource management elsewhere.
 2023-03-11 01:23:12 +02:00
Ruslan Kuprieiev 8ed9b911db deps: bump dvc-data to >= 0.42.2 2.47.1 2023-03-10 22:40:59 +02:00
skshetry ef22b1d77e plots: disable syntax highlighting on plots json output (#9155) 
I don't think it's worth it to highlight given the size of json output.
 2023-03-11 00:50:06 +05:45
skshetry aaa825a9a8 config: ignore unspecified keys from feature section (#9153) 
A warning is shown to the user for the unsupported/ignored keys, and then ignored.
The warning is raised only once per interpreter session.
 2023-03-10 21:52:05 +05:45
Ivan Shcheklein 2a40a55bb4 fix(exp): cleanup help messages 2023-03-10 11:13:06 +09:00
Ruslan Kuprieiev 3aa9f51e36 output: suppress cloud.pull errors on cache collection 
Fixes https://github.com/iterative/studio/issues/5261
2.47.0 		2023-03-09 22:44:32 +02:00
dependabot[bot] 1cfdc22549 build(deps-dev): bump pylint from 2.16.4 to 2.17.0 (#9141) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.16.4 to 2.17.0.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.16.4...v2.17.0)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-09 16:23:47 +05:45
skshetry 2b889baa5e plots: avoid printing anything on --json output (#9144) 2023-03-09 16:10:46 +05:45
Peter Rowlands 8589aa4001 tests: mark test_queue_remove_done xfail 2023-03-09 16:25:09 +09:00
Peter Rowlands 8cbc4c43f1 tests: use session-scoped exp celery fixtures when possible 2023-03-09 16:25:09 +09:00
Peter Rowlands 417c327143 exp: add session-scoped celery test fixtures 2023-03-09 16:25:09 +09:00
Peter Rowlands 204eebaf15 tests: use brancher.switch 2023-03-09 15:02:26 +09:00
Peter Rowlands 3a6fd75d98 repo: use brancher.switch 2023-03-09 15:02:26 +09:00
Peter Rowlands 351b952d5f brancher: add brancher.switch helper 2023-03-09 15:02:26 +09:00
dberenbaum fe7db32311 studio: handle revision errors when posting 2023-03-08 22:19:32 +01:00
Ivan Shcheklein 0bd34c8977 fix(push): allow no default remote 2023-03-08 21:26:47 +02:00
skshetry 48a63ca588 try to improve test speed (#9136) 
* tests: fix logic to gc.collect() on Windows + 3.11

* dvc.cli: cache get_main_parser()

* tests: isolate environment on a session-level

* bring back gc.collect()
 2023-03-08 22:11:42 +05:45
Gao 4459d6ac57 Add back an dvc ignore test 
Original from
https://github.com/iterative/dvc/pull/8767#discussion_r1070778749
 2023-03-08 12:03:07 +02:00
skshetry 08a50ee36e tests: error on ResourceWarning (#9123) 
* tests: error on ResourceWarning

* run-cache: avoid leaking temp files on Windows

* set returncode to popen to avoid ResourceWarning on gc.collect()
 2023-03-08 15:36:10 +05:45
David de la Iglesia Castro f1ea64c4ef studio: Send metrics on done event. (#9027) 2023-03-07 19:47:22 +01:00
dependabot[bot] 7ede45af2d build(deps-dev): bump mypy from 1.0.1 to 1.1.1 (#9126) 
Bumps [mypy](https://github.com/python/mypy) from 1.0.1 to 1.1.1.
- [Release notes](https://github.com/python/mypy/releases)
- [Commits](https://github.com/python/mypy/compare/v1.0.1...v1.1.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-07 23:13:54 +05:45
skshetry 51db1765ee migrate from appdirs to platformdirs (#9131) 
appdirs has been deprecated/unmaintained, platformdirs is now recommended.
Closes #9130.
 2023-03-07 23:13:28 +05:45
Peter Rowlands 9101fce761 proc: remove dvc.proc 2023-03-07 18:17:01 +09:00
Peter Rowlands 3f842a205f exp show: fix cached running checkpoint state bug 2023-03-07 16:55:23 +09:00
dependabot[bot] 77e900fed2 build(deps-dev): bump pylint from 2.16.3 to 2.16.4 (#9127) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.16.3 to 2.16.4.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.16.3...v2.16.4)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-03-07 10:17:04 +05:45
David de la Iglesia Castro dde456c923 plots: Fix match_defs_renderers when plots are defined in nt format. 
`plots.show` returned `definitions` in native `nt` format but sources in `posixpath` format so the matching was failing.

Closes #8689

- Add `test_show_plots_defined_with_native_os_path`.
2.46.0 		2023-03-06 19:16:29 +01:00
David de la Iglesia Castro cb9d3529af plots: Remove --from-config 2023-03-06 19:16:29 +01:00
David de la Iglesia Castro 876c10fcf5 plots: Remove unnecessary OrderedDict usage in load_sv. 2023-03-06 19:16:29 +01:00
skshetry 93eea0cd42 deps: bump scmrepo to 0.1.15; close subrepo.dvc on tests teardown (#9122) 2023-03-06 20:21:42 +05:45
skshetry 25fd274090 increase timeout for tests to 40 minutes 2023-03-06 20:04:26 +05:45
skshetry ddfa607437 tests: don't leak files/descriptors when tests finish (#9119) 
close open files/descriptors when tests finish
 2023-03-06 18:06:37 +05:45
skshetry c5a3566e49 deps (tests): require pytest-xdist>=3.2, use -n=logical workers and workstealing scheduler (#9116) 2023-03-06 15:54:45 +05:45