* dvc: refactor config
Advantages:
- read like a dict
- modify as a dict for testing or manipulation purposes
- wrap modifications into context manager to save them:
with config.edit("local") as conf:
conf["core"]["remote"] = "new-default"
Design decisions:
- dropped string constants
- remote sections are parsed into a simple subdict upon load
- remote config schema is now defined by url scheme
- relative paths are handled transparently upon load/save
- user config and remote manipulations are moved to commands
- same dev manipulations are done by simply modifying a config dict
* test: do not set cache.type for remotes that doesn't support it
* dvc: fix windows absolute paths validation in config
* test: fix windows separate drive test
* dvc: stylistic fixes in config
Some things are fixed along the way:
- no cache shared between incompatible funcs
- some incosistent exceptions are now consistent
- import updates for git files now works both for dvc and git repos
- "auto-generated-upstream already exists" fixed
- dvc.api.open()/read() works with git repos now
Fixes#3124, #2976. Closes#3172.
* imp: dedup and parallelize external fetches
This prevents same fetches for external artifacts, groups them by source
and fetches in parallel.
* imp: test that fetch works on non workspace deps
* get/import/url: handle existing output directory
Currently if you run
```
dvc get https://github.com/iterative/example-get-started model.pkl --out .
```
it will nuke your cwd. To fix that we need to properly handle existing
directories, same as usual unix cli utilities do that.
* travis: specify python version when installing from chocolatey
Fixes#2609
* ui/perf: do not clone same repo multiple times
Closes#2490
External repos are still created in /tmp. They might be reused until
main() end. If no perfect match is found then a repo with the same url
is copied (not cloned again) and adjusted to fit a request: rev and
cache_dir.
NOTE: test_download_error_pulling_imported_stage() was modified to not
rely on clone error.
* test: fix import update test
* test: fix external_repo test under Windows
* dvc: fix non-master default branch for external repos
* dvc: remove unused .unset_dir() method
* wip - pull is working
* use fetch instead of reproduce
* test pull and fetch
* style changes
* fetch: catch config error on fetch instead of get_remote
* output: remove unused guard clause
* fetch: define a more specific error instead of ConfigError
* fetch: display number of errors correctly
* exceptions: remove "Failed" prefix to download/upload
* fetch: catch other exceptions that might halt the download
* fetch: log catched exceptions
* fetch: reformat exception
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* fix formatting
* test: clean up test dirs
Closes#1770. Partially does #1888.
* dvc: close anything opened
* test: finalize git repo explicitly to remove test dir later
This was previously an issue on Windows.
Mr. Outis
Alexander Schepanovski
Ruslan Kuprieiev
Sujith H
Aman Sharma
Chris Kerr
pawel
Wojciech Baranowski
Dani Hodovic
Saugat Pachhai
Ivan Shcheklein
Ruslan Kuprieiev
vyloy