* dvc: refactor config
Advantages:
- read like a dict
- modify as a dict for testing or manipulation purposes
- wrap modifications into context manager to save them:
with config.edit("local") as conf:
conf["core"]["remote"] = "new-default"
Design decisions:
- dropped string constants
- remote sections are parsed into a simple subdict upon load
- remote config schema is now defined by url scheme
- relative paths are handled transparently upon load/save
- user config and remote manipulations are moved to commands
- same dev manipulations are done by simply modifying a config dict
* test: do not set cache.type for remotes that doesn't support it
* dvc: fix windows absolute paths validation in config
* test: fix windows separate drive test
* dvc: stylistic fixes in config
* add: auto convert absolute_path => relative
- fixes#2955
* use path_isin
* catch-all absolute paths in repo
* unneeded leftover
* revert realtive imports for later
* output: move abs2relpath from Base to Local
* output:local:fix parse_path
* remove path_info
* test:local versus base abs2relpath
* test:fix remaining abs2relpath test
* output: local path relative to wdir
* output: local: modify path before inheritance
* output: local: minor inheritance fix
* test: local: split output test
* test: local: positively explicit
- `tmp_dir` descends from Path
- convenient template methods to generate files and dirs, add them to
dvc and git and commit to git
- independent of old basic_env structures
- can go in any order
- modular: start with empty `tmp_dir`, initialize git with `scm`, dvc
repo with `dvc` add "repo template" with `repo_template`, ...
Imagine the scenario of shared cache. To set it up, user would do:
```
dvc config cache.dir /path
dvc config cache.shared group
```
Now, when he runs any dvc operation that involves the Repo instance,
RemoteLOCAL will also be initialized and it will currently auto-create
the directory with a regular `os.mkdir`, which is not using group
permissions. To solve that, we can't simply use `self.makdirs` instead,
as it will cause same issues if you run any dvc command before
`dvc config cache.shared group`. So what we should do is create
directories on demand, instead of creating it in advance, same as we do
in other remotes.
The change itself is pretty trivial and involved removing that `mkdir`
line, adjusting tests to not rely on cache dir existence and also
adjusting `dvc version` to not rely on cache directory existance.
Fixes#2563
* test: remove never used copy of load_stage_file()
* dvc: save comments and custom formatting in stage files
* Add a couple of TODOs
* test: test repro doesn't clear comments and formatting in .dvc
As part of this I started writing pytest-like tests and added commonly
used fixtures.
* test: simplify and prettify dvc formatting retained test
Also test that line comments after md5 fields are retained.
* test: fix tests
* dvc: fix stage and metrics opens not closed
* dvc: always use utf-8 in stage files
It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.
* test: unit test apply_diff()
* test: remove TODOs and ASK
* dvc: polish dvc-formatting PR
* dvc: move stage load/dump utils to a new stage utils file
* dvc: clean up apply_diff()
* test: test stage md5 ignores comments
* test: remove pytests starting marker
Paweł Redzyński
Alexander Schepanovski
Mr. Outis
Casper da Costa-Luis
Ruslan Kuprieiev
Fábio Santos
Jorge Orpinel
Saugat Pachhai
Ruslan Kuprieiev
Ivan Shcheklein
Andrew Grigorev