- `tmp_dir` descends from Path
- convenient template methods to generate files and dirs, add them to
dvc and git and commit to git
- independent of old basic_env structures
- can go in any order
- modular: start with empty `tmp_dir`, initialize git with `scm`, dvc
repo with `dvc` add "repo template" with `repo_template`, ...
* master:
address @efiop's suggestions
s3: add support for top level empty directories
remote: http: calculate length basing on response and not head call
test: remote: http: clear test file name
HTTPError: reformat error message
Update dvc/remote/http.py
remote: http: rename http error, refactor test
remote: http: raise exception when download response with error status code
Support GDrive as remote
setup: don't forget contextlib2
remote: refactor ssh ask password code
remote: protect all remote client/session creation code with locks
test: refactor & remove redundant test fixtures
NoRemoteInExternalRepoError: dont pass cause of exception
perf: optimize cache listing for local, ssh and hdfs
remote: small .save_info()/.get_checksum() cleanup
* tree: make open method use mode and encoding
* api: try to open using self.tree if output not found
* repo: rename OutputMissingException
* test: api: test adding not cached output
* api: open: exists test instead of try
* repo: open: try opening from tree first
* repo: open: handle missing and not cached files
* repo: open: separate find out logic from opening
Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.
Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous
Both families might had caused undefined surprising behavior when local
and remote paths clash.
As a part of this:
- DataCloud._cloud was refactored into `.get_remote()`
- data stopped using internal _cloud
- added a parametrization for remote tests
Alexander Schepanovski
Mr. Outis
Saugat Pachhai
Dani Hodovic
Ruslan Kuprieiev