Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.
Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous
Both families might had caused undefined surprising behavior when local
and remote paths clash.
* Add info for link and filesystem type
* Remove unnecessary `repo_dir` fixture
* Add cache info
* Classify `test_version.py` as functional test
* Remove redundancy
* Optimize some functions involved
* Add pathlib2 to install_requires
* Display fs type for cache directory
* Use `dvc.utils.compat` to import Path
* Remove redundancy in tests and make logs more descriptive
As a part of this:
- DataCloud._cloud was refactored into `.get_remote()`
- data stopped using internal _cloud
- added a parametrization for remote tests
* Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes in dvc.scm.git::Git::get_diff_trees
* diff: Changes to dvc.scm.git so `dvc diff` outputs short SHAs
- Docstring for Git class constructor
- Renames self.git for self.repo throughout module (involves several test changes)
- Better doc and logical refactor in Git._get_diff_trees
- `dvc diff` now outputs short SHA strings (Git._get_diff_trees) - hard coded to 7 length str
(involves some test changes in tests/func/test_diff.py)
For #1902
* Undoes "Renames self.git for self.repo throughout module" introduced in 3d2dc83.
* diff: `dvc diff` uses code for `git rev_parse --short` instead of hard `[:7]` (for short commit hashes).
- Tests updated too.
Per https://github.com/iterative/dvc/pull/1906#discussion_r278510340
* tests: Reformatted with `black tests/func/test_diff.py`
* tests: Remove uneccessary `str()` casting form `dvc diff` tests (and reformat)
* remote: add support for aliyun oss
Usage:
$ dvc remote add myremote oss://my-bucket.endpoint/path
Set key id and key secret using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"
Ref:
oss python SDK: https://www.alibabacloud.com/help/doc-detail/32026.htm
* Not needed, since we don't support external dependencies/outputs on OSS.
See https://github.com/iterative/dvc/pull/1961.
* Add a way to test oss storage using docker.
Start a container running an oss emulator.
$ git clone https://github.com/nanaya-tachibana/oss-emulator.git
$ docker image build -t oss:1.0 oss-emulator
$ docker run --detach -p 8880:8880 --name oss-emulator oss:1.0
Setup environment variables.
$ export OSS_BUCKET='my-bucket'
$ export OSS_ENDPOINT='localhost:8880'
$ export OSS_ACCESS_KEY_ID='AccessKeyID'
$ export OSS_ACCESS_KEY_SECRET='AccessKeySecret'
* Use default key id and key secret when they are not given,
which gives read access to public read bucket and public bucket.
* test: add oss tests to appveyor.
* remove unneeded tests.
* remote: use s3 style url for oss storage and make endpoint a configurable value.
Usage:
$ dvc remote add myremote oss://my-bucket/path
Set key id, key secret and endpoint using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
$ dvc remote modify myremote oss_endpoint endpoint
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"
$ export OSS_ENDPOINT="endpoint"
* remote: fallback to [] if there are no cinfos
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* test: remote: add oss CLI test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* travis: use iterative's fork of oss-emulator
Just to keep things in-house.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Alexander Schepanovski
pawel
Ruslan Kuprieiev
Andrew Grigorev
Aman Sharma
Ruslan Kuprieiev
David Příhoda
Tadas Audinis
TomHortons
Mr. Outis
Jorge Orpinel
nanaya-tachibana
Brett Barkley