* erepo: dvc: implement `update` for imports from ext repos
Fixes#1774Fixes#2139Fixes#2201
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* deps: repo: make private
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* update: only support repo imports
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* dvc: fixes dvc.scm.Git.get_diff_trees docstring imprecision.
Parentheses were used in the "Returns:" description but its a dict, so changed to curly brackets;
and specified the 2 possible forms of the return structure.
* dvc: updates zsh autocomplete script with current command help messages.
Comparing vs actually running each command's `-h` help output.
* dvc: begins updating bask autocomplete script...
Only reorders existing commands and adds comments for those missing.
* scripts: removes temporary comments in bash autocomplete script.
Per https://github.com/iterative/dvc/pull/2064#pullrequestreview-243578115
Related to #2069
* Coalesce `requirements.txt` and `tests/requirements.txt` into `setup.py`
* Remove requirements for tests
* Parse tests requirements in setup.py
* Append extra dependenices in test_requirements in setup.py
* Modify CI build to pass the tests
* Correctly install black requirements
* Install all requirements to pass the tests
* Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes in dvc.scm.git::Git::get_diff_trees
* diff: Changes to dvc.scm.git so `dvc diff` outputs short SHAs
- Docstring for Git class constructor
- Renames self.git for self.repo throughout module (involves several test changes)
- Better doc and logical refactor in Git._get_diff_trees
- `dvc diff` now outputs short SHA strings (Git._get_diff_trees) - hard coded to 7 length str
(involves some test changes in tests/func/test_diff.py)
For #1902
* Undoes "Renames self.git for self.repo throughout module" introduced in 3d2dc83.
* autocomplete: Brings scripts/completion/dvc.zsh up-to-date
- Reorganizes commands in alphabetical order;
- Updates the command dscriptions according to their current usage (-h) output;
- Adds comments for MISSING command entries
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* autocomplete: Adds missing commands to Zsh script
* cache: updates slow link type warning message (and test)
in accordance with https://github.com/iterative/dvc.org/pull/331
* Fixes accidental docstring rewrap
per https://github.com/iterative/dvc/pull/2003#discussion_r286206870
* Actually fixes accidental docstring rewrap...
per https://github.com/iterative/dvc/pull/2003#pullrequestreview-240994592
* Really fixes accidental docstring rewrap this time...
We've contributed these hooks a long time ago and they got finally
released, so we can remove our copies.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
This version added support for oss as a remote, which has caused some
issues when building the binary package. To fix those, we need two
pyinstaller hools:
1) hook-Crypto.py
This one has been added to PyInstaller's upstream, but wasn't
released yet, so we are using a copy of it for now.
2) hook-aliyunsdkcore.py
This one is a new one. Sent it to PyInstaller's team in a PR [1].
[1] https://github.com/pyinstaller/pyinstaller/pull/4228
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* remote: add support for aliyun oss
Usage:
$ dvc remote add myremote oss://my-bucket.endpoint/path
Set key id and key secret using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"
Ref:
oss python SDK: https://www.alibabacloud.com/help/doc-detail/32026.htm
* Not needed, since we don't support external dependencies/outputs on OSS.
See https://github.com/iterative/dvc/pull/1961.
* Add a way to test oss storage using docker.
Start a container running an oss emulator.
$ git clone https://github.com/nanaya-tachibana/oss-emulator.git
$ docker image build -t oss:1.0 oss-emulator
$ docker run --detach -p 8880:8880 --name oss-emulator oss:1.0
Setup environment variables.
$ export OSS_BUCKET='my-bucket'
$ export OSS_ENDPOINT='localhost:8880'
$ export OSS_ACCESS_KEY_ID='AccessKeyID'
$ export OSS_ACCESS_KEY_SECRET='AccessKeySecret'
* Use default key id and key secret when they are not given,
which gives read access to public read bucket and public bucket.
* test: add oss tests to appveyor.
* remove unneeded tests.
* remote: use s3 style url for oss storage and make endpoint a configurable value.
Usage:
$ dvc remote add myremote oss://my-bucket/path
Set key id, key secret and endpoint using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
$ dvc remote modify myremote oss_endpoint endpoint
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"
$ export OSS_ENDPOINT="endpoint"
* remote: fallback to [] if there are no cinfos
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* test: remote: add oss CLI test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* travis: use iterative's fork of oss-emulator
Just to keep things in-house.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Ruslan Kuprieiev
Ruslan Kuprieiev
Naba7
Jorge Orpinel
Kurian Benoy
Jorge Orpinel
Aman Sharma
Alexander Schepanovski
nanaya-tachibana