Fixes#2039
Required by #2012 to be able to fetch data through the API without
interuption from updater.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.
Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous
Both families might had caused undefined surprising behavior when local
and remote paths clash.
* Add info for link and filesystem type
* Remove unnecessary `repo_dir` fixture
* Add cache info
* Classify `test_version.py` as functional test
* Remove redundancy
* Optimize some functions involved
* Add pathlib2 to install_requires
* Display fs type for cache directory
* Use `dvc.utils.compat` to import Path
* Remove redundancy in tests and make logs more descriptive
* Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes in dvc.scm.git::Git::get_diff_trees
* diff: Changes to dvc.scm.git so `dvc diff` outputs short SHAs
- Docstring for Git class constructor
- Renames self.git for self.repo throughout module (involves several test changes)
- Better doc and logical refactor in Git._get_diff_trees
- `dvc diff` now outputs short SHA strings (Git._get_diff_trees) - hard coded to 7 length str
(involves some test changes in tests/func/test_diff.py)
For #1902
* Undoes "Renames self.git for self.repo throughout module" introduced in 3d2dc83.
* autocomplete: Brings scripts/completion/dvc.zsh up-to-date
- Reorganizes commands in alphabetical order;
- Updates the command dscriptions according to their current usage (-h) output;
- Adds comments for MISSING command entries
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* Update scripts/completion/dvc.zsh
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* autocomplete: Adds missing commands to Zsh script
* cache: updates slow link type warning message (and test)
in accordance with https://github.com/iterative/dvc.org/pull/331
* Fixes accidental docstring rewrap
per https://github.com/iterative/dvc/pull/2003#discussion_r286206870
* Actually fixes accidental docstring rewrap...
per https://github.com/iterative/dvc/pull/2003#pullrequestreview-240994592
* Really fixes accidental docstring rewrap this time...
Currently `dvc version` outside of dvc repo raises "not dvc repo" error,
even though it is not really using repo in any way.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* remote: add support for aliyun oss
Usage:
$ dvc remote add myremote oss://my-bucket.endpoint/path
Set key id and key secret using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"
Ref:
oss python SDK: https://www.alibabacloud.com/help/doc-detail/32026.htm
* Not needed, since we don't support external dependencies/outputs on OSS.
See https://github.com/iterative/dvc/pull/1961.
* Add a way to test oss storage using docker.
Start a container running an oss emulator.
$ git clone https://github.com/nanaya-tachibana/oss-emulator.git
$ docker image build -t oss:1.0 oss-emulator
$ docker run --detach -p 8880:8880 --name oss-emulator oss:1.0
Setup environment variables.
$ export OSS_BUCKET='my-bucket'
$ export OSS_ENDPOINT='localhost:8880'
$ export OSS_ACCESS_KEY_ID='AccessKeyID'
$ export OSS_ACCESS_KEY_SECRET='AccessKeySecret'
* Use default key id and key secret when they are not given,
which gives read access to public read bucket and public bucket.
* test: add oss tests to appveyor.
* remove unneeded tests.
* remote: use s3 style url for oss storage and make endpoint a configurable value.
Usage:
$ dvc remote add myremote oss://my-bucket/path
Set key id, key secret and endpoint using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
$ dvc remote modify myremote oss_endpoint endpoint
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"
$ export OSS_ENDPOINT="endpoint"
* remote: fallback to [] if there are no cinfos
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* test: remote: add oss CLI test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* travis: use iterative's fork of oss-emulator
Just to keep things in-house.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
- Keep current SSH remote connection test on real system
- Faked and Mocked SSH server which running on both Linux and Windows
- Add corresponding tests
Fixed#1704
* test: remove never used copy of load_stage_file()
* dvc: save comments and custom formatting in stage files
* Add a couple of TODOs
* test: test repro doesn't clear comments and formatting in .dvc
As part of this I started writing pytest-like tests and added commonly
used fixtures.
* test: simplify and prettify dvc formatting retained test
Also test that line comments after md5 fields are retained.
* test: fix tests
* dvc: fix stage and metrics opens not closed
* dvc: always use utf-8 in stage files
It used to encode in system default encoding. Not sure if PyYAML was
handling this silently or we were just lucky all the way.
* test: unit test apply_diff()
* test: remove TODOs and ASK
* dvc: polish dvc-formatting PR
* dvc: move stage load/dump utils to a new stage utils file
* dvc: clean up apply_diff()
* test: test stage md5 ignores comments
* test: remove pytests starting marker
- Use `version.py` to compute a dynamic version stored in `__version__`.
Version information was scattered across the source code
(`VERSION_BASE`, `VERSION`, `__version__`, and `version.py`).
- Read `__version__` from `setup.py` and **pin** it using custom
`build_py` command class.
This way, we can maintain consistancy between the build version
and the runtime one.
Also, this gives us the ability to run `setup.py` without depending on
on `dvc` itself.
Close#1824
Ruslan Kuprieiev
pared
Alexander Schepanovski
Aman Sharma
Mr. Outis
David Příhoda
Jorge OrpinelandRuslan Kuprieiev
Tadas Audinis
nanaya-tachibana
khamutov
ag613915cao
ag613915cao