Commit Graph
3036 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 373dda6466 dvc: bump to 0.40.1 0.40.1 2019-05-12 00:01:01 +03:00
Ruslan Kuprieiev fc99ebed59 setup: match ruaml version with the one specified in requirements.txt 2019-05-12 00:00:19 +03:00
Ruslan Kuprieiev 750bd822b5 Merge pull request #1985 from efiop/fix-pyinstaller-oss2
pyinstaller: fix build for oss2
0.40.0
2019-05-11 21:55:22 +03:00
Ruslan Kuprieiev 797b356e3a pyinstaller: remove hooks that are already released
We've contributed these hooks a long time ago and they got finally
released, so we can remove our copies.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-11 21:44:55 +03:00
Ruslan Kuprieiev aacb0b4c98 pyinstaller: fix build for oss2
This version added support for oss as a remote, which has caused some
issues when building the binary package. To fix those, we need two
pyinstaller hools:

1) hook-Crypto.py
    This one has been added to PyInstaller's upstream, but wasn't
    released yet, so we are using a copy of it for now.

2) hook-aliyunsdkcore.py
    This one is a new one. Sent it to PyInstaller's team in a PR [1].

[1] https://github.com/pyinstaller/pyinstaller/pull/4228

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-11 21:35:41 +03:00
Ruslan Kuprieiev ea097780de remote: hdfs: fix typo 2019-05-11 17:33:32 +03:00
Ruslan Kuprieiev dc0c0914b2 Merge pull request #1984 from efiop/fix-version
cmd: use CmdBaseNoRepo for commands without repo
2019-05-11 17:28:15 +03:00
Ruslan Kuprieiev d4ef461fce version: use CmdBaseNoRepo
Currently `dvc version` outside of dvc repo raises "not dvc repo" error,
even though it is not really using repo in any way.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-11 17:20:30 +03:00
Ruslan Kuprieiev cc2f28c12b cmd: use CmdBaseNoRepo for commands without repo
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-11 17:09:19 +03:00
Ruslan Kuprieiev 3eaa725b01 Merge pull request #1968 from pared/1588
path info: replace path_info dicts with PathInfo class
2019-05-11 15:50:10 +03:00
Ruslan Kuprieiev 5075ea5feb Merge pull request #1982 from algomaster99/update-version-help
Update version help
2019-05-11 14:35:54 +03:00
pared 3ff10005fa path_info: remove dict path_info, introduce PathInfo class 2019-05-10 22:25:50 -07:00
Aman Sharma b3165a224e Update version help 2019-05-11 10:02:39 +05:30
Ruslan Kuprieiev 0a93de4bfe Merge pull request #1981 from mroutis/fix-1975
status: optimize the _fill_statuses method
2019-05-11 05:44:13 +03:00
Mr. Outis a91b198954 status: optimize the _fill_statuses method
Close #1975
2019-05-10 19:20:13 -07:00
Ruslan Kuprieiev f3c66bc866 Merge pull request #1980 from mroutis/fix-1964
install: append commands if hook already exists
2019-05-11 02:43:29 +03:00
Mr. Outis c39c2f087a install: append commands if hook already exists
Close #1964
2019-05-10 16:20:01 -07:00
Ruslan Kuprieiev 45f94e36b2 Merge pull request #1963 from algomaster99/dvc_version
Add `dvc version` command
2019-05-10 22:20:53 +03:00
Aman Sharma 46325808b8 Change regex pattern and move test under unit/command 2019-05-11 18:29:03 +05:30
Ruslan Kuprieiev 1322a5b2d8 Merge pull request #1978 from mroutis/fix-1973
logger: fix bug that ignored the progress bar
2019-05-10 21:25:15 +03:00
Aman Sharma bf9ca598c0 Pass codeclimate test 2019-05-10 19:45:17 +05:30
Jorge Orpinel b09bd0426e Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes for dvc diff (#1906)
* Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes in dvc.scm.git::Git::get_diff_trees

* diff: Changes to dvc.scm.git so `dvc diff` outputs short SHAs

- Docstring for Git class constructor
- Renames self.git for self.repo throughout module (involves several test changes)
- Better doc and logical refactor in Git._get_diff_trees
- `dvc diff` now outputs short SHA strings (Git._get_diff_trees) - hard coded to 7 length str
  (involves some test changes in tests/func/test_diff.py)

For #1902

* Undoes "Renames self.git for self.repo throughout module" introduced in 3d2dc83.

* diff: `dvc diff` uses code for `git rev_parse --short` instead of hard `[:7]` (for short commit hashes).

- Tests updated too.

Per https://github.com/iterative/dvc/pull/1906#discussion_r278510340

* tests: Reformatted with `black tests/func/test_diff.py`

* tests: Remove uneccessary `str()` casting form `dvc diff` tests (and reformat)
2019-05-10 15:30:36 +03:00
Aman Sharma db07f3a08d Modify tests to use pytest fixture 2019-05-10 13:00:42 +05:30
Mr. Outis 9978bc87a4 logger: fix bug that ignored the progress bar
Close #1973
2019-05-10 00:17:32 -07:00
Aman Sharma 21388952e9 Write tests for dvc version 2019-05-10 09:31:30 +05:30
Aman Sharma 9e8ad95e6d Change import of DVC version 2019-05-10 09:30:09 +05:30
Aman Sharma 7540fd7b48 Add dvc version command 2019-05-10 09:30:09 +05:30
nanaya-tachibana 7ceaf88c54 remote: add support for aliyun oss (#1961)
* remote: add support for aliyun oss

Usage:
$ dvc remote add myremote oss://my-bucket.endpoint/path
Set key id and key secret using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"

Ref:
oss python SDK: https://www.alibabacloud.com/help/doc-detail/32026.htm

* Not needed, since we don't support external dependencies/outputs on OSS.
See https://github.com/iterative/dvc/pull/1961.

* Add a way to test oss storage using docker.

Start a container running an oss emulator.
$ git clone https://github.com/nanaya-tachibana/oss-emulator.git
$ docker image build -t oss:1.0 oss-emulator
$ docker run --detach -p 8880:8880 --name oss-emulator oss:1.0
Setup environment variables.
$ export OSS_BUCKET='my-bucket'
$ export OSS_ENDPOINT='localhost:8880'
$ export OSS_ACCESS_KEY_ID='AccessKeyID'
$ export OSS_ACCESS_KEY_SECRET='AccessKeySecret'

* Use default key id and key secret when they are not given,
which gives read access to public read bucket and public bucket.

* test: add oss tests to appveyor.

* remove unneeded tests.

* remote: use s3 style url for oss storage and make endpoint a configurable value.

Usage:
$ dvc remote add myremote oss://my-bucket/path
Set key id, key secret and endpoint using modify command
$ dvc remote modify myremote oss_key_id my-key-id
$ dvc remote modify myremote oss_key_secret my-key-secret
$ dvc remote modify myremote oss_endpoint endpoint
or environment variables
$ export OSS_ACCESS_KEY_ID="my-key-id"
$ export OSS_ACCESS_KEY_SECRET="my-key-secret"
$ export OSS_ENDPOINT="endpoint"

* remote: fallback to [] if there are no cinfos

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* test: remote: add oss CLI test

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* travis: use iterative's fork of oss-emulator

Just to keep things in-house.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-09 15:35:41 +03:00
Ruslan Kuprieiev 6408b58b8d Merge pull request #1965 from khamutov/master
remote: add support for ssh config
2019-05-07 17:00:09 +03:00
khamutov 69a1d965e4 test: ssh: update mock version to 3.0.0 min 2019-05-07 11:28:50 +03:00
Ruslan Kuprieiev 6da0dd6f1c Merge pull request #1966 from msaroufim/master
removed munched sentence in readme
2019-05-07 01:18:30 +03:00
Mark Saroufim 0b6a29a5dc removed munched sentence in readme 2019-05-06 15:01:52 -07:00
khamutov 02eebd48eb test: ssh: add windows support for ssh config 2019-05-06 21:47:58 +03:00
khamutov 65fd7f9c8f test: ssh: add support for python 2.7 in mocking 2019-05-06 21:21:22 +03:00
khamutov a4632092a5 test: ssh: add mock_open compat version 2019-05-06 19:21:21 +03:00
Ruslan Kuprieiev 43a833dec2 dvc: bump to 0.40.0 2019-05-06 16:46:00 +03:00
khamutov 0afa99028a test: ssh: fixes wrong order of mocks 2019-05-06 09:05:39 +03:00
khamutov 2f1d5d49b4 test: ssh: mock ssh config file exists check 2019-05-05 21:09:05 +03:00
khamutov 0ac256cc96 add tests for ssh config 2019-05-05 19:37:49 +03:00
khamutov ce96334587 extract ssh config parsing 2019-05-05 13:20:28 +03:00
khamutov 7059ae3408 remote: add support for ssh config
Fixes #1613
2019-05-04 20:22:05 +03:00
Ruslan Kuprieiev 92cc9f7e6f Merge pull request #1962 from Suor/test-cloud-small-fix
test: skip data cloud cli tests properly, don't say it's passed
2019-05-02 14:39:33 +03:00
Alexander Schepanovski 7943add4c8 test: skip data cloud cli tests properly, don't say passed 2019-05-02 17:47:42 +07:00
Ruslan Kuprieiev 427b48d069 Merge pull request #1960 from pared/fix_dir_info_dumping_bug
remote: base: dump dir_info json with sorted keys
2019-05-02 09:58:02 +03:00
pared 28f7ccfc6c remote: base: dump dir_info json with sorted keys 2019-05-01 23:34:52 -07:00
Mr. Outis 30e38c91dc tests: use UTF-8 when creating metrics files (#1956) 2019-05-01 20:07:37 -07:00
Ruslan Kuprieiev 5da08289da test: ssh: remove test_connection 2019-05-02 05:56:49 +03:00
Ruslan Kuprieiev 61cb9b4af6 Merge pull request #1959 from mroutis/config-schema-refactor
config: implement bool_schema
2019-05-02 04:49:13 +03:00
Mr. Outis 60e83285e1 config: implement bool_schema 💅
Extracted from https://github.com/iterative/dvc/pull/1804/
2019-05-01 17:37:47 -07:00
Ruslan Kuprieiev 4e5ab29200 Merge pull request #1951 from pared/1551
install: add pre-push hook, refactor test
2019-05-01 21:41:01 +03:00