Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-05 23:09:32 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,744 Commits 7 Branches 579 Tags
8fb3eaaa08fc8bc50dcbe357c482bd7f62c2e67a
Commit Graph
143 Commits
Author SHA1 Message Date
Alexander Schepanovski eac0aa13b2 perf: use packaging.version instead of pkg_resources 
This is the same functionality without a cost of loading a huge package.
 2019-09-17 14:32:31 +07:00
Alexander Schepanovski 293a50d75c perf: use fast entry points on non-wheel install 
This avoids importing `pkg_resources` on startup, which saves around 200
ms.

NOTE: wheel installer uses its own entry points template, which is the same
fastentrypoints.py script provides, so no gain there.
 2019-09-12 22:33:59 +07:00
Paweł RedzyńskiandRuslan Kuprieiev 69f0de0090 google-resumable-media build problems (#2449) 
* google-resumable-media hook for pyinstaller

* working wtih g-r-m build with -e

* update google-cloud-storage

* google-cloud-storage hook: remove hiddenimports

* Update setup.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-09-12 12:12:37 +03:00
Ruslan Kuprieiev 05adcf9a4f Merge pull request #2461 from Suor/nfs 
dvc: switch to flufl lock
 2019-09-11 16:43:40 +03:00
Alexander Schepanovski d9d85f3cb0 dvc: fix locks in Python2.7/Windows 
Return to zc.lockfile lock in Python 2, NFS is not supported there
anyway since sqlite3 doesn't support URIs.
 2019-09-11 14:17:21 +07:00
Mr. Outis e0fd808ab0 setup: iterative/mock-ssh-server -> pypi mock-ssh-server 2019-09-09 11:15:49 -05:00
Mr. Outis b4507444d5 version: adjust get_fs_type to discover all partitions 2019-09-06 19:27:23 -05:00
Alexander Schepanovski 1bdaca0d6c dvc: switch to flufl lock 
This locks employs hardlink trick, which works on NFS and alike unlike
fcntl/open locks.

Closes #1918.
 2019-09-06 16:29:09 +07:00
Mr. Outis d7da4fcd37 fix formatting 2019-09-05 10:52:15 -05:00
Mr. Outis cdee18a7a6 fix formatting 2019-09-03 00:27:10 -05:00
Mr. Outis 69638527cb setup: mroutis/mock-ssh-server -> iterative/mock-ssh-server 2019-09-03 00:01:14 -05:00
Mr. Outis bbfaf3a22b setup: use mock-ssh-server version supporting lstat 2019-09-02 23:21:20 -05:00
pawel 331ab8c983 refactor note 2019-08-30 18:29:35 +02:00
Paweł RedzyńskiandRuslan Kuprieiev bd04972e9e Update setup.py 
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-08-30 18:13:49 +02:00
Paweł RedzyńskiandRuslan Kuprieiev 48ac0b7c27 Update setup.py 
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-08-30 18:13:40 +02:00
pawel 3c01f07ffd setup: add note why we restrict google-resumable-media version 2019-08-30 14:04:12 +02:00
pawel e766595687 workaround for google-resumable-media==0.3.3 build problem 2019-08-30 12:03:19 +02:00
Casper da Costa-Luis c9342754ed fixed with progress 
- fixes #2434
 2019-08-25 01:18:57 +01:00
Ruslan Kuprieiev 096a043e21 dvc: don't include gssapi 
If user needs it, he can install it by himself through pip.

https://github.com/iterative/dvc/pull/2424#issuecomment-523990535

https://github.com/iterative/dvc.org/pull/582
 2019-08-24 07:20:06 +03:00
joshua postel cbc5e1b52d setup.py: add gssapi requirement 
Fixes #2327
 2019-08-21 14:10:53 -04:00
Casper da Costa-Luis f90fd179b1 update tqdm version 2019-08-18 23:43:35 +01:00
Casper da Costa-Luis 7328ff4866 update tqdm requirement 2019-08-11 01:41:01 +01:00
Casper da Costa-Luis 1511fd6dcc add preliminary tqdm work 2019-08-10 14:11:02 +01:00
Ruslan Kuprieiev 60bcd3dc13 Merge pull request #2371 from Suor/streaming 
api: make .open() stream data for Azure
 2019-08-09 18:42:40 +03:00
Alexander Schepanovski e5e29065c8 api: make .open() stream data for Azure 
This employs generating temporary signed public url and then streaming
from it. The bulk of the code might be reused for other remotes.

Url streaming does resume on connection error using "Range" header and
regenerates public url too.

If remote class doesn't provide .open() then this falls back to pulling
a file and opening it locally.
 2019-08-09 21:51:43 +07:00
Ruslan Kuprieiev f8673e79c6 requirements: update ruamel.yaml 
Fixes #2334
 2019-08-08 14:26:35 +03:00
Max Risuhin 078a5c063f Update azure-storage-blob package (== 2.1.0) 2019-08-05 01:27:19 -07:00
Alex JozandRuslan Kuprieiev 2500ce64a6 Restrict ruamel.yaml version due import error (#2335) 
* restrict ruamel.yaml version due import error

* Update setup.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-07-29 00:49:29 +03:00
Ruslan Kuprieiev 4a4ddf078c version: temporarily make psutil optional 
Workaround for https://github.com/iterative/dvc/issues/2284

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-07-25 01:52:11 +03:00
Ruslan Kuprieiev 93f0062482 setup: freeze pyarrow version 2019-07-23 14:27:51 +03:00
Ruslan Kuprieiev f901ba5c10 tests: use moto pre-release 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-07-20 16:09:44 +03:00
Ruslan Kuprieiev 68bebe7944 remote: hdfs: use pyarrow and connection pool 
Related to #1629

Speeds up HDFS tests significantly. E.g. ExternalHDFS test goes from 600
sec to 190sec. test_open_external[HDFS] from 160sec to 3sec.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-07-20 01:12:17 +03:00
Ruslan Kuprieiev 958a4fbd3c tests: s3: use moto for multipart test 
Fixes #1889

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-07-18 20:51:49 +03:00
Ruslan Kuprieiev 99065bd42e dvc: networkx 2.3 doesn't support py2 anymore (#2250) 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-07-10 05:58:23 +03:00
Andrew Grigorev 40c10bd916 ci: pydocstyle==4.0.0 breaks flake8 2019-07-06 21:55:33 +03:00
Ruslan Kuprieiev 12c7689a74 dvc: update paramiko requirements 
Fixes #2155

No longer need to explicitly ignore paramiko warnings.

Big thanks to @cowlinator for the heads-up!

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-06-24 00:24:06 +03:00
Ruslan Kuprieiev 918ef646dc appveyor: test py2 and py3 
Related to #2166

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-06-23 13:33:47 +03:00
Ruslan Kuprieiev c7ca83e226 setup.py: install win-unicode-console when running on windows 
Fixes #2165
 2019-06-21 22:41:37 +03:00
Ruslan Kuprieiev 1ba8e0171d dvc: replace pkg concept with external repo 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-06-21 01:16:00 +03:00
pared c9b3bdc927 ignore: remove dulwich in favour of pathspec 2019-06-20 12:15:42 +02:00
Ruslan Kuprieiev d3bdca3f13 tests: use shortuuid for shorter paths 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-06-18 16:11:39 +03:00
Ruslan Kuprieiev 14dc0c0115 pkg: handle install/version errors 
Fixes #1772

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-06-18 16:09:06 +03:00
Ruslan Kuprieiev 914a587123 pkg: implement dvc pkg get command 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-06-17 18:12:36 +03:00
Alexander Schepanovski d80293c44d dvc: path info refactoring 
Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.

Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous

Both families might had caused undefined surprising behavior when local
and remote paths clash.
 2019-06-11 19:19:17 +07:00
Andrew Grigorev 5fce9a01e4 tests: fix for pytest>=4.6.0 caplog 2019-06-02 15:41:21 +03:00
Ruslan Kuprieiev 616d79f2d8 tests: freeze pytest version 2019-06-02 13:59:20 +03:00
Aman Sharma 688bcaa0e3 Add info for link type and filesystem name (#1997) 
* Add info for link and filesystem type

* Remove unnecessary `repo_dir` fixture

* Add cache info

* Classify `test_version.py` as functional test

* Remove redundancy

* Optimize some functions involved

* Add pathlib2 to install_requires

* Display fs type for cache directory

* Use `dvc.utils.compat` to import Path

* Remove redundancy in tests and make logs more descriptive
 2019-06-01 14:40:59 +03:00
Aman Sharma 55ad81bf9d Coalesce requirements.txt into setup.py (#2017) 
* Coalesce `requirements.txt` and `tests/requirements.txt` into `setup.py`

* Remove requirements for tests

* Parse tests requirements in setup.py

* Append extra dependenices in test_requirements in setup.py

* Modify CI build to pass the tests

* Correctly install black requirements

* Install all requirements to pass the tests
 2019-05-29 10:23:01 +03:00
Mr. Outis 7c0c89a688 upgpkg: requests >= 2.22.0 
Close #2008
 2019-05-16 13:28:36 -07:00
Ruslan Kuprieiev fc99ebed59 setup: match ruaml version with the one specified in requirements.txt 2019-05-12 00:00:19 +03:00