Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-12 15:28:21 +01:00
Code Issues Packages Projects Releases Wiki Activity
2,385 Commits 7 Branches 579 Tags
9b72cf75c4491e4b97e914843f63243c3ba8263d
Commit Graph
68 Commits
Author SHA1 Message Date
Ivan Shcheklein 2e66119465 boto version up to make aws dependencies compatible with test requirements 2019-01-28 20:57:36 -08:00
Ruslan Kuprieiev c02fe1313f config: support 'global' and 'system' configs 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-12-01 02:31:14 +02:00
Ruslan Kuprieiev 532c9df053 dagascii: don't use unreleased wait_for_input method 
Fixes #1359

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-11-23 04:38:26 +02:00
Ruslan Kuprieiev 259f88e58f dagascii: use wait_for_input from asciimatics 
Fixes #1359

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-11-23 02:47:28 +02:00
Ruslan Kuprieiev 181ed5e55c reqs: update google-cloud-storage 2018-11-07 22:46:01 +02:00
Mr. Outis b6842fb089 updater: Add custom update message depending on package manager (#1287) 
* updater: Add custom update message depending on package manager

Close #938
Close #1031

* Reduce cognitive complexity and avoid too many return functions

* Use a dictionary instead of if/else

* Take into account pkg installation with OSX

* updater: add platform detection

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-11-07 19:04:52 +02:00
Ruslan Kuprieiev a6ea2f3db9 dvc: stop using ntfsutils module 
Fixes #1162

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-10-07 17:49:29 +03:00
Ruslan Kuprieiev 53b37bb266 show: use asciimatics to provide scrollable graph 
Fixes #1177

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-10-06 01:04:00 +03:00
Mr. Outis 2a7ec2d2df Update networkx package (== 2.1, >= 2.1) 
The latest version is 2.2 (https://pypi.org/project/networkx).

Updating it to prevent collisions with some package managers already
using the latest one.
 2018-10-04 18:08:27 -05:00
Ruslan Kuprieiev f4fc3f47e4 show: don't use asciicanvas package 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-09-28 21:42:59 +03:00
pared 89e03c8606 pipeline show add --dot option (#1141) 
* pipeline show add --dot option

* refactor common code, pydot in requirements

* code formatting

* code formatting
 2018-09-24 02:50:41 +03:00
Ruslan Kuprieiev ff63376d49 dvc: use ctypes instead of cffi 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-09-23 12:45:16 +03:00
Ruslan Kuprieiev 5279b1ae8f requirements: stop using obsoleted google-cloud 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-09-23 08:36:42 +03:00
Ruslan Kuprieiev 8b513fb336 setup: misc fixes 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-09-23 06:06:01 +03:00
Ruslan Kuprieiev 6a55d1502f dvc: avoid using pyreflink module 
Fixes #1149

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-09-22 12:11:18 +03:00
Ruslan Kuprieiev 9135944d7c reqs: use grandalf 0.6 2018-08-23 04:41:49 +03:00
Ruslan Kuprieiev c378e4b3d2 dvc: pipeline show: create ASCII visualizations for DAG 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-19 21:01:12 +03:00
Ruslan Kuprieiev f02bf73587 reqs: only requre futures pkg on python 2.7 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-03 20:55:23 +03:00
Ruslan Kuprieiev d0ce57adac reqs: move pytest to test-requirements 
Fixes #966

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-03 13:33:24 +03:00
Ruslan Kuprieiev eaa994527d reqs: don't forget futures 
Fixes #967

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-03 13:25:36 +03:00
Ruslan Kuprieiev f1a28c3812 dvc: support dvc add for external files 
Fixes #890

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-16 16:34:16 +03:00
Clemens Wolff 6882a781a4 Add Azure Blob Storage remote 
This change adds a new cloud remote based on Azure Blob Storage [1] and
the azure-storage-blob SDK [2].

The implementation closely follows the patterns laid out by the existing
cloud remotes (AWS and GCP). Notable differences include:

- No use of a lock in the progress-bar callback and no requirement to
  look up total transfer size ahead of time: the Azure Storage SDK
  provides the total transfer bytes and current transfer offset in its
  progress callback.

- When `exists` is called, check the existence of every blob instead of
  listing the entire container. Container list operations can be
  expensive so unless the number of checked files is very large, making
  separate requests is likely going to be more efficient.

- For increased flexibility, access credentials for the Azure Storage
  backend can be provided either connection-string style via the remote
  URL or via environment variables. The Azure Storage backend can be
  emulated via Azurite [3].

[1] https://azure.microsoft.com/en-us/services/storage/blobs/
[2] https://github.com/Azure/azure-storage-python
[3] https://github.com/Azure/azurite

This change was created together with @EricSchles.
 2018-07-09 14:47:40 -04:00
Ruslan Kuprieiev 5d3a009886 metrics: require ply>=3.9 to avoid bug on binary windows package 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-08 01:02:26 +03:00
Ruslan Kuprieiev fba1d8c181 dvc: use heuristics to detect binary files 
Fixes #859

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-06 01:32:14 +03:00
Ruslan Kuprieiev 82890ec2a4 reqs: lock google-cloud==0.32.0 
0.33.0 is broken.
 2018-05-22 23:54:59 +03:00
Ruslan Kuprieiev 7709f57f0b reqs: add pytest(required by setuptools on some systems) 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 18:30:15 +03:00
Ruslan Kuprieiev 0f8f682a6b build: remove uneeded deps of boto 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 16:40:27 +03:00
Ruslan Kuprieiev 91882bf8c4 reqs: freeze google-compute-engine at 2.4.1 2018-05-17 15:15:38 +03:00
Ruslan Kuprieiev d9a2cbb3ce reqs: add wheel to requirements.txt 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 14:04:11 +03:00
Ruslan Kuprieiev 76ed66e1af reqs: require PyInstaller 3.3.1 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 13:09:48 +03:00
Ruslan Kuprieiev 03ebdb5cf2 dvc: use our own api instead of github's for version retrieval 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-10 14:40:59 +03:00
Ruslan Kuprieiev 6e20e1f845 dvc: check updates and warn about using an older version 
Fixes #687

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-06 22:12:28 +03:00
Ruslan Kuprieiev 0a049c232b cache: configurable cache dir and cache type 
From now on we allow specifying standalone cache dirs as well as
specifying linking method(copy, symlink, hardlink, reflink) for it.
If 'cache type' is not specified in the config, dvc will try all of
them starting from reflink all the way down to copy.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-28 21:50:49 +03:00
Ruslan Kuprieiev 832af146c6 requirements: add hidden requirement 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-19 21:03:07 +03:00
Ruslan Kuprieiev 1e7d548ca4 dvc: add dvc metrics 
Fixes #592

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-19 20:47:37 +03:00
Ruslan Kuprieiev f30e6e53c5 utils: convert to unix format when calculating md5 
Fixes #619

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-14 22:07:29 +03:00
Ruslan Kuprieiev 3748164ddb cloud: aws: migrate to boto3 
This allows us to abandon our own additional config parameters for
aws cloud(like region) and use the ones from .aws/config by default.
Plus, boto3 automatically switches to multipartupload if file is bigger
than a certain limit, allowing us to remove a lot of our own code.

Fixes #650
Fixes #636
Fixes #601

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-13 06:06:52 +03:00
Ruslan Kuprieiev 8839e72f07 dvc: stop using checksumdir and use our own way of checksuming a dir 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-10 01:56:06 +03:00
Ruslan Kuprieiev bd6688ff57 cloud: add ssh driver 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-08 22:26:18 +03:00
Ruslan Kuprieiev 13e0e9d2e4 dvc: use getdirinfo from ntfsutils 
We can now use getdirinfo directly, since our patch
https://github.com/sid0/ntfs/pull/10 is now merged
and released in ntfsutils 0.1.4.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-22 13:27:08 +02:00
Ruslan Kuprieiev 6b285e611c dvc: validate stage files 
Fixes #578

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-20 10:40:37 +02:00
Ruslan Kuprieiev e67be7afca dvc: add pyasn1 to requirements 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 13:11:44 +02:00
Ruslan Kuprieiev fa22e56917 state: use str representations for inode and mtime 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 12:35:56 +02:00
Ruslan Kuprieiev b7e9c9f021 pip: add setuptools version dependency 
Fixes #555

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 12:11:22 +02:00
Ruslan Kuprieiev b728d942c7 state: optimize state storage and computation 
Reduces 'dvc add' time for 30K files from 25sec to 9sec.

Fixes #546

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-13 20:18:03 +02:00
Ruslan Kuprieiev 96bb4640a3 Remove pygraphviz from requirements.txt 2018-03-12 18:58:15 +02:00
Ruslan Kuprieiev 068f742852 dvc: use zc.lockfile instead of fasteners 
fasteners' InterProcessLock doesn't work within
one process, thus it is not suitable for our dvc
library.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-02-27 04:20:32 +02:00
Ruslan Kuprieiev 4e2d6d1cd1 dvc: cleanup requirements 
There are a lot of old unused dependecies that need to be cleaned up.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-02-24 02:29:03 +02:00
Ruslan Kuprieiev 385317eb4c dvc: relax version requirements 
Fixes #480

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-02-24 02:29:03 +02:00
Ruslan Kuprieiev e4d9755a7e dvc: cloud RFC add support for directories 
Fixes #367

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-01-22 03:19:29 +02:00