Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-04 15:28:27 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,850 Commits 7 Branches 579 Tags
0.18.13
Commit Graph
65 Commits
Author SHA1 Message Date
Ruslan Kuprieiev cb7eed71de setup: update grandalf reqs 2018-08-23 04:42:27 +03:00
Ruslan Kuprieiev c378e4b3d2 dvc: pipeline show: create ASCII visualizations for DAG 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-19 21:01:12 +03:00
Ruslan Kuprieiev 4ff4a1042e setup: use extras_require for backward compatibility 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-07 05:13:35 +03:00
Ruslan Kuprieiev c345552460 setup: use proper env marker for futures dependency 
Fixes #968

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-07 04:35:13 +03:00
Ruslan Kuprieiev 24be60bf39 setup.py: fix typo 2018-08-06 00:31:27 +03:00
Ruslan Kuprieiev fe38b2768f setup: don't forget to import sys 2018-08-06 00:11:05 +03:00
Ruslan Kuprieiev 105d8fc9b5 setup: fix install requirement 2018-08-06 00:02:35 +03:00
Ruslan Kuprieiev f02bf73587 reqs: only requre futures pkg on python 2.7 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-03 20:55:23 +03:00
Ruslan Kuprieiev d0ce57adac reqs: move pytest to test-requirements 
Fixes #966

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-03 13:33:24 +03:00
Ruslan Kuprieiev eaa994527d reqs: don't forget futures 
Fixes #967

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-08-03 13:25:36 +03:00
Ruslan Kuprieiev f1a28c3812 dvc: support dvc add for external files 
Fixes #890

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-16 16:34:16 +03:00
Clemens Wolff 6882a781a4 Add Azure Blob Storage remote 
This change adds a new cloud remote based on Azure Blob Storage [1] and
the azure-storage-blob SDK [2].

The implementation closely follows the patterns laid out by the existing
cloud remotes (AWS and GCP). Notable differences include:

- No use of a lock in the progress-bar callback and no requirement to
  look up total transfer size ahead of time: the Azure Storage SDK
  provides the total transfer bytes and current transfer offset in its
  progress callback.

- When `exists` is called, check the existence of every blob instead of
  listing the entire container. Container list operations can be
  expensive so unless the number of checked files is very large, making
  separate requests is likely going to be more efficient.

- For increased flexibility, access credentials for the Azure Storage
  backend can be provided either connection-string style via the remote
  URL or via environment variables. The Azure Storage backend can be
  emulated via Azurite [3].

[1] https://azure.microsoft.com/en-us/services/storage/blobs/
[2] https://github.com/Azure/azure-storage-python
[3] https://github.com/Azure/azurite

This change was created together with @EricSchles.
 2018-07-09 14:47:40 -04:00
Ruslan Kuprieiev 5d3a009886 metrics: require ply>=3.9 to avoid bug on binary windows package 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-08 01:02:26 +03:00
Ruslan Kuprieiev fba1d8c181 dvc: use heuristics to detect binary files 
Fixes #859

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-06 01:32:14 +03:00
Victor Villas ebee3ceae1 Add dependency checking as config validation 2018-06-26 21:27:49 +00:00
Victor Villas 0831414d1c Move remote dependencies to extras_require section 2018-06-26 18:56:17 +00:00
Ruslan Kuprieiev 456ba271b3 dvc: rename dataversioncontrol/dvc into iterative/dvc 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-30 04:14:16 +03:00
Ruslan Kuprieiev 4ab9b68fbd setup: lock google-cloud==0.32.0 2018-05-22 23:55:28 +03:00
Ruslan Kuprieiev 7709f57f0b reqs: add pytest(required by setuptools on some systems) 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 18:30:15 +03:00
Ruslan Kuprieiev 0f8f682a6b build: remove uneeded deps of boto 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 16:40:27 +03:00
Ruslan Kuprieiev 91882bf8c4 reqs: freeze google-compute-engine at 2.4.1 2018-05-17 15:15:38 +03:00
Ruslan Kuprieiev 03ebdb5cf2 dvc: use our own api instead of github's for version retrieval 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-10 14:40:59 +03:00
Ruslan Kuprieiev 6e20e1f845 dvc: check updates and warn about using an older version 
Fixes #687

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-06 22:12:28 +03:00
Ruslan Kuprieiev 0a049c232b cache: configurable cache dir and cache type 
From now on we allow specifying standalone cache dirs as well as
specifying linking method(copy, symlink, hardlink, reflink) for it.
If 'cache type' is not specified in the config, dvc will try all of
them starting from reflink all the way down to copy.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-28 21:50:49 +03:00
Ruslan Kuprieiev 832af146c6 requirements: add hidden requirement 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-19 21:03:07 +03:00
Ruslan Kuprieiev 1e7d548ca4 dvc: add dvc metrics 
Fixes #592

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-19 20:47:37 +03:00
Ruslan Kuprieiev 79d7100104 setup.py: don't mixup download_url and url 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-14 23:26:40 +03:00
Ruslan Kuprieiev f30e6e53c5 utils: convert to unix format when calculating md5 
Fixes #619

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-14 22:07:29 +03:00
Ruslan Kuprieiev 3748164ddb cloud: aws: migrate to boto3 
This allows us to abandon our own additional config parameters for
aws cloud(like region) and use the ones from .aws/config by default.
Plus, boto3 automatically switches to multipartupload if file is bigger
than a certain limit, allowing us to remove a lot of our own code.

Fixes #650
Fixes #636
Fixes #601

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-13 06:06:52 +03:00
Ruslan Kuprieiev 8839e72f07 dvc: stop using checksumdir and use our own way of checksuming a dir 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-10 01:56:06 +03:00
Ruslan Kuprieiev bd6688ff57 cloud: add ssh driver 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-08 22:26:18 +03:00
Ruslan Kuprieiev 7e93120754 setup: use long_description with README inside 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-22 21:23:30 +02:00
Ruslan Kuprieiev 13e0e9d2e4 dvc: use getdirinfo from ntfsutils 
We can now use getdirinfo directly, since our patch
https://github.com/sid0/ntfs/pull/10 is now merged
and released in ntfsutils 0.1.4.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-22 13:27:08 +02:00
Ruslan Kuprieiev 20c1bfe5ff setup: pypi fixes 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-21 12:17:26 +02:00
Ruslan Kuprieiev 46bb7e1c63 setup.py: use m2r to convert README.md to rst 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-21 12:01:16 +02:00
Ruslan Kuprieiev 93194ecd2d dvc: setup.py: add long description 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-21 11:14:19 +02:00
Dmitry c177e1c52d Update description in the package 2018-03-21 01:53:56 -07:00
Ruslan Kuprieiev 6b285e611c dvc: validate stage files 
Fixes #578

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-20 10:40:37 +02:00
Ruslan Kuprieiev e67be7afca dvc: add pyasn1 to requirements 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 13:11:44 +02:00
Ruslan Kuprieiev fa22e56917 state: use str representations for inode and mtime 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 12:35:56 +02:00
Ruslan Kuprieiev b7e9c9f021 pip: add setuptools version dependency 
Fixes #555

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 12:11:22 +02:00
Ruslan Kuprieiev b728d942c7 state: optimize state storage and computation 
Reduces 'dvc add' time for 30K files from 25sec to 9sec.

Fixes #546

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-13 20:18:03 +02:00
Ruslan Kuprieiev 04d11c1762 Remove pygraphviz from install_requires 2018-03-12 18:58:46 +02:00
Ruslan Kuprieiev 068f742852 dvc: use zc.lockfile instead of fasteners 
fasteners' InterProcessLock doesn't work within
one process, thus it is not suitable for our dvc
library.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-02-27 04:20:32 +02:00
Ruslan Kuprieiev 4e2d6d1cd1 dvc: cleanup requirements 
There are a lot of old unused dependecies that need to be cleaned up.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-02-24 02:29:03 +02:00
Ruslan Kuprieiev 385317eb4c dvc: relax version requirements 
Fixes #480

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-02-24 02:29:03 +02:00
Ruslan Kuprieiev e4d9755a7e dvc: cloud RFC add support for directories 
Fixes #367

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-01-22 03:19:29 +02:00
Ruslan Kuprieiev 19cd599b1a macos: remove old dependencies 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-01-20 04:35:36 +02:00
Ruslan Kuprieiev 3b09290ed8 test: improve coverage and fix bugs 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-01-10 19:39:41 +02:00
Ruslan Kuprieiev 9c10a643a7 win: enable unit tests 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-01-02 13:03:06 +02:00