Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,297 Commits 7 Branches 579 Tags
9389422f0b93361c05058c786004b295b32c4245
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev 9389422f0b dvc: add --cwd for dvc repro 
Fixes #401

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-06 01:42:06 +03:00
Ruslan Kuprieiev 5d169916de Merge pull request #740 from efiop/master 
Add initial support for s3/gs outputs/deps/cache_dirs
 2018-06-04 22:38:24 +03:00
Ruslan Kuprieiev dd610c04fb test: add test for s3/gs out/dep/cache 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:15 +03:00
Ruslan Kuprieiev 585c27f354 cloud: use project.cache.local instead of project.cache 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:15 +03:00
Ruslan Kuprieiev 346ef139dd out: properly remove outputs when reproducing 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:15 +03:00
Ruslan Kuprieiev 331e3b2bf9 cache: fix cache_dir resolving 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:15 +03:00
Ruslan Kuprieiev 7a7c9e7713 dvc: fix import conflicts 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 1ef21039c4 remote: use Remote() func to determine remote type 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 42265339a8 remote: rename Remote -> RemoteBase 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev fc53971f2b dvc: temporarily only count local cache as used cache 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 484b04cb68 outs: raise exception if no cache location setup for s3/gs 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 88397ef430 dvc: use Remote concept in existing code 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 6f4d0cd74a cache: use atomic move to avoid using Lock() 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev f03ea7a915 dvc: move link_state from Cache to Project 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 08dcd8490e outs/deps: remove loadd 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 5c1748c15c outs/deps: remove loads method 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev fbfdecd1de outs/deps: remove type-specific SCHEMAs 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev f3b2497fb9 deps/outs: remove load{s,d}_from methods 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev e0a42f1265 outs: support gs/s3 as an output without cache 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 2e78bb2a69 deps: s3/gs: check if key exists 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 456ba271b3 dvc: rename dataversioncontrol/dvc into iterative/dvc 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-30 04:14:16 +03:00
Ruslan Kuprieiev 95870ce769 README: fix typo in format 2018-05-24 20:08:22 +03:00
Ruslan Kuprieiev 92cbda749f README: use remotes instead of an obsoleted model 2018-05-24 20:07:57 +03:00
Ruslan Kuprieiev 33b26278fe Merge pull request #731 from efiop/master 
test: temporarily disable gs dependency test on CI
 2018-05-24 19:05:03 +03:00
Ruslan Kuprieiev ac3d37e58c test: temporarily disable gs dependency test on CI 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-24 18:58:20 +03:00
Ruslan Kuprieiev a35b3aa67a travis: specify BOTO_CONFIG=/dev/null 
See https://github.com/travis-ci/travis-ci/issues/7940
 2018-05-24 18:50:52 +03:00
Ruslan Kuprieiev 7a3aa54293 Merge pull request #730 from efiop/master 
gs/s3 as dependency
 2018-05-24 18:31:09 +03:00
Ruslan Kuprieiev b85c41c734 updater: use response.json() 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-24 18:24:18 +03:00
Ruslan Kuprieiev 2bbefc91ef deps: introduce s3/gs dependencies 
E.g.:
dvc run -d gs://bucket/file -o file gsutil cp gs://bucket/file file
dvc run -d s3://bucket/file -o file aws s3 cp s3://bucket/file file

Etags are used to track deps in both cases.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-24 18:24:18 +03:00
Ruslan Kuprieiev 69f9c5d837 Merge pull request #724 from efiop/master 
dvc: split output and dependency classes
 2018-05-23 01:51:09 +03:00
Ruslan Kuprieiev 56e76d0d8b outs/deps: use 'stage' instead of 'project' 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-23 00:51:08 +03:00
Ruslan Kuprieiev 21bf52ee9d dvc: split output and dependency classes 
Preparation for the 'external deps' patches.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-23 00:01:04 +03:00
Ruslan Kuprieiev 4ab9b68fbd setup: lock google-cloud==0.32.0 2018-05-22 23:55:28 +03:00
Ruslan Kuprieiev 82890ec2a4 reqs: lock google-cloud==0.32.0 
0.33.0 is broken.
 2018-05-22 23:54:59 +03:00
Ruslan Kuprieiev 962ee4ab77 Merge pull request #723 from efiop/master 
dvc: add cache dir to gitignore only if it is not external
 2018-05-21 17:08:31 +03:00
Ruslan Kuprieiev c819ac9292 dvc: add cache dir to gitignore only if it is not external 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-21 17:00:37 +03:00
Ruslan Kuprieiev 1cc2d103f4 Merge pull request #721 from efiop/master 
run: properly generate automatic name when output is a directory and …
 2018-05-21 12:58:07 +03:00
Ruslan Kuprieiev d15143bb6f run: properly generate automatic name when output is a directory and no -f specified 
Fixed #720

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-21 12:38:21 +03:00
Ruslan Kuprieiev 91c16e2112 Merge pull request #719 from efiop/master 
Misc build fixes
 2018-05-17 18:32:01 +03:00
Ruslan Kuprieiev 7709f57f0b reqs: add pytest(required by setuptools on some systems) 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 18:30:15 +03:00
Ruslan Kuprieiev 62c43f6c4f aws: remove unneeded botocore dependency 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 18:23:43 +03:00
Ruslan Kuprieiev 03d477499b Merge pull request #718 from efiop/master 
build: remove unneeded deps of boto
 2018-05-17 16:46:32 +03:00
Ruslan Kuprieiev 0f8f682a6b build: remove uneeded deps of boto 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 16:40:27 +03:00
Ruslan Kuprieiev cb57446c52 pyinstaller: specify hidden imports for google_cloud_engine hook 2018-05-17 16:00:15 +03:00
Ruslan Kuprieiev 8b75d2ddb7 Merge pull request #717 from efiop/master 
pyinstaller: add google_compute_engine hook
 2018-05-17 15:48:39 +03:00
Ruslan Kuprieiev b77ff668f6 pyinstaller: add google_compute_engine hook 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 15:38:12 +03:00
Ruslan Kuprieiev 56cb5134b9 Merge pull request #716 from efiop/master 
reqs: freeze google-compute-engine at 2.4.1
 2018-05-17 15:24:36 +03:00
Ruslan Kuprieiev 91882bf8c4 reqs: freeze google-compute-engine at 2.4.1 2018-05-17 15:15:38 +03:00
Ruslan Kuprieiev e49952f8aa scripts: posix: fix order 2018-05-17 14:54:15 +03:00
Ruslan Kuprieiev 2cb43e29dc Merge pull request #715 from efiop/master 
Misc build fixes
 2018-05-17 14:24:41 +03:00
Ruslan Kuprieiev adaf3f749f scripts: merge build_linux and build_macos 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 14:18:40 +03:00
Ruslan Kuprieiev d9a2cbb3ce reqs: add wheel to requirements.txt 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 14:04:11 +03:00
Ruslan Kuprieiev b116e885eb Merge pull request #714 from efiop/master 
reqs: require PyInstaller 3.3.1
 2018-05-17 13:15:33 +03:00
Ruslan Kuprieiev 76ed66e1af reqs: require PyInstaller 3.3.1 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 13:09:48 +03:00
Ruslan Kuprieiev be870d0729 travis: osx: fix typo 2018-05-17 03:17:44 +03:00
Ruslan Kuprieiev d0563fc7ed travis: osx: install wheel 2018-05-17 03:15:59 +03:00
Ruslan Kuprieiev ca668bf930 Merge pull request #713 from efiop/master 
travis: osx: add --enable-framework for pyenv
 2018-05-17 02:55:08 +03:00
Ruslan Kuprieiev 7ec1a05195 travis: osx: add --enable-framework for pyenv 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 02:50:07 +03:00
Ruslan Kuprieiev 2a04c17817 dvc: bump to 0.9.7 2018-05-17 01:43:42 +03:00
Ruslan Kuprieiev f21699c479 Merge pull request #712 from efiop/master 
dvc: reuse collected stages
 2018-05-17 00:49:25 +03:00
Ruslan Kuprieiev bde70498cb dvc: reuse collected stages 
Fixes #710
Fixes #711

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-17 00:40:47 +03:00
Ruslan Kuprieiev 679bcae9d4 Merge pull request #708 from efiop/master 
utils: file_md5: don't forget to decode data on python 2
 2018-05-16 14:48:22 +03:00
Ruslan Kuprieiev 2b7d75d389 utils: file_md5: don't forget to decode data on python 2 
Fixes #686

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-16 14:38:15 +03:00
Ruslan Kuprieiev 09e4582af9 Merge pull request #707 from efiop/master 
dvc: print warning when file/dir is empty
 2018-05-15 23:08:58 +03:00
Ruslan Kuprieiev 1bd9e9fbb1 dvc: print warning when file/dir is empty 
Fixes #677

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-15 22:57:35 +03:00
Ruslan Kuprieiev ad54ec2091 Merge pull request #704 from efiop/master 
dvc: try to add to gitignore every time for backward compatibility
 2018-05-14 17:14:44 +03:00
Ruslan Kuprieiev c063ce9ff5 dvc: try to add to gitignore every time for backward compatibility 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-14 17:09:29 +03:00
Ruslan Kuprieiev fca93eae60 Merge pull request #703 from efiop/master 
dvc: store list of links to cleanup later
 2018-05-13 21:20:25 +03:00
Ruslan Kuprieiev c7fec74516 dvc: store list of links to cleanup later 
Fixes #685

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-13 21:02:54 +03:00
Ruslan Kuprieiev 0911810632 Merge pull request #702 from efiop/master 
cloud: use status to determine which targets to push/pull
 2018-05-11 23:10:59 +03:00
Ruslan Kuprieiev 018e9d4807 cloud: use status to determine which targets to push/pull 
Fixes #697

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-11 23:05:56 +03:00
Ruslan Kuprieiev e05b2adce1 Merge pull request #701 from efiop/master 
git: untracked_files: don't forget to join paths
 2018-05-11 18:28:58 +03:00
Ruslan Kuprieiev 0b0e513877 git: untracked_files: don't forget to join paths 
Fixes #700

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-11 18:21:22 +03:00
Ruslan Kuprieiev 8adb9319b8 Merge pull request #698 from efiop/master 
Misc fixes
 2018-05-10 17:21:43 +03:00
Ruslan Kuprieiev 7ad5bed5b8 dvc: use Updater class 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-10 17:06:38 +03:00
Ruslan Kuprieiev 0f45d2aa33 dvc: don't forget to add state.lock to gitignore 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-10 17:06:38 +03:00
Ruslan Kuprieiev 8d13f3210c Merge pull request #695 from efiop/master 
Use our server to check for updates and do so once a week
 2018-05-10 15:30:01 +03:00
Ruslan Kuprieiev 2f97be3782 dvc: check for updates once a week 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-10 15:21:14 +03:00
Ruslan Kuprieiev 03ebdb5cf2 dvc: use our own api instead of github's for version retrieval 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-10 14:40:59 +03:00
Ruslan Kuprieiev c563632f6e Merge pull request #694 from efiop/master 
dvc: check updates and warn about using an older version
 2018-05-06 22:19:11 +03:00
Ruslan Kuprieiev 6e20e1f845 dvc: check updates and warn about using an older version 
Fixes #687

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-06 22:12:28 +03:00
Ruslan Kuprieiev f11e19a92c Merge pull request #692 from efiop/master 
dvc: introduce 'dvc remote list'
 2018-05-05 14:26:25 +03:00
Ruslan Kuprieiev 6156b7e1b1 dvc: introduce 'dvc remote list' 
Fixes #690

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-05-05 14:17:35 +03:00
Ruslan Kuprieiev adf44878d9 Merge pull request #684 from efiop/master 
dvc: reproduce stage if it has no dependencies and has cmd
 2018-04-29 17:49:30 +03:00
Ruslan Kuprieiev 9a88a412d0 dvc: reproduce stage if it has no dependencies and has cmd 
This allows as to define conditional stages in the dependency graph.

Fixes #646
Fixes #648

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-29 17:43:32 +03:00
Ruslan Kuprieiev 77d478cc20 Merge pull request #683 from efiop/master 
travis: remove unused code
 2018-04-29 16:20:26 +03:00
Ruslan Kuprieiev eb9560c459 travis: remove unused code 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-29 16:15:01 +03:00
Ruslan Kuprieiev 131b7b7fec travis: osx pyenv debug 2018-04-29 15:59:02 +03:00
Ruslan Kuprieiev 1e7c2690e3 travis: fix typo 2018-04-29 15:38:35 +03:00
Ruslan Kuprieiev 5ca497e9ae travis: osx debug 2018-04-29 15:35:09 +03:00
Ruslan Kuprieiev 3fffdc9a32 travis: call pyenv for osx 2018-04-29 15:22:09 +03:00
Ruslan Kuprieiev fbfc68963f travis: update pip cmd 2018-04-29 14:42:38 +03:00
Ruslan Kuprieiev cb18902e93 travis: use xcode9.3 image for osx 2018-04-29 14:29:45 +03:00
Ruslan Kuprieiev 68bed24d68 Merge pull request #681 from efiop/master 
Configurable cache dir + copy/symlink/hardlink/reflink support
 2018-04-28 22:17:42 +03:00
Ruslan Kuprieiev b668f23167 Revert "build: win: don't look for symlink ps script" 
This reverts commit 4164eddb92.
 2018-04-28 21:54:58 +03:00
Ruslan Kuprieiev ba6a57264a Revert "win: innosetup: don't add symlink perms" 
This reverts commit e94b78d677.
 2018-04-28 21:54:36 +03:00
Ruslan Kuprieiev 21fc712edf cache: remove unused code 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-28 21:50:49 +03:00
Ruslan Kuprieiev 0a049c232b cache: configurable cache dir and cache type 
From now on we allow specifying standalone cache dirs as well as
specifying linking method(copy, symlink, hardlink, reflink) for it.
If 'cache type' is not specified in the config, dvc will try all of
them starting from reflink all the way down to copy.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-28 21:50:49 +03:00
Ruslan Kuprieiev d8550a63cc Merge pull request #680 from efiop/master 
dvc: checkout single file
 2018-04-24 13:56:34 +03:00
Ruslan Kuprieiev 52c08b2926 dvc: checkout single file 
Fixes #666

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-04-24 13:45:30 +03:00