Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-08 07:18:27 +01:00
Code Issues Packages Projects Releases Wiki Activity
402 Commits 7 Branches 579 Tags
126505e25bd2ed54c256edf04eac751c3e4e6818
Commit Graph
402 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev 126505e25b Merge pull request #187 from efiop/master 
travis: enable aws tests
 2017-08-27 00:37:25 +03:00
Ruslan Kuprieiev 68c06acd94 travis: enable aws tests 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-27 00:21:39 +03:00
Ruslan Kuprieiev 4625368acb Merge pull request #183 from efiop/master 
misc patches
 2017-08-26 11:34:38 +03:00
Ruslan Kuprieiev f0ba4c1ff9 test: add simple push/pull/status test for aws 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-26 11:20:36 +03:00
Ruslan Kuprieiev 9d9a91d55d data: remove redundant code 
Aws credentials are now managed by dvc/cloud/creds_aws.py

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-26 11:20:36 +03:00
Ruslan Kuprieiev 98c29c80d3 config: add ability to add and remove options 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-26 11:20:36 +03:00
Ruslan Kuprieiev b707dbd0fe dvc: add 'dvc visual' 
This command generates dependency graph for specified target,
saves it in .dot file and also draws it into .svg file.

Fixes #39

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-26 09:18:45 +03:00
Ruslan Kuprieiev 30bacf7f95 dvc: rename empty to .empty 
This will allow us to hide it.

Fixes #104

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-26 09:18:45 +03:00
Dmitry Petrov 5fdfe5060d Close #184 2017-08-25 02:38:53 -07:00
Ruslan Kuprieiev 73022d74d6 cloud: fix import 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-23 22:14:31 +03:00
Ruslan Kuprieiev fcaccfabcd data_cloud: fix cloud config for testing 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-23 22:05:46 +03:00
Dmitry Petrov 6c74a85150 Fix last merge issues 2017-08-23 01:46:36 -07:00
Dmitry Petrov 4befe3f7c6 Manual merge 2017-08-23 00:48:27 -07:00
Dmitry Petrov 4c5e314d33 Merge branch 'basic_cloud2' 2017-08-23 00:42:16 -07:00
Dmitry Petrov ee0e012f86 Config redesign 2017-08-23 00:38:28 -07:00
Ruslan Kuprieiev f0b75172e7 Merge pull request #182 from efiop/master 
aws: fix progress and chunk size for multipart upload
 2017-08-21 08:51:14 +03:00
Ruslan Kuprieiev 6430982ef0 aws: fix progress and chunk size for multipart upload 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-21 08:30:06 +03:00
Ruslan Kuprieiev b167911822 Merge pull request #181 from efiop/master 
aws: use multipart to push changes, v2
 2017-08-20 19:23:25 +03:00
Ruslan Kuprieiev 04e66f94fc aws: use multipart to push changes, v2 
Fixes #100

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-20 19:04:58 +03:00
Ruslan Kuprieiev a86a831467 Merge pull request #180 from efiop/master 
cloud: fix typo
 2017-08-20 18:37:23 +03:00
Ruslan Kuprieiev 90762da18f cloud: fix typo 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-20 17:58:34 +03:00
Ruslan Kuprieiev 8b84d098cb Merge pull request #179 from dataversioncontrol/revert-178-master 
Revert "aws: use multipart to push changes"
 2017-08-20 17:56:17 +03:00
Ruslan Kuprieiev 163558c9c2 Revert "aws: use multipart to push changes" 2017-08-20 17:55:23 +03:00
Ruslan Kuprieiev b5866962a8 Merge pull request #178 from efiop/master 
aws: use multipart to push changes
 2017-08-20 16:36:15 +03:00
Ruslan Kuprieiev 78cad172c6 aws: use multipart to push changes 
Fixes #100.
 2017-08-20 16:12:38 +03:00
Ruslan Kuprieiev 78b337034d Merge pull request #176 from efiop/master 
[WIP] Misc improvements and features
 2017-08-20 10:11:22 +03:00
Ruslan Kuprieiev dc77e03459 travis: run functests 
Fixes #175

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-20 09:32:32 +03:00
Ruslan Kuprieiev 0f45afb3a4 functest: rework test system and introduce new tests 
Reworked existing tests and test-system, added a few new
tests(init, reworked repro, import).

Fixes #175

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-20 09:32:32 +03:00
Ruslan Kuprieiev 194e1ec419 dvc: add 'config' command 
Fixes #177

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-20 06:19:38 +03:00
Ruslan Kuprieiev e7bf8054da functest: fix existing scripts 
A few minor adjustments to make what we currently have work,
while I prepare new func tests.

Fixes #175

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-20 04:53:13 +03:00
Ruslan Kuprieiev eb533f9a9c RFC: fix issues reported by codeclimate 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-20 04:53:13 +03:00
Ruslan Kuprieiev 4fe831712e import: don't create state files for failed imports 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-19 20:10:54 +03:00
Ruslan Kuprieiev 3acc2146fa data_item: handle __eq__ if other == None 
Currently it is not possible to check if data_item == None,
as __eq__ tries to access .data in 'None' object and thus
throws an exception.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-19 20:10:54 +03:00
Ruslan Kuprieiev c3a43c6d84 cloud: return data_item in sync/pull/push/import 
This will help us distinguish if target was processed successfully.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-19 20:10:52 +03:00
Ruslan Kuprieiev 149c921576 dvc: add status command 
This command shows status for data files. I.e.

    $ dvc status data/
        new file:       .cache/Posts-train.tsv_2ccd8e7
        new file:       data/llss
        modified:       .cache/Posts.xml_512a612
        new file:       .cache/empty_0000000
        new file:       data/oss
        new file:       .cache/matrix-test.p_69b924d
        new file:       .cache/dir21111/Posts.xml.tgz_cd71d01

so it will tell us if there are any new/modified/deleted data
files that you might want to push/pull/etc.

Fixes #158

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-19 20:10:24 +03:00
Ruslan Kuprieiev ec76c1aa8a utils: collect return value from map 
Zip it together with the targets list and return back
to the caller so he can use it if he wants.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-19 20:10:24 +03:00
Ruslan Kuprieiev 5f7f608529 import: import from clouds 
From now on, 'dvc import' supports s3 and gcp links.
This patch also does some RFC to properly parse url's
scheme using urllib instead of relying on hand-made
regex. Code for downloding files from http urls is moved
to data_cloud.py in preparation for pull/push support
for http/https/ftp schemes in the future.
Also added dummy progress updates(start/finish) for gcp,
as i couldn't find an API call that allows callbacks to
update progress properly.

Fixes #49

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-19 20:10:10 +03:00
Ruslan Kuprieiev 7f78331003 import: use content-md5 to verify download 
Fixes #167

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-19 20:02:43 +03:00
Dmitry Petrov 761546b858 Move cloud cmd to experimental 2017-08-17 21:57:21 -07:00
Ruslan Kuprieiev 552b571bb2 Merge pull request #174 from efiop/master 
dvc: bring back frozen versions for deps
 2017-08-16 22:43:00 +03:00
Ruslan Kuprieiev 7a3b500390 dvc: bring back frozen versions for deps 
Fixes #173.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-16 19:39:05 +03:00
Ophir Yoktan ed797b0998 Merge pull request #172 from ophiry/testing_none_value 
replace eq\neq to is is not none - bug fix
 2017-08-15 09:05:53 +03:00
Ophir Yoktan b73798270c replace eq\neq to is is not none - bug fix 2017-08-15 09:04:38 +03:00
Ophir Yoktan f3001dc2e1 Merge pull request #171 from ophiry/testing_none_value 
replace eq\neq to is is not none
 2017-08-15 09:03:04 +03:00
Ophir Yoktan 81b39b2ad7 replace eq\neq to is is not none 2017-08-15 09:00:24 +03:00
Ruslan Kuprieiev 49d286e5ad Merge pull request #170 from ophiry/md5_handle_missing_file 
return empty md5 if file doesnt exist
 2017-08-15 01:08:28 +03:00
Ophir Yoktan c21261b69f return empty md5 if file doesnt exist 2017-08-14 23:27:05 +03:00
Ruslan Kuprieiev 4fa18884d7 Merge pull request #165 from efiop/basic_cloud2 
cli: replace "dvc instance-create" with "dvc cloud create"
 2017-08-13 16:35:16 -07:00
Ruslan Kuprieiev 55fc9b7e0d cli: replace "dvc instance-create" with "dvc cloud create" 
This looks and sounds much better.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-08-14 02:23:23 +03:00
Ruslan Kuprieiev 348f04462f Merge pull request #164 from efiop/master 
import: resume download
 2017-08-13 15:33:13 -07:00