Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-11 15:28:27 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,475 Commits 7 Branches 579 Tags
66caa345b1cc1a49cf5fcd1befe72cd052dbe834
Commit Graph
6475 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Dmitry Petrov 2465f5589f Increment version 2017-07-01 16:43:09 -07:00
Dmitry Petrov cfa9731b3e Remove uploading info messages 2017-06-28 01:10:17 -07:00
Dmitry Petrov 3eda97d8ff download_url format 2017-06-27 02:53:49 -07:00
Dmitry Petrov a588d71a83 Small GC changes 2017-06-27 02:53:00 -07:00
Dmitry Petrov 059fcc028d Zero size file bugfix 2017-06-25 00:33:03 -04:00
Ruslan Kuprieiev a96b61bd4e Merge pull request #99 from efiop/master 
Progress bar improvements, --jobs and progress bar for LOCAL cloud and AWS
 2017-06-25 04:51:03 +03:00
Ruslan Kuprieiev 76db9db5c2 dvc: introduce --jobs option and use map_progress 
It replaces POOL_SIZE constant and by default equals to the
number of cpus available.

map_progress is a helper for 'sync' and 'import' commands to
run in a multi-threaded mode.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-25 04:04:19 +03:00
Ruslan Kuprieiev 7c6765c196 import/sync: add progress bar for local copy 
Currently progress bar is only supported for downloading
from web(aws,http* etc), but now it is also supported for
local files(i.e. LOCAL for 'data sync' and just local files
for 'import')

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-25 02:03:46 +03:00
Ruslan Kuprieiev bdb30cc602 sync: add progress bar for AWS 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-21 03:51:06 +03:00
Ruslan Kuprieiev 11d1fb502a progress: use global variable 'progress' 
There is no point in having more than just one
global progress bar.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-21 03:37:42 +03:00
Dmitry Petrov cd43160414 Increase version to 0.8.5 0.8.5 2017-06-19 23:42:39 -07:00
Dmitry Petrov fc576f3d77 Merge pull request #98 from dataversioncontrol/sync_subfolder_fix 
Sync subfolder fix
 2017-06-19 22:07:31 -07:00
Dmitry Petrov 3f962deb25 Fix: NamedTemporaryFile(mode=w) 2017-06-19 21:59:26 -07:00
Dmitry Petrov ed923db203 Import cmd unit-tests 2017-06-19 21:31:31 -07:00
Dmitry Petrov 674f20c800 Verify output refactoring. Small fixes. 2017-06-19 19:30:01 -07:00
Dmitry Petrov 47e1b421f5 Verify output refactoring. Attempt 2 2017-06-19 19:20:04 -07:00
Dmitry Petrov c16f97a3cf Verify output refactoring 2017-06-19 19:07:54 -07:00
Dmitry Petrov 15cadc44f6 Simplify output validation logic 2017-06-19 18:48:05 -07:00
Dmitry Petrov 71566b1a24 Rename CmdDataRemove class 2017-06-19 18:32:30 -07:00
Dmitry Petrov 05dfaa3d48 Rename file test_cmd_data_remove 2017-06-19 18:29:44 -07:00
Dmitry Petrov 0e3278641c Small tests headers fixes 2017-06-19 18:21:58 -07:00
Dmitry Petrov 17ffcea10f Error when syncing data subfolders. Close #94 2017-06-19 18:13:55 -07:00
Dmitry Petrov df6abc7528 Small sync and import fixes 2017-06-19 16:44:18 -07:00
Ruslan Kuprieiev 57691297a8 Merge pull request #95 from efiop/master 
progress: add progress bar to 'import'
 2017-06-19 04:52:05 +03:00
Ruslan Kuprieiev e9a9e32357 progress: add progress bar to 'import' 
This one is a bit like the one you see in yum/dnf.
It has one single line updating current progress
for different targets and once they are finished
prints them as standalone line. So it looks smth
like this:

  $ dvc import https://foo.bar/one.data \
               https://foo.bar/two.data \
               https://foo.bar/three.data
  (1/3): [########################################] 100% one.data
  (2/3): [#########################               ] 60% three.data

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-19 04:30:32 +03:00
Dmitry Petrov 9062da3f79 Tests for GC command 2017-06-17 17:08:17 -07:00
Dmitry Petrov f2bac9aac1 GC command 2017-06-14 23:48:58 -07:00
Ruslan Kuprieiev 5e4c11e8b5 Merge pull request #93 from efiop/master 
innosetup: add logos for installer
 2017-06-12 03:49:07 +03:00
Ruslan Kuprieiev c4bc971d26 innosetup: add logos for installer 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-12 03:27:08 +03:00
Dmitry Petrov edff97d4a9 Codeclimat badges 2017-06-11 16:41:17 -07:00
Ruslan Kuprieiev cf8cd9e5b0 Update travis.yml with correct code-climate token 2017-06-12 01:28:48 +03:00
Ruslan Kuprieiev 619d9691b9 Merge pull request #92 from efiop/master 
Misc changes for build and testing
 2017-06-12 00:28:36 +03:00
Ruslan Kuprieiev 4078e902d1 test_config: python3 fix 
1) use io for StringIO in python3
2) use builtins instead of __builtin__
3) use StringIO instead of BytesIO when mocking aws config
Othervise this test fails on python3.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-12 00:22:19 +03:00
Ruslan Kuprieiev 52d4c001a9 travis: add dummy aws credentials to make unitests work 
Our tests assume that user has his default credentials at
~/.aws/credentials, but this is not the case with travis.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 23:17:36 +03:00
Ruslan Kuprieiev 8253f449ca data_cloud: check if cloud section exists 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 22:49:15 +03:00
Ruslan Kuprieiev b502691b0e unittests: exit if nosetests exits with error code 
For now unitetests.sh is useless, as it returns 0 anyway.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 22:42:29 +03:00
Ruslan Kuprieiev 016229d193 travis: add dummy git user and mail 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 22:28:43 +03:00
Ruslan Kuprieiev 56b07a4ee3 debug: add assert Settings to DataCloud 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 22:26:23 +03:00
Ruslan Kuprieiev 0d3ef3c22f test: fix test_config 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 22:25:43 +03:00
Ruslan Kuprieiev 577878c887 travis: use test-requirements.txt 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 21:32:47 +03:00
Ruslan Kuprieiev 31b25f7015 test: add coverage to test requirements 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 21:32:11 +03:00
Ruslan Kuprieiev 88692eb37c travis: add code-climate hook 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-11 21:20:05 +03:00
Ruslan Kuprieiev fb68982d07 build_linux: add "rpm" package for debian-based distros 
Without it, dvc.rpm is not generated when ./build_linux.sh is
ran on debian-based distro.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-07 23:10:09 +03:00
Ruslan Kuprieiev 08d897a7c5 Merge pull request #90 from efiop/master 
dvc: introduce multithreaded import and RFC
 2017-06-07 14:35:04 +03:00
Ruslan Kuprieiev 4f938cec71 dvc: introduce multithreaded import and RFC, v2 
This allows us to download files in a few threads.
Also includes RFC, as we can now get rid of import-bulk.py.

v2: use POOL_SIZE=4

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-07 14:16:15 +03:00
Ruslan Kuprieiev 5b810fed99 build: create standalone binary with WinPython 
Recent changes for building standalone dvc binary on Linux
made it possible to do the same thing on Windows. So now
we are getting rid of WinPython and instead building
standalone dvc binary using PyInstall and packaing it
into an installer with InnoSetup.

One of the benefits is dramatically decreased installer size.
From 125M we got to only 13M!

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-02 12:53:25 -07:00
Ruslan Kuprieiev e70aa8eec1 Merge pull request #89 from efiop/master 
build_linux: hotfix for build
 2017-06-01 22:35:39 +03:00
Ruslan Kuprieiev 89311c1af5 build_linux: hotfix for build 
Wrong INSTALL_DIR resulted in dvc being installed into /build/usr/bin.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-01 22:31:33 +03:00
Ruslan Kuprieiev 3499bc32d6 Merge pull request #88 from efiop/master 
Build rpm and deb packages using PyInstaller and FPM
 2017-06-01 22:05:51 +03:00
Ruslan Kuprieiev cb263cad3f build: add build_linux.sh to build rpm and deb 
It uses pyintsaller to create standalone dvc binary and
then uses fpm to pack it into deb and rpm.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2017-06-01 21:57:34 +03:00