Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,087 Commits 7 Branches 579 Tags
a7ca3fcc658b92b587db4b67cc4f1a4f27174bf5
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev a7ca3fcc65 Merge pull request #585 from efiop/master 
pypi fixes and 0.9.1
 2018-03-21 11:36:16 +02:00
Ruslan Kuprieiev a803917ac9 dvc: bump to 0.9.1 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-21 11:14:19 +02:00
Ruslan Kuprieiev 6bbb33e892 scripts: add pypi uploader 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-21 11:14:19 +02:00
Ruslan Kuprieiev 93194ecd2d dvc: setup.py: add long description 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-21 11:14:19 +02:00
Ruslan Kuprieiev 7bc861876a readme: 'dvc repro' -> 'dvc repro model.p' 2018-03-21 10:34:11 +02:00
Ruslan Kuprieiev 65e4ef0a57 readme: installation: add a note about homebrew 2018-03-21 10:16:38 +02:00
Ruslan Kuprieiev 3488404074 Merge pull request #582 from efiop/master 
dvc: validate stage files
 2018-03-20 10:46:14 +02:00
Ruslan Kuprieiev 7d5f6164d0 test: add test for stage file format 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-20 10:40:38 +02:00
Ruslan Kuprieiev 6b285e611c dvc: validate stage files 
Fixes #578

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-20 10:40:37 +02:00
Ruslan Kuprieiev 5270818b5f Merge pull request #581 from efiop/master 
stage: set default values for fields from stage files
 2018-03-20 08:08:28 +02:00
Ruslan Kuprieiev cf74f9b5e5 stage: set default values for fields from stage files 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-20 08:03:46 +02:00
Ruslan Kuprieiev 91c0a82c73 Merge pull request #580 from efiop/master 
checkout: don't print 'cache not found' if cache is None
 2018-03-20 07:57:28 +02:00
Ruslan Kuprieiev 30ba97fce4 checkout: don't print 'cache not found' if cache is None 
Fixes #579

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-20 07:48:20 +02:00
Ruslan Kuprieiev a46e4ea647 Merge pull request #576 from efiop/master 
dvc: fix checkout/repro with stage files without md5
 2018-03-20 00:35:28 +02:00
Ruslan Kuprieiev 958eab2f23 dvc: fix checkout/repro with stage files without md5 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-20 00:31:30 +02:00
Ruslan Kuprieiev 69b5ec71c4 Merge pull request #575 from efiop/master 
build: win: don't look for symlink ps script
 2018-03-18 18:03:02 +02:00
Ruslan Kuprieiev 4164eddb92 build: win: don't look for symlink ps script 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 17:53:53 +02:00
Ruslan Kuprieiev de4437fb90 Merge pull request #574 from efiop/master 
scm: dynamically import gitpython
 2018-03-18 17:24:34 +02:00
Ruslan Kuprieiev 8c3bcc20c8 scm: dynamically import gitpython 
Fixes #573

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 17:17:10 +02:00
Ruslan Kuprieiev 39d42a7a54 Merge pull request #572 from efiop/master 
win: innosetup: don't add symlink perms
 2018-03-18 17:07:12 +02:00
Ruslan Kuprieiev e94b78d677 win: innosetup: don't add symlink perms 
We no longer use symlinks, so there is no reason to setup permissions for it.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 16:58:33 +02:00
Ruslan Kuprieiev 9d402a7a77 Merge pull request #571 from efiop/master 
dvc: add pyasn1 to requirements
 2018-03-18 13:19:32 +02:00
Ruslan Kuprieiev e67be7afca dvc: add pyasn1 to requirements 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 13:11:44 +02:00
Ruslan Kuprieiev 684c4ff3d3 Merge pull request #570 from efiop/master 
test: add test for state reload consistency
 2018-03-18 12:53:09 +02:00
Ruslan Kuprieiev e9acd5a321 test: add test for state reload consistency 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 12:45:50 +02:00
Ruslan Kuprieiev f1e94b84fe Merge pull request #569 from efiop/master 
state: use str representations for inode and mtime
 2018-03-18 12:40:20 +02:00
Ruslan Kuprieiev fa22e56917 state: use str representations for inode and mtime 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 12:35:56 +02:00
Ruslan Kuprieiev d413f53008 Merge pull request #568 from efiop/master 
dvc: state: don't forget to load db
 2018-03-18 11:45:48 +02:00
Ruslan Kuprieiev 27a5b1e356 dvc: state: don't forget to load db 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-18 11:39:02 +02:00
Ruslan Kuprieiev 4f2ff5f79e Merge pull request #567 from efiop/master 
pyinstaller: fix google-api-core hook
 2018-03-18 00:00:01 +02:00
Ruslan Kuprieiev bf251d6dfd pyinstaller: fix google-api-core hook 
Fixes #555

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 23:55:42 +02:00
Ruslan Kuprieiev ebdb62e9aa Merge pull request #565 from efiop/master 
dvc: output: use shutil.rmtree
 2018-03-17 18:55:39 +02:00
Ruslan Kuprieiev 9dfdc227c6 dvc: output: use shutil.rmtree 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 18:51:20 +02:00
Ruslan Kuprieiev 1e9192b733 Merge pull request #564 from efiop/master 
output: remove: remove unused code
 2018-03-17 15:18:50 +02:00
Ruslan Kuprieiev 5c711b72e4 output: remove: remove unused code 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 14:50:40 +02:00
Ruslan Kuprieiev 6a08d2bc8e Merge pull request #563 from efiop/master 
Misc fixes
 2018-03-17 14:24:47 +02:00
Ruslan Kuprieiev 8bdeca0b6f dvc: stage: print stage status only once 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 13:27:24 +02:00
Ruslan Kuprieiev 4a49ee9639 dvc: don't check if stage is changed twice 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 13:23:58 +02:00
Ruslan Kuprieiev aee7dfb994 dvc: print relpath instead of dvc_path 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 13:13:45 +02:00
Ruslan Kuprieiev fc034db9d1 Merge pull request #562 from efiop/master 
repro: fix --force option
 2018-03-17 11:46:29 +02:00
Ruslan Kuprieiev 286002f9cb test: add '--force' test for repro 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 11:42:05 +02:00
Ruslan Kuprieiev 557e734fec repro: fix --force option 
Fixes #561

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 11:42:05 +02:00
Ruslan Kuprieiev 34ec940dc6 appveyor: pip fixes 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-17 11:42:05 +02:00
Ruslan Kuprieiev c5d5cb4043 Merge pull request #560 from efiop/master 
dvc: print warning when corrupted cache is detected
 2018-03-16 18:58:35 +02:00
Ruslan Kuprieiev f1e8904516 dvc: print warning when corrupted cache is detected 
Fixes #549

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 14:29:25 +02:00
Ruslan Kuprieiev 3cbe292ce5 Merge pull request #558 from efiop/master 
aws: rename CredentialSection to Profile
 2018-03-16 14:09:31 +02:00
Ruslan Kuprieiev 61976cb91f aws: rename CredentialSection to Profile 
Fixes #553

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 14:01:16 +02:00
Ruslan Kuprieiev 8c631d4b47 Merge pull request #557 from efiop/master 
'dvc status' and bug fixes
 2018-03-16 12:20:39 +02:00
Ruslan Kuprieiev b7e9c9f021 pip: add setuptools version dependency 
Fixes #555

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 12:11:22 +02:00
Ruslan Kuprieiev b38b12962d dvc: remove fsck 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 11:23:54 +02:00
Ruslan Kuprieiev 80a53f6d77 pyinstaller: add google-api-core hook 
Fixes #555

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 11:17:19 +02:00
Ruslan Kuprieiev 71fef4d857 dvc: add basic 'dvc status' 
(dvc) ➜  myrepo git:(master) ✗ dvc status
foo.dvc
        outs
                changed:  foo
foo2.dvc
        outs
                changed:  foo2
        deps
                changed:  foo1
foo1.dvc
        outs
                changed:  foo1
        deps
                changed:  foo

(dvc) ➜  myrepo git:(master) ✗ dvc status --cloud

        new:      c1/57a79031e1c40f85931829bc5fc552

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 10:58:09 +02:00
Ruslan Kuprieiev 06ff91af62 aws: properly get Region value from config 
Fixes #556

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-16 10:55:18 +02:00
Ruslan Kuprieiev 2447415657 dvc: move CmdDataStatus to a separate file 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-15 10:30:12 +02:00
Ruslan Kuprieiev 7c34cae64b Merge pull request #551 from efiop/master 
increase test coverage to 90%
 2018-03-14 20:31:30 +02:00
Ruslan Kuprieiev 8db50b88e2 test: add test for bad dir cache files 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-14 20:11:37 +02:00
Ruslan Kuprieiev 75c0ffaa68 test: test CLI failures for push/pull/fetch/status 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-14 20:11:05 +02:00
Ruslan Kuprieiev 1d17961732 sync: capture generic exceptions 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-14 20:10:42 +02:00
Ruslan Kuprieiev 7355c15f61 Merge pull request #548 from efiop/master 
dvc: don't change mode of files
 2018-03-14 12:56:26 +02:00
Ruslan Kuprieiev 2fe6091ffb dvc: don't change mode of files 
Since we can now detect cache corruption, we don't need
to force any permissions on our data files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-14 12:48:16 +02:00
Ruslan Kuprieiev 7019b796ba Merge pull request #547 from efiop/master 
tests: checkout: sleep(1) to emulate realistic behavior
 2018-03-13 20:58:58 +02:00
Ruslan Kuprieiev 3ab62555bd tests: checkout: sleep(1) to emulate realistic behavior 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-13 20:51:29 +02:00
Ruslan Kuprieiev 3e29e29c0b Merge pull request #545 from efiop/master 
State and travis fixes
 2018-03-13 20:24:34 +02:00
Ruslan Kuprieiev 26a7562242 travis: don't install graphviz 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-13 20:18:03 +02:00
Ruslan Kuprieiev b728d942c7 state: optimize state storage and computation 
Reduces 'dvc add' time for 30K files from 25sec to 9sec.

Fixes #546

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-13 20:18:03 +02:00
Ruslan Kuprieiev c9f4a4076c test: state: sleep 1 sec to simulate realistic behavior 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-13 18:57:33 +02:00
Ruslan Kuprieiev d8bdc7fea1 Merge pull request #544 from efiop/master 
test: add debug msgs for mtime issue
 2018-03-12 22:51:39 +02:00
Ruslan Kuprieiev 97643de654 test: add debug msgs for mtime issue 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-12 22:51:17 +02:00
Ruslan Kuprieiev 786171db20 Merge pull request #543 from efiop/master 
tests: add debug msg
 2018-03-12 20:40:32 +02:00
Ruslan Kuprieiev 440768b056 tests: add debug msg 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-12 20:38:57 +02:00
Ruslan Kuprieiev 04d11c1762 Remove pygraphviz from install_requires 2018-03-12 18:58:46 +02:00
Ruslan Kuprieiev 96bb4640a3 Remove pygraphviz from requirements.txt 2018-03-12 18:58:15 +02:00
Ruslan Kuprieiev d62f9da6b0 Merge pull request #541 from efiop/master 
dvc: remove 'show pipeline'
 2018-03-12 18:57:26 +02:00
Ruslan Kuprieiev 79e034b493 dvc: remove 'show pipeline' 
Fixes #533

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-12 01:18:52 +02:00
Ruslan Kuprieiev 10293e1006 Merge pull request #540 from efiop/master 
cache: check for cache corruption
 2018-03-12 01:03:09 +02:00
Ruslan Kuprieiev 0cd0e6700f cache: check of cache corruption 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-12 00:58:45 +02:00
Ruslan Kuprieiev 3091564e10 codeclimate: specify coverage file 2018-03-11 09:27:47 +02:00
Ruslan Kuprieiev 78cbc9c790 travis: add debug info for codeclimate 2018-03-11 00:00:14 +02:00
Ruslan Kuprieiev 2f9eac0b85 Merge pull request #539 from efiop/master 
travis: try using python reporter for codeclimate
 2018-03-10 23:46:36 +02:00
Ruslan Kuprieiev 85664061a6 travis: try using python reporter for codeclimate 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 23:41:36 +02:00
Ruslan Kuprieiev be87288f95 Merge pull request #538 from efiop/master 
Misc fixes
 2018-03-10 19:21:21 +02:00
Ruslan Kuprieiev 200d1dec29 dvc: move remove_untracked_hardlinks to inside the project 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 19:15:06 +02:00
Ruslan Kuprieiev 04855c8631 test: remove directory from gc test 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 19:13:14 +02:00
Ruslan Kuprieiev 0068e90074 test: enable 'local cloud' tests for windows 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 18:46:14 +02:00
Ruslan Kuprieiev f913fd8ad9 dvc: properly remove cached dir 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 18:41:23 +02:00
Ruslan Kuprieiev 5a7bee4508 dvc: fix progress bar 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 18:41:09 +02:00
Ruslan Kuprieiev 5fa796d95c Merge pull request #537 from efiop/master 
checkout: optimize lost hardlink search
 2018-03-10 13:14:26 +02:00
Ruslan Kuprieiev dce1935651 checkout: optimize lost hardlink search 
Speeds up checkout for 30K files from 6sec to 3.5sec

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 13:10:15 +02:00
Ruslan Kuprieiev 066151617e Merge pull request #536 from efiop/master 
dvc: calculate 'effective inode' from getfileinfo() on windows
 2018-03-10 12:47:38 +02:00
Ruslan Kuprieiev 7094ec0151 dvc: calculate 'effective inode' from getfileinfo() on windows 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-10 12:40:46 +02:00
Ruslan Kuprieiev 4cd789bf2a Merge pull request #532 from efiop/master 
dvc: switch from yaml to json for dir cache file
 2018-03-09 08:40:07 +02:00
Ruslan Kuprieiev 3b9259ff17 dvc: switch from yaml to json for dir cache file 
yaml is extremely slow without CLoader, which caused our checkout()
and similar functions to take way too long to process dir cache files.
These files are not really meant to be read by the users, so we can
sacrifice the readability if yaml for better performance of json module.

E.g. checkout for 30K files used to take 30sec and after this patch only
takes 6sec, which is still quite slow, but much more bearable.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-09 08:22:44 +02:00
Ruslan Kuprieiev a32ac41e3c Merge pull request #531 from efiop/master 
Fixes for pull/push CLI and cache
 2018-03-08 22:48:32 +02:00
Ruslan Kuprieiev 54a6532360 cache: create cache dirs only when needed 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-08 19:45:58 +02:00
Ruslan Kuprieiev 2e21c39c00 cli: move push/pull from under 'dvc cache' 
Fixes #528

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-08 14:03:49 +02:00
Ruslan Kuprieiev 1248b56f44 Merge pull request #529 from efiop/master 
config and cloud fixes
 2018-03-08 13:27:22 +02:00
Ruslan Kuprieiev 4bc62cbf66 aws: don't forget to use relpath in progress callback 
Fixes #519

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-08 12:44:44 +02:00
Ruslan Kuprieiev ff77d9dc46 dvc: calculate n jobs as 8*n_cpu 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-08 09:51:24 +02:00
Ruslan Kuprieiev fa43309222 config: rename 'Global' section to 'Core' 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-03-08 09:04:28 +02:00
Ruslan Kuprieiev 2e83045d72 Merge pull request #526 from efiop/master 
repro: don't fail if dir cache is not present
 2018-03-08 08:44:17 +02:00