Commit Graph
3378 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 36d924c4c2 test: add test for ssh cloud
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-08 22:26:18 +03:00
Ruslan Kuprieiev 9fdf949723 dvc: use DataCloud.supported() method for url testing
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-08 00:28:31 +03:00
Ruslan Kuprieiev 3988397a3d tests: data cloud: limit jobs to 2
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-07 05:23:58 +03:00
Ruslan Kuprieiev eef12da7b6 tests: fix typo 2018-04-07 05:14:57 +03:00
Ruslan Kuprieiev f5ef256447 test: don't forget to copy gcp credentials
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-07 05:09:45 +03:00
Ruslan Kuprieiev e8b5af6cd9 test: test remotes in CLI
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-07 02:34:00 +03:00
Ruslan Kuprieiev a948f5ca90 gs: don't require ProjectName
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-05 21:31:20 +03:00
Ruslan Kuprieiev fb4633072e dvc: introduce remotes
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-05 21:31:20 +03:00
Ruslan Kuprieiev 5a8aef28f5 config: empty config by default + format verification
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-05 01:51:52 +03:00
Ruslan Kuprieiev 7d5f6164d0 test: add test for stage file format
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-20 10:40:38 +02:00
Ruslan Kuprieiev 958eab2f23 dvc: fix checkout/repro with stage files without md5
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-20 00:31:30 +02:00
Ruslan Kuprieiev e9acd5a321 test: add test for state reload consistency
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-18 12:45:50 +02:00
Ruslan Kuprieiev 286002f9cb test: add '--force' test for repro
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-17 11:42:05 +02:00
Ruslan Kuprieiev b38b12962d dvc: remove fsck
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-16 11:23:54 +02:00
Ruslan Kuprieiev 2447415657 dvc: move CmdDataStatus to a separate file
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-15 10:30:12 +02:00
Ruslan Kuprieiev 8db50b88e2 test: add test for bad dir cache files
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-14 20:11:37 +02:00
Ruslan Kuprieiev 75c0ffaa68 test: test CLI failures for push/pull/fetch/status
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-14 20:11:05 +02:00
Ruslan Kuprieiev 2fe6091ffb dvc: don't change mode of files
Since we can now detect cache corruption, we don't need
to force any permissions on our data files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-14 12:48:16 +02:00
Ruslan Kuprieiev 3ab62555bd tests: checkout: sleep(1) to emulate realistic behavior
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-13 20:51:29 +02:00
Ruslan Kuprieiev b728d942c7 state: optimize state storage and computation
Reduces 'dvc add' time for 30K files from 25sec to 9sec.

Fixes #546

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-13 20:18:03 +02:00
Ruslan Kuprieiev c9f4a4076c test: state: sleep 1 sec to simulate realistic behavior
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-13 18:57:33 +02:00
Ruslan Kuprieiev 97643de654 test: add debug msgs for mtime issue
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-12 22:51:17 +02:00
Ruslan Kuprieiev 440768b056 tests: add debug msg
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-12 20:38:57 +02:00
Ruslan Kuprieiev 79e034b493 dvc: remove 'show pipeline'
Fixes #533

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-12 01:18:52 +02:00
Ruslan Kuprieiev 0cd0e6700f cache: check of cache corruption
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-12 00:58:45 +02:00
Ruslan Kuprieiev 04855c8631 test: remove directory from gc test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-10 19:13:14 +02:00
Ruslan Kuprieiev 0068e90074 test: enable 'local cloud' tests for windows
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-10 18:46:14 +02:00
Ruslan Kuprieiev 54a6532360 cache: create cache dirs only when needed
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-08 19:45:58 +02:00
Ruslan Kuprieiev 2e21c39c00 cli: move push/pull from under 'dvc cache'
Fixes #528

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-08 14:03:49 +02:00
Ruslan Kuprieiev fa43309222 config: rename 'Global' section to 'Core'
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-08 09:04:28 +02:00
Ruslan Kuprieiev 783fad935b remove: use stage files as targets and only remove outputs
Fixes #516

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-08 06:59:01 +02:00
Ruslan Kuprieiev 7f4c7783e7 dvc: store cached dir structure in a single json file
Fixes #507

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-04 11:15:20 +02:00
Ruslan Kuprieiev f814269b4c dvc: properly collect used cache
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-03 04:25:46 +02:00
Ruslan Kuprieiev 8b85d1c201 dvc: cloud: reuse connections when processing targets
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-02 04:00:40 +02:00
Ruslan Kuprieiev 44bcf9fa6b test: fix data cloud test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-02 03:28:20 +02:00
Ruslan Kuprieiev 539ac41b70 dvc: use two-level hash for cache
Fixes #488

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-02 02:51:20 +02:00
Ruslan Kuprieiev f96f1f7ee3 tests: add test for reproducing directories
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-01 09:15:13 +02:00
Ruslan Kuprieiev 068f742852 dvc: use zc.lockfile instead of fasteners
fasteners' InterProcessLock doesn't work within
one process, thus it is not suitable for our dvc
library.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-27 04:20:32 +02:00
Ruslan Kuprieiev d87e8e47af test: add test for non-existing sections/fields in config
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-27 04:20:32 +02:00
Ruslan Kuprieiev a05eddd746 tests: add test for cli data cloud
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-27 04:20:32 +02:00
Ruslan Kuprieiev 36323010fc tests: add test for removing directory
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-27 04:20:32 +02:00
Ruslan Kuprieiev 576ba96e25 Revert "cloud: process targets one-by-one"
This reverts commit 61b469144e.
2018-02-18 23:10:59 +02:00
Ruslan Kuprieiev 61b469144e cloud: process targets one-by-one
In all operations we're dealing with a single stream,
so there is no good reason to process targets in parallel
as they all use the same resource to pull\push data to.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-17 19:36:51 +02:00
Ruslan Kuprieiev d938e44ceb test: don't forget to import shutils 2018-02-15 21:30:46 +02:00
Ruslan Kuprieiev a58f3b8184 test: enable gcp tests
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-15 20:47:45 +02:00
Ruslan Kuprieiev bccdbfb15f Revert "travis/appveyor: test: enable gcp tests" 2018-02-14 23:18:41 +02:00
Ruslan Kuprieiev f109b69903 test: enable gcp tests
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-14 22:39:14 +02:00
Ruslan Kuprieiev 77e61f40be test: test AWS on appveyor
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-14 18:36:19 +02:00
Ruslan Kuprieiev d54f803eb6 system: remove unused code
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-14 18:22:33 +02:00
Ruslan Kuprieiev ecea487614 Revert "test: test AWS on appveyor" 2018-02-14 14:09:17 +02:00