Commit Graph
77 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 5c762f316e repro: don't check for missing dependencies when running with --dry
Fixes #1113

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-09-14 00:23:28 +03:00
Ruslan Kuprieiev a3aefe5f16 state: don't forget subdirs when computing effective mtime
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-09-01 15:41:42 +03:00
Ruslan Kuprieiev 9c041a375f test: sleep to compensate for poor mtime resolution 2018-08-31 05:09:32 +03:00
Ruslan Kuprieiev 392cd006e3 tests: sleep() before creating a file 2018-08-30 17:27:05 +03:00
Ruslan Kuprieiev ac80d59704 dvc: properly compute mtime of the directory
Fixes #1067

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-29 19:17:50 +03:00
Ruslan Kuprieiev 076a2145db test: don't forget to sleep before modifying a file
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-29 14:54:27 +03:00
Ruslan Kuprieiev 49baf38871 dvc: properly compute mtime for the directory
Fixes #1067

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-29 13:52:20 +03:00
Ruslan Kuprieiev 6f0e293c9a test: cover pipeline show
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-27 14:40:21 +03:00
Ruslan Kuprieiev 18f0ca0d92 dependency: only use rel_path for local deps
Fixes #1048

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-24 14:36:38 +03:00
Ruslan Kuprieiev c6f642fd6b dvc: add --recursive
Fixes #1026

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-21 10:41:19 +03:00
Ruslan Kuprieiev 86322fbe35 repro: print help when Dvcfile doesn't exist
Fixes #1003

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-14 02:08:12 +03:00
Ruslan Kuprieiev f5bb3e4501 tests: use shallow=False with filecmp.cmp()
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-05 00:52:29 +03:00
Ruslan Kuprieiev ed8052c229 repro: introduce --dry option
Fixes #972
Related to #894

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-05 00:21:35 +03:00
Ruslan Kuprieiev b9a5bde22f test: cover status
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-04 19:59:23 +03:00
Ruslan Kuprieiev 07fa6442b8 test: add dvc status simple test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-19 22:56:53 +03:00
Ruslan Kuprieiev e598b0b4c7 test: repro: add test for failed stage in CLI
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-19 08:36:32 +03:00
Ruslan Kuprieiev 870d98bc18 test: fix expected number of stages 2018-07-17 15:57:31 +03:00
Ruslan Kuprieiev dc5fec6a75 test: remove unused add() call 2018-07-17 15:47:38 +03:00
Ruslan Kuprieiev 805eb908d0 import: support external urls
Fixes #891

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-17 12:00:12 +03:00
Ruslan Kuprieiev b952e599bf debug: add a few more sleep()s to test_repro 2018-07-03 02:43:01 +03:00
Ruslan Kuprieiev 71d03b5a43 test: add sleep() calls
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-03 01:33:13 +03:00
Ruslan Kuprieiev f634bc1916 test: remote: add basic test for remove() and checkout()
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-01 22:34:48 +03:00
Ruslan Kuprieiev a392289cbf travis: run hdfs test only on Linux
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-01 10:02:43 +03:00
Ruslan Kuprieiev 591cd90583 travis: enable hdfs test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-01 09:44:42 +03:00
Ruslan Kuprieiev 432fa56315 test: don't set gs env when running a command 2018-07-01 02:58:30 +03:00
Ruslan Kuprieiev 24d44695f7 test: fix typo 2018-07-01 02:45:02 +03:00
Ruslan Kuprieiev fde11a3b7f test: use GOOGLE_APPLICATION_CREDENTIALS when running GS repro test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-01 02:36:22 +03:00
Ruslan Kuprieiev fb4fc72642 tests: try enable GS test on CI
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-01 02:08:06 +03:00
Ruslan Kuprieiev 12edcbaa04 remote: support HDFS for push/pull/status from local repo
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-30 22:54:49 +03:00
Ruslan Kuprieiev 50578a5923 gc: support s3/gs/ssh/hdfs
Fixes #738

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-28 21:32:57 +03:00
Ruslan Kuprieiev b8371d1c76 dvc: bring back dvc lock with dvc unlock
Fixes #523

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-26 02:53:07 +03:00
Ruslan Kuprieiev 0b29f8748f test: use contents without \n
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-24 18:21:20 +03:00
Ruslan Kuprieiev 71752f68a8 hdfs: fix drivers and tests
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-24 17:23:47 +03:00
Ruslan Kuprieiev db7251b3ea dvc: support ssh deps/outs/cache
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-23 18:47:52 +03:00
Ruslan Kuprieiev 6886fc29d5 dvc: support dependecy files under tracked directory
Fixes #722

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-17 09:32:29 +03:00
Ruslan Kuprieiev 4271ddf11c dvc introduce dvc import
Fixes #705

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-15 18:37:50 +03:00
Ruslan Kuprieiev b074c49318 dvc: support external local deps
Fixes #764

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-15 03:35:53 +03:00
Ruslan Kuprieiev ff0d91737b dvc: initial hdfs support
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-10 05:32:16 +03:00
Ruslan Kuprieiev a734e07776 dvc: support deps/outputs that are relative to specific remote
Fixes #748

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-09 02:03:02 +03:00
Ruslan Kuprieiev c136297b82 stage: try to specify user's shell when running commands
Fixes #749

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-07 23:19:29 +03:00
Ruslan Kuprieiev 9389422f0b dvc: add --cwd for dvc repro
Fixes #401

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-06 01:42:06 +03:00
Ruslan Kuprieiev dd610c04fb test: add test for s3/gs out/dep/cache
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-04 22:28:15 +03:00
Ruslan Kuprieiev 88397ef430 dvc: use Remote concept in existing code
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev e0a42f1265 outs: support gs/s3 as an output without cache
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev ac3d37e58c test: temporarily disable gs dependency test on CI
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-05-24 18:58:20 +03:00
Ruslan Kuprieiev 2bbefc91ef deps: introduce s3/gs dependencies
E.g.:
dvc run -d gs://bucket/file -o file gsutil cp gs://bucket/file file
dvc run -d s3://bucket/file -o file aws s3 cp s3://bucket/file file

Etags are used to track deps in both cases.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-05-24 18:24:18 +03:00
Ruslan Kuprieiev 9a88a412d0 dvc: reproduce stage if it has no dependencies and has cmd
This allows as to define conditional stages in the dependency graph.

Fixes #646
Fixes #648

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-29 17:43:32 +03:00
Ruslan Kuprieiev 958eab2f23 dvc: fix checkout/repro with stage files without md5
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-20 00:31:30 +02:00
Ruslan Kuprieiev 286002f9cb test: add '--force' test for repro
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-17 11:42:05 +02:00
Ruslan Kuprieiev 2fe6091ffb dvc: don't change mode of files
Since we can now detect cache corruption, we don't need
to force any permissions on our data files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-14 12:48:16 +02:00