Commit Graph
43 Commits
Author SHA1 Message Date
pared 8ee53f8c81 stage: extract yaml loading to util method 2019-02-28 16:27:16 +01:00
pared f8ef28c4c6 stage: check if local path contains symlink and, if so, treat added files as relpath 2019-02-28 15:00:05 +01:00
pared a8a32091f7 remote local: add dir state update after processing the files 2019-02-26 12:15:02 +01:00
pared 4464e29c15 remote local/state : add check for cache dir metadata changes 2019-02-25 18:35:02 +01:00
Ruslan Kuprieiev 9ca00adde8 dvc: modify flake8 configuration
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-22 19:36:19 +02:00
Ruslan Kuprieiev 01a120f9fc dvc: introduce dvc commit
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-16 21:19:36 +02:00
Ruslan Kuprieiev 99f4107d58 add: introduce --no-commit
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-16 21:18:54 +02:00
pared a3c873d953 add: update state for known checksum for dir 2019-02-16 18:45:28 +01:00
pared c4274bd1a4 add: recalculate checksum after link creation, test 2019-02-16 16:30:20 +01:00
Ruslan Kuprieiev ba8d83e9b1 dvc: refactor dvc project class
1) split Project's methods into separate files;
2) rename Project into Repo(similar to gitpython's Repo)

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-13 03:31:23 +02:00
Mr. Outis 2be7232d1a py2: Add unicode support for open and os.walk 2019-02-07 23:33:30 -06:00
Ruslan Kuprieiev 441fc3ad97 add: don't try to save cached stages
Cached stages are already saved, so there is no need to try to save them
once again.

Fixes #1581

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-02-06 14:26:21 +02:00
Ruslan Kuprieiev 4069e34130 dvc: use black instead of flake8
Fixes #1546

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-01-30 03:48:36 +02:00
Mr. Outis 0d46921ec7 py2: add unicode literals
Fix #1512
2019-01-27 20:40:21 -06:00
Ruslan Kuprieiev c5b8d098d8 local: support remote:// syntax
Fixes #1451

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-01-02 17:43:24 +02:00
Mr. Outis 4c420d5f4d logger [rfc]: Use a module instead of a class
- Remove deprecation warnings
- Format error output and remove duplicated info

Close #1086
2018-12-23 21:06:39 -06:00
Ruslan Kuprieiev a67988a80e test: enable flake8 checks for tests
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-12-06 06:40:55 +02:00
Ruslan Kuprieiev de76db6f02 dvc: catch cached stages
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-12-05 05:47:44 +02:00
Ruslan Kuprieiev 31fed211f6 test: use suffixes and prefixes in tmpdir names
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-10-14 00:37:02 +03:00
Ruslan Kuprieiev c6f642fd6b dvc: add --recursive
Fixes #1026

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-21 10:41:19 +03:00
Ruslan Kuprieiev a46b1be925 test: add test for dvc add for external local files
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-17 14:07:55 +03:00
Ruslan Kuprieiev f5bb3e4501 tests: use shallow=False with filecmp.cmp()
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-08-05 00:52:29 +03:00
Ruslan Kuprieiev 2f7a7638e2 dvc: properly resolve cwd for dvc add
Fixes #943

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-24 15:36:28 +03:00
Ruslan Kuprieiev 0cea36a2dd dvc: don't forget to convert to ospath for dir cache entries
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-24 15:36:13 +03:00
Ruslan Kuprieiev 6a6a4fe0b4 test: test unsupported chemes
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-07-19 23:05:41 +03:00
Ruslan Kuprieiev 78e5b166fe dvc: replace StageNotFoundError with NotDvcFileError
Fixes #760

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-13 14:00:03 +03:00
Ruslan Kuprieiev 88397ef430 dvc: use Remote concept in existing code
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-06-04 22:28:14 +03:00
Ruslan Kuprieiev 2bbefc91ef deps: introduce s3/gs dependencies
E.g.:
dvc run -d gs://bucket/file -o file gsutil cp gs://bucket/file file
dvc run -d s3://bucket/file -o file aws s3 cp s3://bucket/file file

Etags are used to track deps in both cases.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-05-24 18:24:18 +03:00
Ruslan Kuprieiev 21bf52ee9d dvc: split output and dependency classes
Preparation for the 'external deps' patches.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-05-23 00:01:04 +03:00
Ruslan Kuprieiev 29a44689c5 Merge pull request #658 from seqpipe/add-directory-with-slash
Proper adding of 'directory/'
2018-04-17 12:46:42 +03:00
Emil Gotsev 286153a609 Proper adding of 'directory/'
* added a test for adding single level directory with forward slash in the end
 * normalizing the path that is passed to the add method
2018-04-17 12:21:32 +03:00
Ruslan Kuprieiev 8839e72f07 dvc: stop using checksumdir and use our own way of checksuming a dir
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-04-10 01:56:06 +03:00
Ruslan Kuprieiev 2fe6091ffb dvc: don't change mode of files
Since we can now detect cache corruption, we don't need
to force any permissions on our data files.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-03-14 12:48:16 +02:00
Ruslan Kuprieiev 7c61c75d23 dvc: split stage and output files
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-06 13:51:11 +02:00
Ruslan Kuprieiev 14ae7811c1 test: add modified dir test
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-03 14:46:13 +02:00
Ruslan Kuprieiev 24f8c4ca9f test: add test for adding same data in and inside dir
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-02-02 16:53:35 +02:00
Ruslan Kuprieiev e4d9755a7e dvc: cloud RFC add support for directories
Fixes #367

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-22 03:19:29 +02:00
Ruslan Kuprieiev a130a25e39 dvc: refactor data_cloud
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-15 21:11:25 +02:00
Ruslan Kuprieiev 6a6c11ffdb dvc: remove lock
As discussed with @dmpetrov, we don't really see a good usage scenario
for this and thus removing it from dvc.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-13 01:54:20 +02:00
Dmitry 8c4712710c Missing data source error message 2018-01-11 09:45:43 -08:00
Ruslan Kuprieiev 487eae3f07 test: improve test coverage
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-10 17:15:42 +02:00
Ruslan Kuprieiev 03ce77f8c1 dvc: don't track files that are already tracked by git
Fixes #371

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-04 13:34:31 +02:00
Ruslan Kuprieiev f47a2c7585 dvc: initial unittests
Initial unittests for new design.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2018-01-01 16:52:06 +02:00