Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-07 23:09:21 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,990 Commits 7 Branches 579 Tags
0.67.0
Commit Graph
982 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 54760c5604 Merge pull request #2758 from kss682/test_2672 
Test for issue #2672 added.
 2019-11-08 19:42:24 +02:00
kss682 7e336a8e7a test for issue 2672 --typo correction 2019-11-09 21:12:24 +05:30
kss682 4409c60468 Test for issue #2672 added. 2019-11-09 20:41:40 +05:30
Aman Sharma 204cfcb879 Ensure makedirs accepts str and Path-like objects 2019-11-08 20:37:32 +05:30
Ruslan Kuprieiev ec44a614b0 Merge pull request #2753 from efiop/analytics 
analytics: don't verify config from repo
 2019-11-08 14:38:02 +02:00
Ruslan Kuprieiev 1058df0f77 analytics: don't forget to convert to bool 2019-11-08 05:21:47 +02:00
Ruslan Kuprieiev 910a67b35f analytics: don't verify config from repo 
Otherwise this will raise an exception from analytics worker:
```
dvc config cache.type sometypo
```
But running that same command with `--system` or `--global` flag outside
of repo won't raise that.

Also need to better handle ConfigObjError, but that is a separate issue.
 2019-11-08 02:37:11 +02:00
Aman Sharma d4aa27d6ab Ensure copyfile accepts str and Path-like objects 2019-11-08 01:51:49 +05:30
pawel 4acfaa293e ignore: add test for dvcignore on branch 2019-11-07 17:51:00 +01:00
pawel f14aca630f ignore: make DvcIgnoreFilter use tree 2019-11-07 15:37:24 +01:00
Ruslan Kuprieiev fb8315997e Merge pull request #2746 from algomaster99/test-arg-types-file_md5 
Ensure `file_md5` accept str and Path-like objects
 2019-11-06 14:01:20 +02:00
Aman Sharma 34a868b420 Ensure file_md5 accept str and Path-like objects 2019-11-06 09:25:35 +05:30
Ruslan Kuprieiev ab0ba77ab2 Merge pull request #2745 from shcheklein/fix-2744 
dvc get leaves dangling symlinks if source repo is set to use symlinks as its cache type
 2019-11-06 03:08:37 +02:00
Ivan Shcheklein ffd8c2cf6e tests: add dvc import test to check that local cache type is enforced 2019-11-05 16:40:24 -08:00
Ivan Shcheklein c25715dc72 get: fix override external repo cache type 2019-11-05 16:27:01 -08:00
Ivan Shcheklein 73f95ee6b4 tests: minor style and typo fixes in dvc gc tests 2019-11-05 16:25:38 -08:00
Ruslan Kuprieiev 7caf28ca11 dvcignore: don't forget to use dvcignore when collecting dvcignores 
Fixes #2737
 2019-11-06 01:39:31 +02:00
Ruslan Kuprieiev 5e2fa3742b Merge pull request #2741 from algomaster99/remove-get-parent-dirs 
Remove `get_parent_dirs_up_to`
 2019-11-05 23:01:31 +02:00
Aman Sharma 8491ddf43f Remove get_parent_dirs_up_to 2019-11-06 02:03:47 +05:30
Aman Sharma f642ff76a5 Ensure contains_symlink_up_to accept both str and Path-like objects 2019-11-06 00:24:10 +05:30
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
n3hrox b97865e165 fix test 2019-11-04 21:18:07 +01:00
n3hrox 8e2d0bdcd6 Merge branch 'master' of github.com:iterative/dvc into enhance/conda_detection 2019-11-04 21:10:57 +01:00
n3hrox e3417f78be revert changes 2019-11-04 21:02:51 +01:00
n3hrox 952fee57c1 resolve conflicts 2019-11-04 20:52:24 +01:00
Ruslan Kuprieiev 4efd048796 test: try removing get_url test 2019-11-04 00:05:44 +02:00
Ruslan Kuprieiev 46391cb4f9 test: fix formatting 2019-11-03 23:47:34 +02:00
Ruslan Kuprieiev 71f8d25ab3 test: get-url: try using with mock_s3() instead of a decorator 2019-11-03 23:42:44 +02:00
Ruslan Kuprieiev 2df7eda159 Merge pull request #2674 from n3hrox/enhance/footnote_deduplication 
logger: output footer once for multithreaded push/pull errors
 2019-11-03 23:37:13 +02:00
Ruslan Kuprieiev 46d221fade test: s3: try using boto3.client 2019-11-03 22:38:48 +02:00
Ruslan Kuprieiev 0fece74d00 test: get-url: don't use get_aws_url 2019-11-03 14:18:25 +02:00
Ruslan Kuprieiev c71e5562bb Merge pull request #2713 from GeorgeSabu-mec/master 
fix typo(correct spelling of existence)
 2019-11-03 12:44:03 +02:00
GeorgeSabu-MEC 1e22fabece fix typo(correct spelling of existence) 2019-11-03 15:42:07 +05:30
Ruslan Kuprieiev e21fc34c84 Merge pull request #2698 from algomaster99/test-argument-types-get_mtime_and_size 
Verify if `get_mtime_and_size` can take both Path objects and strings
 2019-11-03 00:25:47 +02:00
Aman Sharma d454cb8637 Modify test to check for directory as well 2019-11-03 02:23:46 +05:30
Ruslan Kuprieiev 2e6485f24b Merge pull request #2709 from mroutis/fix-2704 
sftp: make move across drives possible
 2019-11-02 14:07:46 +02:00
Mr. Outis 283f848822 sftp: make move an atomic operation 2019-11-01 13:17:22 -06:00
n3hrox 89502bee88 fix tests 2019-11-01 18:05:27 +01:00
Ivan Shcheklein 98750c95c5 get-url: fix NPE when src non local and dst not absolute 2019-10-31 09:58:07 -07:00
Ruslan Kuprieiev d932405ee1 Merge pull request #2695 from mroutis/test-s3-etag 
tests: preserve the same ETag across buckets during copy
 2019-10-31 01:55:17 +02:00
Aman Sharma b9a17752e3 Test acceptable path types of get_mtime_and_size 2019-10-31 00:30:54 +05:30
Mr. Outis 732ed904ce tests: same ETag across buckets 2019-10-30 11:42:39 -06:00
Ivan ShchekleinandRuslan Kuprieiev 2ec0aa0c37 failing while pulling imported directories (#2692) 
* pull: fix-2684 - failing while pulling imported directories

* pull: fix-2684 - fix code indentation

* minor imported directories pull test update

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-10-30 17:18:07 +02:00
Jorge Orpinel a48481b01f dvc: change ` for ' in dynamic parts of output messages (#2644) 
* dvc: change ` for ' in dinamic parts of DVC outputs

per https://github.com/iterative/dvc/pull/2644#issuecomment-545151398

* dvc: use ' for dynamic output and ` for DVC names

per https://github.com/iterative/dvc/pull/2644#discussion_r338417698
 2019-10-28 22:00:53 +02:00
Aman Sharma 5f644745c7 Write test for get_inode (#2673) 2019-10-28 16:43:47 +02:00
n3hrox 752e584a9b refactor function and add test 2019-10-27 22:25:27 +01:00
Ruslan Kuprieiev f3bacb7c28 gs: retry upload with smaller chunk size (#2661) 
Default chunk size for gs is 100M, which might be too much for
particular network. So if we are getting ConnectionError, we
should try lowering the chunk size until we reach the minimum
allowed chunk size of 256K. Also note that `chunk_size` must be a
multiple of 256K per the API specification.

Fixes #2572
 2019-10-25 22:45:54 +03:00
Mr. Outis 27b63e2674 s3: support for directories (#2619) 
* s3: list_objects -> list_objects_api

* s3: use path_info for _list_paths

* s3: separate _list_paths from _list_objects

* s3: support max items for listing objects/paths

* s3: take into account directories on `exists`

* s3: overwrite `isdir` method

* s3: custom collect directory entry for s3

* s3: strip checksum when collecting directories

* remote: change walk implementation to walk_files

* s3: add comment about isdir logic

* s3: use `dir_path` correctly on `isdir`

* tests: add a few unit tests for S3

* s3: use path_info join instead of posixpath

* tests: use parametrize for s3 tests

* remote: use posixpath accordingly

* tests: remove undeeded AWS variables from S3 test

* tests: rearrange parametrize values for readability

* tests: add encoding, use a single s3 instance, for loop instead of parameterize

* py2: fix encoding
 2019-10-25 03:41:04 +03:00
Casper da Costa-Luis 9ce3b0c50c add: output improvements (#2546) 
* some doc improvements

* add: intermediate progress

* update after rebase

* linting

* fix checksum progress cleanup by ensuring gc

* progress-aware logger.info

* re-fix checksum pbar closure

* multi-file add

* make stage.dump() less verbose.

TODO: check all instances of stage.dump and possibly make verbose

* decrease verbosity of checksum calculations

* leave dvc.add stats

* update/fix logger tests

* minor doc reword

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* tidy API target_;list => targets

- as per https://github.com/iterative/dvc/pull/2546#discussion_r337042535

* tidy RecursiveAddingWhileUsingFilename message duplication

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337044670

* minor logging uncomment

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337051097

* minor doc improvements

- https://github.com/iterative/dvc/pull/2546#discussion_r337116255
- https://github.com/iterative/dvc/pull/2546#discussion_r337116809

* tidy exception

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337434142

* minor doc update

- fixes https://github.com/iterative/dvc/pull/2546/files#r337294491
- related to https://github.com/iterative/dvc.org/issues/719

* Adding => add

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337037137

* various review fixes

- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r337927837
- revert warning style fix
  https://github.com/iterative/dvc/pull/2546#discussion_r338046035
- Abstract error consistency
  https://github.com/iterative/dvc/pull/2546#discussion_r337931070
- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r338254958

* remove diabling progress option

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r338314050
 2019-10-25 01:09:19 +03:00
Ruslan Kuprieiev 480a612da4 output: base: don't forget to return empty cache (#2656) 
* output: base: don't forget to return empty cache

CC @Suor

Fixes https://gist.github.com/leimbernon/21924e02a998e3f1604990f8a7824c37

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* gc: add test for missing .dir cache file
 2019-10-23 22:47:50 +03:00