Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-09 23:10:24 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,139 Commits 7 Branches 579 Tags
128f60ef68227c8f7bcfe2de79de18eab215121f
Commit Graph
197 Commits
Author SHA1 Message Date
Tymoteusz Jankowski 128f60ef68 Add test_dagascii.py + make find_pager() use DVC_PAGER 2019-12-04 20:51:24 +01:00
Saugat Pachhai da0d9e8599 s3: implement isfile() 2019-12-01 10:58:53 +05:45
Saugat Pachhai 2e1f3114cd dvc: write tests 2019-11-30 19:47:30 +05:45
Ruslan Kuprieiev 0404a2324e Merge pull request #2551 from MaxRis/gdrive-support 
GDrive remote support
 2019-11-30 01:21:31 +02:00
Mr. Outis 1f9a0f0e0f address @efiop's suggestions 2019-11-29 12:38:12 -06:00
Mr. Outis 845cd352f7 s3: add support for top level empty directories 2019-11-29 11:46:28 -06:00
pawel f7846a5bed test: remote: http: clear test file name 2019-11-29 15:30:55 +01:00
pawel c102a781c2 HTTPError: reformat error message 2019-11-29 15:30:55 +01:00
Paweł Redzyński 7218ff202c remote: http: rename http error, refactor test 2019-11-29 15:30:55 +01:00
Paweł Redzyński 72abfb0572 remote: http: raise exception when download response with error status code 2019-11-29 15:30:55 +01:00
Max Risuhin e7227898e4 Support GDrive as remote 2019-11-29 01:21:10 -08:00
Saugat Pachhai 461a79e1e8 Fix typos 2019-11-26 20:07:20 +05:45
Ruslan Kuprieiev 7bc7a572f2 Merge pull request #2809 from casperdcl/gs-progress 
GS progress for push & pull
 2019-11-21 05:38:35 +02:00
Ruslan Kuprieiev 91ac75d023 Merge pull request #2798 from skshetry/fix-str-rel-repo 
Display path relative to temporary repo when get/import-ing files
 2019-11-20 11:34:31 +02:00
Casper da Costa-Luis b811a773c6 update dynamic chunk tests 2019-11-19 17:44:01 +00:00
Ruslan Kuprieiev 404620e267 test: update: use dvc_repo fixture 
To fix test failure.
 2019-11-19 19:12:49 +02:00
Saugat Pachhai 83c8ab5c26 test: Refactor tests 2019-11-19 21:55:13 +05:45
Saugat Pachhai f07a70a052 get/import: display path relative to temporary repo 2019-11-19 05:36:11 +05:45
Saugat Pachhai 68c7366ef0 utils: implement path_isin to compare child and parent paths 2019-11-19 05:34:33 +05:45
Aman Sharma 3edad21b41 Write test for tmp_fname to check if it accepts Path-like and str objects 2019-11-18 11:49:40 +05:30
Aman Sharma 636d850c84 Ensure remove accepts str and Path-like objects 2019-11-15 01:54:40 +05:30
Aman Sharma 9427bbe8d0 Move move to filesystem utility 2019-11-09 23:57:57 +05:30
Aman Sharma 44268251b4 Ensure move accepts str and Path-like objects 2019-11-09 18:47:01 +05:30
Aman Sharma 204cfcb879 Ensure makedirs accepts str and Path-like objects 2019-11-08 20:37:32 +05:30
Ruslan Kuprieiev ec44a614b0 Merge pull request #2753 from efiop/analytics 
analytics: don't verify config from repo
 2019-11-08 14:38:02 +02:00
Ruslan Kuprieiev 1058df0f77 analytics: don't forget to convert to bool 2019-11-08 05:21:47 +02:00
Ruslan Kuprieiev 910a67b35f analytics: don't verify config from repo 
Otherwise this will raise an exception from analytics worker:
```
dvc config cache.type sometypo
```
But running that same command with `--system` or `--global` flag outside
of repo won't raise that.

Also need to better handle ConfigObjError, but that is a separate issue.
 2019-11-08 02:37:11 +02:00
Aman Sharma d4aa27d6ab Ensure copyfile accepts str and Path-like objects 2019-11-08 01:51:49 +05:30
pawel f14aca630f ignore: make DvcIgnoreFilter use tree 2019-11-07 15:37:24 +01:00
Aman Sharma 34a868b420 Ensure file_md5 accept str and Path-like objects 2019-11-06 09:25:35 +05:30
Ruslan Kuprieiev 5e2fa3742b Merge pull request #2741 from algomaster99/remove-get-parent-dirs 
Remove `get_parent_dirs_up_to`
 2019-11-05 23:01:31 +02:00
Aman Sharma 8491ddf43f Remove get_parent_dirs_up_to 2019-11-06 02:03:47 +05:30
Aman Sharma f642ff76a5 Ensure contains_symlink_up_to accept both str and Path-like objects 2019-11-06 00:24:10 +05:30
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Ruslan Kuprieiev 2df7eda159 Merge pull request #2674 from n3hrox/enhance/footnote_deduplication 
logger: output footer once for multithreaded push/pull errors
 2019-11-03 23:37:13 +02:00
Ruslan Kuprieiev c71e5562bb Merge pull request #2713 from GeorgeSabu-mec/master 
fix typo(correct spelling of existence)
 2019-11-03 12:44:03 +02:00
GeorgeSabu-MEC 1e22fabece fix typo(correct spelling of existence) 2019-11-03 15:42:07 +05:30
Ruslan Kuprieiev e21fc34c84 Merge pull request #2698 from algomaster99/test-argument-types-get_mtime_and_size 
Verify if `get_mtime_and_size` can take both Path objects and strings
 2019-11-03 00:25:47 +02:00
Aman Sharma d454cb8637 Modify test to check for directory as well 2019-11-03 02:23:46 +05:30
Mr. Outis 283f848822 sftp: make move an atomic operation 2019-11-01 13:17:22 -06:00
n3hrox 89502bee88 fix tests 2019-11-01 18:05:27 +01:00
Ruslan Kuprieiev d932405ee1 Merge pull request #2695 from mroutis/test-s3-etag 
tests: preserve the same ETag across buckets during copy
 2019-10-31 01:55:17 +02:00
Aman Sharma b9a17752e3 Test acceptable path types of get_mtime_and_size 2019-10-31 00:30:54 +05:30
Mr. Outis 732ed904ce tests: same ETag across buckets 2019-10-30 11:42:39 -06:00
Jorge Orpinel a48481b01f dvc: change ` for ' in dynamic parts of output messages (#2644) 
* dvc: change ` for ' in dinamic parts of DVC outputs

per https://github.com/iterative/dvc/pull/2644#issuecomment-545151398

* dvc: use ' for dynamic output and ` for DVC names

per https://github.com/iterative/dvc/pull/2644#discussion_r338417698
 2019-10-28 22:00:53 +02:00
Aman Sharma 5f644745c7 Write test for get_inode (#2673) 2019-10-28 16:43:47 +02:00
Ruslan Kuprieiev f3bacb7c28 gs: retry upload with smaller chunk size (#2661) 
Default chunk size for gs is 100M, which might be too much for
particular network. So if we are getting ConnectionError, we
should try lowering the chunk size until we reach the minimum
allowed chunk size of 256K. Also note that `chunk_size` must be a
multiple of 256K per the API specification.

Fixes #2572
 2019-10-25 22:45:54 +03:00
Mr. Outis 27b63e2674 s3: support for directories (#2619) 
* s3: list_objects -> list_objects_api

* s3: use path_info for _list_paths

* s3: separate _list_paths from _list_objects

* s3: support max items for listing objects/paths

* s3: take into account directories on `exists`

* s3: overwrite `isdir` method

* s3: custom collect directory entry for s3

* s3: strip checksum when collecting directories

* remote: change walk implementation to walk_files

* s3: add comment about isdir logic

* s3: use `dir_path` correctly on `isdir`

* tests: add a few unit tests for S3

* s3: use path_info join instead of posixpath

* tests: use parametrize for s3 tests

* remote: use posixpath accordingly

* tests: remove undeeded AWS variables from S3 test

* tests: rearrange parametrize values for readability

* tests: add encoding, use a single s3 instance, for loop instead of parameterize

* py2: fix encoding
 2019-10-25 03:41:04 +03:00
Casper da Costa-Luis 9ce3b0c50c add: output improvements (#2546) 
* some doc improvements

* add: intermediate progress

* update after rebase

* linting

* fix checksum progress cleanup by ensuring gc

* progress-aware logger.info

* re-fix checksum pbar closure

* multi-file add

* make stage.dump() less verbose.

TODO: check all instances of stage.dump and possibly make verbose

* decrease verbosity of checksum calculations

* leave dvc.add stats

* update/fix logger tests

* minor doc reword

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* tidy API target_;list => targets

- as per https://github.com/iterative/dvc/pull/2546#discussion_r337042535

* tidy RecursiveAddingWhileUsingFilename message duplication

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337044670

* minor logging uncomment

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337051097

* minor doc improvements

- https://github.com/iterative/dvc/pull/2546#discussion_r337116255
- https://github.com/iterative/dvc/pull/2546#discussion_r337116809

* tidy exception

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337434142

* minor doc update

- fixes https://github.com/iterative/dvc/pull/2546/files#r337294491
- related to https://github.com/iterative/dvc.org/issues/719

* Adding => add

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337037137

* various review fixes

- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r337927837
- revert warning style fix
  https://github.com/iterative/dvc/pull/2546#discussion_r338046035
- Abstract error consistency
  https://github.com/iterative/dvc/pull/2546#discussion_r337931070
- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r338254958

* remove diabling progress option

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r338314050
 2019-10-25 01:09:19 +03:00
Alexander Schepanovski 33b0eefec0 dvc: dedup used cache (#2630) 
* dvc: dedup used cache

This is a preparation step to make garbage collector work with more
commits. It has a value of its own:
- less exists calls to show cloud status or prepare push/pull
- same names are not repeated in status and pbar texts

* dvc: delay checksum names concatenation
 2019-10-21 12:12:56 +03:00