Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-10 15:28:34 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,879 Commits 7 Branches 579 Tags
c71e5562bbdb5074fca9be9aa4071ec44b438753
Commit Graph
3879 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev c71e5562bb Merge pull request #2713 from GeorgeSabu-mec/master 
fix typo(correct spelling of existence)
 2019-11-03 12:44:03 +02:00
Ruslan Kuprieiev 601a77ed4b Merge pull request #2712 from the-thirteenth-star/master 
fixed typo
 2019-11-03 12:42:37 +02:00
GeorgeSabu-MEC 1e22fabece fix typo(correct spelling of existence) 2019-11-03 15:42:07 +05:30
Naina 1afc40c1ec Update fetch.py 2019-11-03 15:40:27 +05:30
Ruslan Kuprieiev 372e1070ac Merge pull request #2703 from kurianbenoy/master 
Remove multi-threaded prefix before progress bar(#2686)
 2019-11-03 00:34:19 +02:00
Ruslan Kuprieiev e21fc34c84 Merge pull request #2698 from algomaster99/test-argument-types-get_mtime_and_size 
Verify if `get_mtime_and_size` can take both Path objects and strings
 2019-11-03 00:25:47 +02:00
Aman Sharma cffc9c8ed9 Remove unnecessary wrapping of path 2019-11-03 02:38:50 +05:30
Aman Sharma d454cb8637 Modify test to check for directory as well 2019-11-03 02:23:46 +05:30
Casper da Costa-Luis 440b323e39 TqdmThreadPoolExecutor => ThreadPoolExecutor 
- Fixes https://github.com/iterative/dvc/pull/2703#discussion_r341763150
 2019-11-02 15:22:46 +00:00
Casper da Costa-Luis 82dae6c25b remove special TqdmThreadPoolExecutor logic 
- fixes #2686
- closes #2703
 2019-11-02 15:22:46 +00:00
kurianbenoy 306c916626 Change import order in progress file 2019-11-02 15:22:46 +00:00
kurianbenoy 21bafccc96 Remove multi-threaded prefix before progress bar(#2686) 2019-11-02 15:22:46 +00:00
Ruslan Kuprieiev 2e6485f24b Merge pull request #2709 from mroutis/fix-2704 
sftp: make move across drives possible
 2019-11-02 14:07:46 +02:00
Ruslan Kuprieiev 91a7af03e7 Merge pull request #2707 from Suor/urlinfo 
perf: refactor URLInfo to not use ParseResult inside
 2019-11-02 13:49:49 +02:00
Mr. Outis 16164e65ad sftp: move -> use atomic copying 2019-11-01 17:01:50 -06:00
Ruslan Kuprieiev 809a85a340 dvc: bump to 0.66.3 0.66.3 2019-11-01 23:58:17 +02:00
Ruslan Kuprieiev ce8e019dd1 Merge pull request #2705 from shcheklein/fix-2702 
get-url: fix NPE when src non local and dst not absolute
 2019-11-01 23:54:53 +02:00
Ruslan Kuprieiev 7ecbdab564 Merge pull request #2693 from sadielbartholomew/i2685-pull-cmd-output 
pull, at import stage: replace warning with info report
 2019-11-01 23:15:23 +02:00
Aman Sharma f4f9f3aac1 Replace path with path_info 2019-11-02 01:15:17 +05:30
Mr. Outis 8f7000056b sftp: move -> fallback to copy when sftp.rename fails 2019-11-01 13:26:03 -06:00
Mr. Outis 283f848822 sftp: make move an atomic operation 2019-11-01 13:17:22 -06:00
Mr. Outis ab1ead881b sftp: make move across drives possible 
Fix #2704
 2019-11-01 12:40:16 -06:00
Ivan Shcheklein 3660d9eac4 local output: address PR comment, fix style a bit 2019-11-01 09:49:56 -07:00
Alexander Schepanovski b83564d80e perf: refactor URLInfo to not use ParseResult inside 
This permites URLInfo construction from parts without building/reparsing
string, which makes it around 20% faster. Code is also simpler and a bit
shorter now.

Other improvements:
- added URLInfo.replace(path=...)
- URLInfo._path now stringifies properly
- protected against passing path not starting with /
- do not allow query, params, fragments and passwords in urls (they were
silently ignored previously)
 2019-11-01 17:33:00 +01:00
Ruslan Kuprieiev c999b5d270 remove info message 2019-11-01 16:45:41 +02:00
Ruslan Kuprieiev 5c8a312002 fix order 2019-11-01 16:41:22 +02:00
Ruslan Kuprieiev 0dd4d2361e fix formatting 2019-11-01 16:40:06 +02:00
Sadie Bartholomew 31551e6e2e CLI: change return output for 'checkout' and 'pull' commands 
Change messages output after return of some commands:

* Remove warning for 'dvc checkout' & commands calling it under the hood
* Add an informational message output for 'dvc pull' command

Fixes #2685.
 2019-10-31 23:01:24 +00:00
Ivan Shcheklein 98750c95c5 get-url: fix NPE when src non local and dst not absolute 2019-10-31 09:58:07 -07:00
Ruslan Kuprieiev d932405ee1 Merge pull request #2695 from mroutis/test-s3-etag 
tests: preserve the same ETag across buckets during copy
 2019-10-31 01:55:17 +02:00
Aman Sharma b9a17752e3 Test acceptable path types of get_mtime_and_size 2019-10-31 00:30:54 +05:30
Mr. Outis 732ed904ce tests: same ETag across buckets 2019-10-30 11:42:39 -06:00
Ruslan Kuprieiev e623c4faf7 dvc: bump to 0.66.2 0.66.2 2019-10-30 17:20:05 +02:00
Ivan ShchekleinandRuslan Kuprieiev 2ec0aa0c37 failing while pulling imported directories (#2692) 
* pull: fix-2684 - failing while pulling imported directories

* pull: fix-2684 - fix code indentation

* minor imported directories pull test update

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
 2019-10-30 17:18:07 +02:00
Alexander Schepanovski 77fcd49695 dvc: fix tqdm dep (#2694) 
tqdm==4.35.0 was throwing KeyError: 'percentage'.
 2019-10-30 16:43:18 +02:00
Ruslan Kuprieiev 877e38551e lock: add a workaround for a flufl.lock bug (#2688) 
This bug doesn't affect us during regular dvc CLI commands, but is not
visualy pleasing during API usage.

Fixes #2573
 2019-10-29 20:54:56 +02:00
Jorge Orpinel a48481b01f dvc: change ` for ' in dynamic parts of output messages (#2644) 
* dvc: change ` for ' in dinamic parts of DVC outputs

per https://github.com/iterative/dvc/pull/2644#issuecomment-545151398

* dvc: use ' for dynamic output and ` for DVC names

per https://github.com/iterative/dvc/pull/2644#discussion_r338417698
 2019-10-28 22:00:53 +02:00
Sagun Karanjit 5d6918a04c Remove --all-commits options from usage output (#2682) 
* Comment out --all-commits optional arg

* Suppress --all-commits optional arg
 2019-10-28 17:27:34 +02:00
Aman Sharma 5f644745c7 Write test for get_inode (#2673) 2019-10-28 16:43:47 +02:00
Ruslan Kuprieiev b9c7d88fcc dvc: bump to 0.66.1 0.66.1 2019-10-26 23:36:13 +03:00
Ruslan Kuprieiev 64f78c164d Revert "Added path.lstrip for / in imports (#2651)" (#2675) 
This reverts commit bfc0a3596f.
 2019-10-26 23:35:22 +03:00
Ruslan Kuprieiev 1baf020de7 s3: don't use walk_files for list_cache_paths 
No need to convert back and forth from string to PathInfo
 2019-10-26 00:29:23 +03:00
Ruslan Kuprieiev 1ab9319682 gs: fix typo 0.66.0 2019-10-25 23:24:43 +03:00
Ruslan Kuprieiev 32ba773aee s3: don't forget to convert to regular path strings 2019-10-25 23:07:17 +03:00
Ruslan Kuprieiev fc485581f9 dvc: bump to 0.66.0 2019-10-25 22:46:50 +03:00
Ruslan Kuprieiev f3bacb7c28 gs: retry upload with smaller chunk size (#2661) 
Default chunk size for gs is 100M, which might be too much for
particular network. So if we are getting ConnectionError, we
should try lowering the chunk size until we reach the minimum
allowed chunk size of 256K. Also note that `chunk_size` must be a
multiple of 256K per the API specification.

Fixes #2572
 2019-10-25 22:45:54 +03:00
Sanket Saurav 88f45c2348 Exclude compat files from DeepSource analysis (#2670) 
Hi team,

I just [noticed](https://github.com/iterative/dvc/pull/2548#issuecomment-536255251) that there are could be quite a number of false-positives in `dvc/utils/compat.py` because of, well, the things we have to do for compatibility. I would recommend adding that file to `exclude_patterns` to prevent the noise. I've proposed that in this PR.

We've seen some other projects report this as well, and we're working on proactively suggesting this via the product UI to make it easier.

Hope this helps!

Best,
Sanket
Founder, DeepSource
 2019-10-25 15:28:09 +03:00
Mr. Outis 27b63e2674 s3: support for directories (#2619) 
* s3: list_objects -> list_objects_api

* s3: use path_info for _list_paths

* s3: separate _list_paths from _list_objects

* s3: support max items for listing objects/paths

* s3: take into account directories on `exists`

* s3: overwrite `isdir` method

* s3: custom collect directory entry for s3

* s3: strip checksum when collecting directories

* remote: change walk implementation to walk_files

* s3: add comment about isdir logic

* s3: use `dir_path` correctly on `isdir`

* tests: add a few unit tests for S3

* s3: use path_info join instead of posixpath

* tests: use parametrize for s3 tests

* remote: use posixpath accordingly

* tests: remove undeeded AWS variables from S3 test

* tests: rearrange parametrize values for readability

* tests: add encoding, use a single s3 instance, for loop instead of parameterize

* py2: fix encoding
 2019-10-25 03:41:04 +03:00
Ruslan Kuprieiev 35499faa7d readme: remove brew cask and update brew install cmd 2019-10-25 01:41:46 +03:00
Casper da Costa-Luis 9ce3b0c50c add: output improvements (#2546) 
* some doc improvements

* add: intermediate progress

* update after rebase

* linting

* fix checksum progress cleanup by ensuring gc

* progress-aware logger.info

* re-fix checksum pbar closure

* multi-file add

* make stage.dump() less verbose.

TODO: check all instances of stage.dump and possibly make verbose

* decrease verbosity of checksum calculations

* leave dvc.add stats

* update/fix logger tests

* minor doc reword

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* tidy API target_;list => targets

- as per https://github.com/iterative/dvc/pull/2546#discussion_r337042535

* tidy RecursiveAddingWhileUsingFilename message duplication

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337044670

* minor logging uncomment

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337051097

* minor doc improvements

- https://github.com/iterative/dvc/pull/2546#discussion_r337116255
- https://github.com/iterative/dvc/pull/2546#discussion_r337116809

* tidy exception

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337434142

* minor doc update

- fixes https://github.com/iterative/dvc/pull/2546/files#r337294491
- related to https://github.com/iterative/dvc.org/issues/719

* Adding => add

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r337037137

* various review fixes

- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r337927837
- revert warning style fix
  https://github.com/iterative/dvc/pull/2546#discussion_r338046035
- Abstract error consistency
  https://github.com/iterative/dvc/pull/2546#discussion_r337931070
- minor comment as per
  https://github.com/iterative/dvc/pull/2546#discussion_r338254958

* remove diabling progress option

- fixes https://github.com/iterative/dvc/pull/2546#discussion_r338314050
 2019-10-25 01:09:19 +03:00