Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,880 Commits 7 Branches 579 Tags
c3628c87d5d4fb1ca0995e8e293623a9cbfbc0d0
Commit Graph
4880 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Restyled.io c3628c87d5 Restyled by black 2020-01-17 18:00:19 +00:00
Andy Sarroff 0bd433d153 s3: copy large objects without chunking 
When executing `boto3.s3.Client.Copy`, large files are chunked into
multiple parts. The default chunk threshold and size is 8MB. This
caused an issue when caching an object larger than 8M, where the
original object had not been uploaded in multiple parts. The original
object and the cached object would have different ETags, causing an
exception. This fixes the issue by dynamically setting the chunk
threshold to be one byte larger than the size of the original object
therefore avoiding multipart copies.

Fixes #3174
 2020-01-17 12:00:42 -05:00
Ruslan Kuprieiev 4d49b6ba1d Merge pull request #3170 from pared/3105 
utils: resolve_output: normpath before extracting basename
 2020-01-17 12:41:25 +02:00
Paweł Redzyński cf08529362 utils: add tests for resolve out 2020-01-17 09:51:07 +01:00
Paweł Redzyński 34d3261e58 utils: resolve_output: normpath before extracting basename 2020-01-17 09:29:52 +01:00
Ruslan Kuprieiev bdd1d61803 Merge pull request #3156 from skshetry/get-url-from-cache 
get: implement --show-url to display only url/path to remote
 2020-01-17 10:15:06 +02:00
Ruslan Kuprieiev 6ebd053c0f Merge pull request #3165 from mroutis/remove-py2-compat 
setup: remove py2 backports
 2020-01-17 04:28:22 +02:00
Ruslan Kuprieiev 422ff579f4 Merge pull request #3173 from casperdcl/snap-deploy 
minor comments
 2020-01-17 01:11:49 +02:00
Casper da Costa-Luis d34aa4471b minor comments 
- explains review concerns in #3158
 2020-01-16 20:36:42 +00:00
Saugat PachhaiandPaweł Redzyński 98f6d56e8c test: refactor, apply suggestion from @pared 
Co-Authored-By: Paweł Redzyński <[email protected]>
 2020-01-16 22:59:58 +05:45
Saugat Pachhai 02a73a0d9b unit: test: check output of get --show-url command 2020-01-16 22:52:35 +05:45
Ruslan Kuprieiev 74bbc3bc04 Merge pull request #3135 from rxxg/master 
Perf improvement for file copies
 2020-01-16 18:46:58 +02:00
rxxg 083a630b44 Fallback to standard Python code in case of missing packages 2020-01-16 17:08:07 +01:00
Saugat Pachhai 50206d44dc test: fix test related to path on windows 
On windows, `str(exc_info)` is returning escaped path, eg:
'<ExceptionInfo UrlNotDvcRepoError("URL \'C:\\\\Users\\\\travis\\\\AppData\\\\Local\\\\Temp\\\\pytest-of-travis\\\\pytest-0\\\\popen-gw2\\\\test_get_url_git_only_repo_thr0\' is not a dvc repository.") tblen=2>'

This commit just replaces the double slashes with single one. Bit of a hack, I know.
 2020-01-16 21:06:52 +05:45
Ruslan Kuprieiev d081627501 Merge pull request #3150 from fabiosantoscode/feature/2959-status-of-imported-files 
status: implement support for imported files
 2020-01-16 17:11:31 +02:00
Fábio Santos fcb7681683 take exception transformation into account 2020-01-16 14:49:51 +00:00
Fábio Santos cd3ef1850d simplify control flow 2020-01-16 14:44:09 +00:00
Saugat Pachhai aa7119a741 get: test get_url raises exception if not a dvc repo 2020-01-16 19:28:14 +05:45
Saugat Pachhai f130e18e2c get: use new exception from public api 2020-01-16 19:28:14 +05:45
Saugat Pachhai fadf60eea6 api: throw exception by get_url if not a dvc repo 2020-01-16 19:28:14 +05:45
Saugat Pachhai eb252e055b get: split command for getting url and getting files from url functions 2020-01-16 19:28:14 +05:45
Saugat Pachhai 4509f82bc3 get: implement --show-url to display only url/path to remote 2020-01-16 19:28:14 +05:45
Fábio Santos a9bdaec63c rework code path; remove extraneous context generator 2020-01-16 13:37:13 +00:00
Fábio Santos 5883d3dabf reorder imports 2020-01-16 12:54:05 +00:00
Fábio Santos a0bdaa4c9e use checksum facilities 2020-01-16 12:51:20 +00:00
Ruslan Kuprieiev 1f1fb03138 Merge pull request #3142 from pared/remove_fixtures 
conftest: remove dvc_repo, erepo, git, git_erepo
 2020-01-16 13:20:45 +02:00
Ruslan Kuprieiev d160f83717 Merge pull request #3017 from pared/errno_24_message 
point to troubleshooting guide on too many open files error
 2020-01-16 12:56:14 +02:00
Paweł Redzyński 36c64c66db test_get_from_non_dvc_master: add comment about git logs 2020-01-16 11:49:03 +01:00
Ruslan Kuprieiev d1f172f332 Merge pull request #3158 from casperdcl/snap-deploy 
major deploy tidy
 2020-01-16 12:23:22 +02:00
Paweł Redzyński 18446996ab get: tes_get_from_non_dvc_master: capture git logs at INFO 2020-01-16 10:43:48 +01:00
pawel c2f5e35f1d conftest: remove repo_dir 2020-01-16 10:43:48 +01:00
pawel a4b55a00ec conftest: remove dvc_repo, erepo, git, git_erepo 2020-01-16 10:43:48 +01:00
Ruslan Kuprieiev 7d249d23e1 Merge pull request #3166 from mroutis/fix-3111 
api: identify local repos with os.path.exists
 2020-01-16 05:51:55 +02:00
Mr. Outis 388fcd6d32 api: identify local repos with os.path.exists 
Close #3111
 2020-01-15 21:31:09 -06:00
Mr. Outis 48326620ec setup: remove py2 backports 2020-01-15 19:13:01 -06:00
Ruslan Kuprieiev 6753e7b1e6 Merge pull request #3133 from JIoJIaJIu/core_checksum 
remote: config checksum_jobs
 2020-01-15 23:36:33 +02:00
Ruslan Kuprieiev ed0bc38e15 Merge pull request #3163 from MaxRis/3162 
Fix gdrive multiple parents file list query
 2020-01-15 23:32:57 +02:00
Max Risuhin 2d2ba0addf Fix gdrive multiple parents file list query 2020-01-15 13:10:55 -08:00
Ruslan Kuprieiev d2f81da696 Merge pull request #3161 from efiop/pydrive2 
dvc: use pydrive2 instead of pydrive
 2020-01-15 20:34:25 +02:00
Ruslan Kuprieiev 97a29a1f7e dvc: use pydrive2 instead of pydrive 
PyDrive2 is our maintained fork.
 2020-01-15 19:13:29 +02:00
Casper da Costa-Luis d8e6e34c34 minor use of new public API env var 2020-01-15 16:23:59 +00:00
Casper da Costa-Luis 4b16252c48 tidy deploy_condition 2020-01-15 15:41:02 +00:00
Casper da Costa-Luis 7b94181a0d deploy only on push 2020-01-15 15:26:29 +00:00
Casper da Costa-Luis d603048698 capitalisation 2020-01-15 15:20:09 +00:00
Casper da Costa-Luis 930b6cf45e fix pkg deploy condition 2020-01-15 15:16:29 +00:00
Casper da Costa-Luis 5bddfa7a92 major deploy tidy 
- fixes #3152
 2020-01-15 15:10:56 +00:00
Guro Bokum 4918c56ca3 remote: config checksum_jobs 
Close #2920
 2020-01-15 22:23:09 +08:00
pawel 0fcbfd04ce main: point to error.dvc.org after too many open files error 2020-01-15 15:14:24 +01:00
pawel d386237d86 main: format link to troubleshooting guide 2020-01-15 15:14:24 +01:00
pawel bc9a94b5ee point to troubleshooting guide on too many open files error 2020-01-15 15:14:24 +01:00