Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,902 Commits 7 Branches 579 Tags
4117206c67e1d1ee32fb1649112b190defc78f24
Commit Graph
4902 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexander Schepanovski 4117206c67 test: drop repo_template fixture 2020-01-20 11:49:13 +07:00
Ruslan Kuprieiev 7d65002bf0 Merge pull request #3178 from efiop/1843 
logger: remote: use lazy formatting
 2020-01-19 12:16:01 +02:00
Ruslan Kuprieiev 19acded31f logger: utils: fs: use lazy formatting 2020-01-19 11:53:29 +02:00
Ruslan Kuprieiev 65c0c7e60d logger: state: use lazy formatting 2020-01-19 11:53:29 +02:00
Ruslan Kuprieiev c2dae4b49c logger: remote: use lazy formatting 
Part of #1843

As our previous investigations showed, path stringification and stuff
like relpath are taking a very large chunk of time when working with
giant directories. This patch removes `format()`s and uses lazy
formatting provided by logger instead, so stuff like path_info is not
stringified until actually needed (e.g. on `-v`).
 2020-01-19 11:53:29 +02:00
Ruslan Kuprieiev 6f8ac0c1b1 logger: support {} formatting 2020-01-19 11:53:29 +02:00
Ruslan Kuprieiev 3987086275 Merge pull request #3169 from fabiosantoscode/feature/2963-dvc-pull-of-s3-remote-with-https-url 
pull: treat HTTP redirects without Location header as error
 2020-01-19 10:00:05 +02:00
Ruslan Kuprieiev 74c5af1a84 Merge pull request #3130 from jorgeorpinel/master 
api: better code documentation
 2020-01-19 09:05:52 +02:00
Ruslan Kuprieiev cf013576d4 Merge pull request #3120 from sharidas/no_remote_issue 
Fix import issue without remote config in the target
 2020-01-19 09:03:38 +02:00
Ruslan Kuprieiev 1fb79fa6e9 Merge pull request #3188 from mroutis/fix-3184 
completion: get --show-url
 2020-01-19 08:55:48 +02:00
Ruslan Kuprieiev 3c73e94b4e Merge pull request #3195 from mroutis/fix-boto3-import 
s3: import boto3 inside _copy instead of module level
 2020-01-19 08:29:52 +02:00
Sujith H 1b7eda2745 Fix import issue without remote config in the target 
Fix the import issue when target does not have
remote config.

Signed-off-by: Sujith H <[email protected]>
 2020-01-19 11:42:05 +05:30
Mr. Outis de1de68f90 s3: import boto3 inside _copy instead of module level 
Kudos to @skshetry for bringing this out during a review
 2020-01-18 23:01:50 -06:00
Mr. Outis 6ad55ec3b0 completion: get --show-url 2020-01-17 18:50:08 -06:00
Jorge Orpinel 2b818e203f api: note get_url doesn't chekc file existence 
per https://github.com/iterative/dvc/pull/3130#discussion_r367359439
 2020-01-17 17:40:02 -06:00
Jorge Orpinel 4d05ad2789 Merge branch 'master' of github.com:iterative/dvc 2020-01-17 17:39:51 -06:00
Ruslan Kuprieiev 7f5e4a9fe0 Merge pull request #3187 from iterative/pull-3186-restyled 
Restyle s3: copy large objects without chunking
 2020-01-17 20:38:36 +02:00
Restyled.io c3628c87d5 Restyled by black 2020-01-17 18:00:19 +00:00
Andy Sarroff 0bd433d153 s3: copy large objects without chunking 
When executing `boto3.s3.Client.Copy`, large files are chunked into
multiple parts. The default chunk threshold and size is 8MB. This
caused an issue when caching an object larger than 8M, where the
original object had not been uploaded in multiple parts. The original
object and the cached object would have different ETags, causing an
exception. This fixes the issue by dynamically setting the chunk
threshold to be one byte larger than the size of the original object
therefore avoiding multipart copies.

Fixes #3174
 2020-01-17 12:00:42 -05:00
Ruslan Kuprieiev 4d49b6ba1d Merge pull request #3170 from pared/3105 
utils: resolve_output: normpath before extracting basename
 2020-01-17 12:41:25 +02:00
Paweł Redzyński cf08529362 utils: add tests for resolve out 2020-01-17 09:51:07 +01:00
Paweł Redzyński 34d3261e58 utils: resolve_output: normpath before extracting basename 2020-01-17 09:29:52 +01:00
Ruslan Kuprieiev bdd1d61803 Merge pull request #3156 from skshetry/get-url-from-cache 
get: implement --show-url to display only url/path to remote
 2020-01-17 10:15:06 +02:00
Ruslan Kuprieiev 6ebd053c0f Merge pull request #3165 from mroutis/remove-py2-compat 
setup: remove py2 backports
 2020-01-17 04:28:22 +02:00
Ruslan Kuprieiev 422ff579f4 Merge pull request #3173 from casperdcl/snap-deploy 
minor comments
 2020-01-17 01:11:49 +02:00
Casper da Costa-Luis d34aa4471b minor comments 
- explains review concerns in #3158
 2020-01-16 20:36:42 +00:00
Fábio Santos df22dd4def specifically look for Location header in response 2020-01-16 17:58:44 +00:00
Saugat PachhaiandPaweł Redzyński 98f6d56e8c test: refactor, apply suggestion from @pared 
Co-Authored-By: Paweł Redzyński <[email protected]>
 2020-01-16 22:59:58 +05:45
Saugat Pachhai 02a73a0d9b unit: test: check output of get --show-url command 2020-01-16 22:52:35 +05:45
Ruslan Kuprieiev 74bbc3bc04 Merge pull request #3135 from rxxg/master 
Perf improvement for file copies
 2020-01-16 18:46:58 +02:00
rxxg 083a630b44 Fallback to standard Python code in case of missing packages 2020-01-16 17:08:07 +01:00
Saugat Pachhai 50206d44dc test: fix test related to path on windows 
On windows, `str(exc_info)` is returning escaped path, eg:
'<ExceptionInfo UrlNotDvcRepoError("URL \'C:\\\\Users\\\\travis\\\\AppData\\\\Local\\\\Temp\\\\pytest-of-travis\\\\pytest-0\\\\popen-gw2\\\\test_get_url_git_only_repo_thr0\' is not a dvc repository.") tblen=2>'

This commit just replaces the double slashes with single one. Bit of a hack, I know.
 2020-01-16 21:06:52 +05:45
Ruslan Kuprieiev d081627501 Merge pull request #3150 from fabiosantoscode/feature/2959-status-of-imported-files 
status: implement support for imported files
 2020-01-16 17:11:31 +02:00
Fábio Santos fcb7681683 take exception transformation into account 2020-01-16 14:49:51 +00:00
Fábio Santos cd3ef1850d simplify control flow 2020-01-16 14:44:09 +00:00
Fábio Santos c5eb13c114 update redirect check 2020-01-16 14:34:09 +00:00
Fábio Santos d3f925ae63 add missing return 2020-01-16 14:30:03 +00:00
Fábio Santos 968e74890d pull: treat HTTP redirects without Location header as error 
Fixes #2963
 2020-01-16 14:21:59 +00:00
Saugat Pachhai aa7119a741 get: test get_url raises exception if not a dvc repo 2020-01-16 19:28:14 +05:45
Saugat Pachhai f130e18e2c get: use new exception from public api 2020-01-16 19:28:14 +05:45
Saugat Pachhai fadf60eea6 api: throw exception by get_url if not a dvc repo 2020-01-16 19:28:14 +05:45
Saugat Pachhai eb252e055b get: split command for getting url and getting files from url functions 2020-01-16 19:28:14 +05:45
Saugat Pachhai 4509f82bc3 get: implement --show-url to display only url/path to remote 2020-01-16 19:28:14 +05:45
Fábio Santos a9bdaec63c rework code path; remove extraneous context generator 2020-01-16 13:37:13 +00:00
Fábio Santos 5883d3dabf reorder imports 2020-01-16 12:54:05 +00:00
Fábio Santos a0bdaa4c9e use checksum facilities 2020-01-16 12:51:20 +00:00
Ruslan Kuprieiev 1f1fb03138 Merge pull request #3142 from pared/remove_fixtures 
conftest: remove dvc_repo, erepo, git, git_erepo
 2020-01-16 13:20:45 +02:00
Ruslan Kuprieiev d160f83717 Merge pull request #3017 from pared/errno_24_message 
point to troubleshooting guide on too many open files error
 2020-01-16 12:56:14 +02:00
Paweł Redzyński 36c64c66db test_get_from_non_dvc_master: add comment about git logs 2020-01-16 11:49:03 +01:00
Ruslan Kuprieiev d1f172f332 Merge pull request #3158 from casperdcl/snap-deploy 
major deploy tidy
 2020-01-16 12:23:22 +02:00