Commit Graph
4131 Commits
Author SHA1 Message Date
Saugat Pachhai 63cf117ee9 Move to tests/remotes.py 2019-11-30 08:58:43 +05:45
Saugat Pachhai 4714929b5b Merge branch 'master' into gs-external-deps
* master:
  address @efiop's suggestions
  s3: add support for top level empty directories
  remote: http: calculate length basing on response and not head call
  test: remote: http: clear test file name
  HTTPError: reformat error message
  Update dvc/remote/http.py
  remote: http: rename http error, refactor test
  remote: http: raise exception when download response with error status code
  Support GDrive as remote
  setup: don't forget contextlib2
  remote: refactor ssh ask password code
  remote: protect all remote client/session creation code with locks
  test: refactor & remove redundant test fixtures
  NoRemoteInExternalRepoError: dont pass cause of exception
  perf: optimize cache listing for local, ssh and hdfs
  remote: small .save_info()/.get_checksum() cleanup
2019-11-30 08:39:56 +05:45
Saugat Pachhai caa67c725e dvc: refactor tests 2019-11-30 08:22:39 +05:45
Ruslan Kuprieiev 0404a2324e Merge pull request #2551 from MaxRis/gdrive-support
GDrive remote support
2019-11-30 01:21:31 +02:00
Ruslan Kuprieiev 94f803e043 Merge pull request #2683 from mroutis/fix-2678
s3: ignore empty directories while walking files
2019-11-29 21:00:39 +02:00
Mr. Outis 1f9a0f0e0f address @efiop's suggestions 2019-11-29 12:38:12 -06:00
Mr. Outis 845cd352f7 s3: add support for top level empty directories 2019-11-29 11:46:28 -06:00
Ruslan Kuprieiev 6ad278f54c Merge pull request #2794 from pared/2510
remote: http: raise exception when response with error status code
2019-11-29 16:58:47 +02:00
pawel 03c27192a6 remote: http: calculate length basing on response and not head call 2019-11-29 15:31:04 +01:00
pawel f7846a5bed test: remote: http: clear test file name 2019-11-29 15:30:55 +01:00
pawel c102a781c2 HTTPError: reformat error message 2019-11-29 15:30:55 +01:00
Paweł RedzyńskiandAlexander Schepanovski e084b427e3 Update dvc/remote/http.py
Co-Authored-By: Alexander Schepanovski <[email protected]>
2019-11-29 15:30:55 +01:00
Paweł Redzyński 7218ff202c remote: http: rename http error, refactor test 2019-11-29 15:30:55 +01:00
Paweł Redzyński 72abfb0572 remote: http: raise exception when download response with error status code 2019-11-29 15:30:55 +01:00
Max Risuhin e7227898e4 Support GDrive as remote 2019-11-29 01:21:10 -08:00
Ruslan Kuprieiev 510ff8a9da Merge pull request #2863 from Suor/locks
remote: protect all remote client/session creation code with locks
2019-11-29 11:18:56 +02:00
Saugat Pachhai ee69ac835d gs: make it work with empty dir 2019-11-29 07:24:11 +05:45
Ruslan Kuprieiev 593ab17d25 setup: don't forget contextlib2 2019-11-29 02:14:25 +02:00
Ruslan Kuprieiev 6b4242a904 Merge pull request #2861 from danihodovic/refactor/remove-redundant-vars
test: refactor & remove redundant test fixtures
2019-11-29 01:24:27 +02:00
Alexander Schepanovski 8af1aeefb4 remote: refactor ssh ask password code 2019-11-28 21:10:35 +01:00
Alexander Schepanovski 73eac3b6cc remote: protect all remote client/session creation code with locks
With @cached_property last write wins, so all threads will use the same
session. However, several might be created concurrently wasting
resources and even potentially asking something from the user.

This change guarantees only one client instance will be created for a
remote.
2019-11-28 21:05:20 +01:00
Dani Hodovic 78c505f6cb test: refactor & remove redundant test fixtures
https://github.com/iterative/dvc/pull/2837#discussion_r351318487
2019-11-28 19:04:56 +01:00
Ruslan Kuprieiev 838efb293f Merge pull request #2836 from Suor/lazy-cache-paths
perf: optimize cache listing for local, ssh and hdfs
2019-11-28 13:23:46 +02:00
Ruslan Kuprieiev 00744bc794 Merge pull request #2835 from Suor/opt-checkout
remote: small .save_info()/.get_checksum() cleanup
2019-11-28 08:55:25 +02:00
Saugat Pachhai 54dd9890dc gs: return relative path in list_cache_paths 2019-11-28 07:37:14 +05:45
Saugat Pachhai db12416eac remove extra args 2019-11-28 06:05:30 +05:45
Saugat Pachhai fe248aa130 dvc: use replace instead of posixpath 2019-11-28 05:47:04 +05:45
Saugat Pachhai 0c08c09e5d dvc: append slash at the end for dir path_info 2019-11-28 05:45:30 +05:45
Ruslan Kuprieiev 78942ca4da Merge pull request #2854 from pared/2711_2
RemoteNotSpecifiedInExternalRepoError: dont pass cause of exception
2019-11-27 22:30:05 +02:00
Saugat Pachhai 5e2150d509 remote/gs: set max_results to improve performance when checking exists/isdir 2019-11-28 00:07:49 +05:45
Saugat Pachhai e750496123 remote/gs: support adding directories 2019-11-28 00:07:49 +05:45
Paweł Redzyński 31aed9b27b NoRemoteInExternalRepoError: dont pass cause of exception 2019-11-27 10:05:16 +01:00
Ruslan Kuprieiev f9d4ef8abd Update version.py 0.71.0 2019-11-26 20:55:49 +02:00
Ruslan Kuprieiev 5703b17190 Merge pull request #2856 from peper0/master
Possible fix for #2850.
2019-11-26 20:48:07 +02:00
Ruslan Kuprieiev a1cc2e66e1 Merge pull request #2821 from kaiogu/master
remote: fix remote default call with no arguments
2019-11-26 20:37:31 +02:00
Ruslan Kuprieiev 424931f54a Merge pull request #2855 from skshetry/fix-typo
Fix typos
2019-11-26 20:28:44 +02:00
Kaio Giurizatto Utsch 82a535b97e Change logging to print
Python logging recommends using print instead of logging for ordinary
output of command line program.

Also changed the test to reflect this.
2019-11-26 15:02:03 -03:00
Tomasz Łakota a986c6a09d Possible fix for #2850. 2019-11-26 18:25:36 +01:00
Kaio Giurizatto Utsch 95924a34e6 Undo last commit
Turns out that the argument was needed after all
2019-11-26 14:19:09 -03:00
Saugat Pachhai 461a79e1e8 Fix typos 2019-11-26 20:07:20 +05:45
Kaio Giurizatto Utsch 9611107896 Remove unused argument 2019-11-25 17:51:49 -03:00
Kaio Giurizatto Utsch c2def64dc7 remote: add test case to show default 2019-11-25 17:35:45 -03:00
Ruslan Kuprieiev 38be142497 Merge pull request #2842 from skshetry/fix-typo
Fix typo and grammar
2019-11-24 12:58:03 +02:00
Saugat Pachhai 7a678f3322 dvc: fix grammar in PR template 2019-11-24 15:44:42 +05:45
Saugat Pachhai 2859b53cae dvc: fix typo in data_cloud.py module docstring 2019-11-24 15:38:54 +05:45
Alexander Schepanovski 8681cc3cd1 perf: optimize cache listing for local, ssh and hdfs
- make all three lazy
- simplify local one
- use deque instead of list for hdfs
2019-11-22 14:43:01 +01:00
Alexander Schepanovski 8da0390c57 remote: small .save_info()/.get_checksum() cleanup 2019-11-22 12:58:08 +01:00
Ruslan Kuprieiev 3de46dd25f Merge pull request #2832 from skshetry/refactor-startswith-pathisin
Refactor `startswith` with `path_isin` util
2019-11-22 05:08:48 +02:00
saugat pachhai 9ae74073f2 dvc: remove extra newline 2019-11-22 05:36:06 +05:45
Saugat Pachhai 751bdb2197 dvc: refactor startswith usage with path_isin util 2019-11-21 22:22:08 +05:45