Peter Rowlands (변기호) 9b38aa4680 remote: Optimize traverse/no_traverse behavior (#3501) ...

* remote: Optimize traverse/no_traverse behavior

- estimate remote file count by fetching a single parent cache dir, then
  determine whether or not to use no_traverse method for checking
  remainder of cache entries in `cache_exists`
- thread requests when traversing full remote file lists (by fetching
  one parent cache dir per thread)

* Use arbitrary prefix length for traverse remote cache queries

- default to 3
- for remotes that only support per-directory query use 2 (gdrive, hdfs)

* Add unit tests for new traverse/no_traverse behavior

* Fix deepsource warning

* Fix review issues

* fix CI

* Fix review issues

* obsolete no_traverse config option

- remove RemoteBASE.DEFAULT_NO_TRAVERSE and replace it with
  RemoteBASE.CAN_TRAVERSE (True for all remotes except http)

* review issues - use 'object exists' instead of 'no_traverse'

* fix obsolete no_traverse test issues

* fix SSHMocked func tests

2020-03-25 05:04:20 +02:00

..

remote

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

__init__.py

test: move func tests to tests/func

2019-04-16 20:19:22 +03:00

test_add.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_analytics.py

analytics: fix resolving used scm (#3405)

2020-02-26 02:29:46 +02:00

test_api.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_cache.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_checkout.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_cli.py

dvc: automatically reorder imports

2019-11-05 02:34:40 +02:00

test_commit.py

black: bump version 19.3b0 -> 19.10b0 (#3307)

2020-02-12 02:41:24 +02:00

test_config.py

dvc: do not set cache dir if not a dvc repo (#3329)

2020-02-14 19:07:23 +02:00

test_data_cloud.py

remote: Optimize traverse/no_traverse behavior (#3501)

2020-03-25 05:04:20 +02:00

test_dependency.py

dependency: test: migrate to dir helpers

2020-01-06 16:06:52 +01:00

test_diff.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_external_repo.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_fs.py

Move copyfile to fs.py

2020-01-09 04:57:53 +05:30

test_gc.py

dvc: use consistent messages for InvalidArgumentError (#3506)

2020-03-18 21:34:46 +02:00

test_get_url.py

test: get_url: convert to dir helpers

2020-01-13 13:34:33 +02:00

test_get.py

get/import: retrieve files inside directory outs (#3309)

2020-02-17 03:24:02 +02:00

test_ignore.py

SCM: allow multiple DVC repos inside single SCM repo (#3257)

2020-02-25 14:40:20 +02:00

test_import_url.py

imp_url: make default stage fname accompany import target (#3312)

2020-02-14 17:05:08 +02:00

test_import.py

external_repo: do not try too hard to fix upstream (#3512)

2020-03-20 15:54:47 +02:00

test_init.py

SCM: allow multiple DVC repos inside single SCM repo (#3257)

2020-02-25 14:40:20 +02:00

test_install.py

Use pre-commit for Git hooks (#3406)

2020-03-16 20:34:01 +02:00

test_lock.py

dvc: automatically reorder imports

2019-11-05 02:34:40 +02:00

test_ls.py

list: rename target param to path, update docs (#3462)

2020-03-16 20:22:48 +02:00

test_metrics.py

diff: use rev instead of ref (#3299)

2020-02-11 00:13:23 +02:00

test_move.py

test: move: convert to dir helpers

2019-12-09 19:27:28 +01:00

test_output.py

Merge pull request #3083 from pared/2896_output

2020-01-08 22:14:07 +02:00

test_pipeline.py

pipeline: fix --outs output (#3507)

2020-03-18 23:03:41 +02:00

test_remote.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_remove.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_repo.py

dvc: refactor config (#3298)

2020-02-13 19:35:37 +02:00

test_repro.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_run.py

Revert "Restyle #3393 introduce hyper parameters and config (#3516)" (#3517)

2020-03-20 00:54:21 +02:00

test_s3.py

test: merge _should_test_aws and _get_aws_url inside S3 test helper

2020-01-06 22:08:28 +05:45

test_scm.py

SCM: allow multiple DVC repos inside single SCM repo (#3257)

2020-02-25 14:40:20 +02:00

test_stage.py

SCM: allow multiple DVC repos inside single SCM repo (#3257)

2020-02-25 14:40:20 +02:00

test_state.py

checkout: show summary by default and introduce --show-changes flag (#3401)

2020-03-16 16:29:36 +02:00

test_status.py

black: bump version 19.3b0 -> 19.10b0 (#3307)

2020-02-12 02:41:24 +02:00

test_system.py

Fix typos

2019-11-26 20:07:20 +05:45

test_tag.py

dvc: automatically reorder imports

2019-11-05 02:34:40 +02:00

test_tree.py

py3: use new magic from PEP 3135

2020-01-02 16:24:31 -06:00

test_unprotect.py

dvc: use protected mode by default (#3472)

2020-03-17 19:16:21 +02:00

test_update.py

update: support specifying revision when updating (#3337)

2020-02-15 17:55:41 +02:00

test_utils.py

Move copyfile to fs.py

2020-01-09 04:57:53 +05:30

test_version.py

version: fix bug when .git dir is removed, show more info regarding repo (#3508)

2020-03-19 13:06:34 +02:00