Commit Graph
14 Commits
Author SHA1 Message Date
Saugat Pachhai b8aa8ad951 pyupgrade: 3.6+ (#3805)
* pyupgrade: 3.6+

* tests: pyupgrade 3.6+
2020-05-15 16:48:02 +03:00
Peter Rowlands (변기호) 833e82a7ea Drop Python 3.5 support (#3792)
* drop python 3.5 support

* dvc: remove fspath() use inside os.path calls

* dvc: replace dvc.compat.fspath with os.fspath

* tests: replace dvc.compat.fspath with os.fspath

* utils: relpath() should use fspath/str

* utils: os.readlink doesn't support path-like obj on Windows until 3.8
2020-05-14 01:46:42 +03:00
Ivan ShchekleinandJorge Orpinel 8aefbac223 gdrive: fix multi-remote workflow, cont. cleanup (#3686)
* remote, minor: fix parameter method name for consistency

* gdrive: cleanup, fix workflow with multiple gdrive remotes

* config: resolve gdrive cred file parth, typo fix

* grdive: address deepsource warning

* gdrive: fix tests after simplifying auth flow

* gdrive: address PR review, use backticks where appropriate

Co-Authored-By: Jorge Orpinel <[email protected]>

* gdrive: exception text improvements

Co-Authored-By: Jorge Orpinel <[email protected]>

* gdrive: fix exception message

Co-Authored-By: Jorge Orpinel <[email protected]>

* gdrive: fix root not found exception message

Co-Authored-By: Jorge Orpinel <[email protected]>

* gdrive: minor warnings/exceptions text improvement

* gdrive: add tests for the gdrive_user_credentials_file relative path

* gdrive: address review, slightly change text

* gdrive: comments -> docstrings, addressing PR review

Co-authored-by: Jorge Orpinel <[email protected]>
2020-04-29 23:25:56 +03:00
nik123 fefbcc0b4c refactor: dvc/remotes name unification (#3684)
Partially fixes #2089
2020-04-26 20:55:00 +03:00
Peter Rowlands (변기호) b04964f4b7 remote: locally index list of checksums available on cloud remotes (#3634)
* remote: locally index list of checksums available on cloud remotes

* fix review issues

- use pytest fixtures
- use sha256 digest
- s/fd/fobj/
- check more specific IO errors

* remote: use .dir checksums to validate index

* command: add --drop-index option for push/pull/status

* tests: --drop-index unit tests

* remote: make index threadsafe

* remote: force re-index after gc

* only save/dump index once per command

* remote: add helper index functions update_all/replace_all

* remote: store index as sqlite3 database

* fix deepsource warnings

* rename --drop-index to --clear-index

* rename index.invalidate to index.clear

* functional tests for remote.index

* fix index not being updated on status -c

* remote: add index.intersection()

* remote: use index when making assumptions about remote .dir contents

* add missing func tests

* do not index standalone files

* cleanup index test fixture

* fix deepsource warnings

* update autocompletion scripts/cli help message for --clear-index

* tests: remove unused mocker

* push: only index successfully pushed dirs

- do not index files on partial push/upload

* remote: revert behavior to trust .dir files on remote

- skip unnecessary index check for dir contents if .dir file exists on
  the remote

* fix missing RemoteIndexNoop functions

* separate index validation from status dir exists query

* bugfix: include indexed checksums for all cache_exists cases

* review fix: cleanup debug messages

* tests: fix DS warning

* remove --clear-index

* remove unused index functions
2020-04-21 15:13:45 +03:00
Ruslan Kuprieiev 6559b9d222 dvc: use protected mode by default (#3472)
* dvc: use protected mode by default

Fixes #3261
Fixes #2041

* unprotect: adjust help message

https://github.com/iterative/dvc.org/pull/1058/files/21aab371a487acf6f6e6201b29cd832e7c55ed23#r393324668
2020-03-17 19:16:21 +02:00
Mr. Outis 753aef4910 black: bump version 19.3b0 -> 19.10b0 (#3307) 2020-02-12 02:41:24 +02:00
pawel ce0eb08ba6 tests: remote: local: migrate to dir fixtures 2020-01-04 02:47:58 +01:00
Alexander Schepanovski d80293c44d dvc: path info refactoring
Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.

Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous

Both families might had caused undefined surprising behavior when local
and remote paths clash.
2019-06-11 19:19:17 +07:00
pawel fe3f83ee05 remote: local: test: refactor unpacked dir tests 2019-06-11 10:21:57 +02:00
pawel ac5ca4656a remote: progress callback for unpacked dir creation 2019-06-10 20:19:47 +02:00
pawel cfbd381d42 remote: create unpacked dir after status check if needed 2019-06-10 14:44:03 +02:00
pawel b5464cbc59 remote: local: check if status verification dir already exists 2019-06-10 14:44:03 +02:00
pared 90ca5bd35a remote: local: add cache status check hardlink optimization 2019-06-10 14:44:03 +02:00