Commit Graph
6 Commits
Author SHA1 Message Date
Saugat Pachhai b9000ed926 pre-commit: update black to 21.5b1 (#6032)
* pre-commit: update black to 21.5b1

also updated others, but black is more noticeable

* remove magic trailing comma
2021-05-18 15:51:15 +00:00
Ruslan Kuprieiev 89b40afee7 fs: make constructor closer to fsspec (#5911)
* fs: don't use repo for jobs/hash_jobs

* fs: local: exists: use fsspec's LocalFileSystem

* fs: make constructor comply with fsspec
2021-05-12 22:03:09 +03:00
Ruslan Kuprieiev e3a7857b13 repo: move index_dir to RemoteIndex (#4631) 2020-09-29 03:32:33 +03:00
Saugat Pachhai ad306d7bf3 Enable most of the pylint checks (#4111) 2020-06-25 18:10:14 +03:00
Peter Rowlands (변기호) 0fb9870de6 remote: rename "checksum" -> "hash" (#4051)
* remote: s/get_checksum/get_hash/

* remote/tree: s/get_file_checksum/get_file_hash/

* remote: s/get_dir_checksum/get_dir_hash/

* remote: s/checksum_to_path/hash_to_path/

* remote: s/path_to_checksum/path_to_hash/

* remote: cleanup remaining checksum references

* remote.index: s/checksum/hash/

* note: does not rename "checksum" field in the index DB schema
2020-06-16 12:16:51 +03:00
Peter Rowlands (변기호) b04964f4b7 remote: locally index list of checksums available on cloud remotes (#3634)
* remote: locally index list of checksums available on cloud remotes

* fix review issues

- use pytest fixtures
- use sha256 digest
- s/fd/fobj/
- check more specific IO errors

* remote: use .dir checksums to validate index

* command: add --drop-index option for push/pull/status

* tests: --drop-index unit tests

* remote: make index threadsafe

* remote: force re-index after gc

* only save/dump index once per command

* remote: add helper index functions update_all/replace_all

* remote: store index as sqlite3 database

* fix deepsource warnings

* rename --drop-index to --clear-index

* rename index.invalidate to index.clear

* functional tests for remote.index

* fix index not being updated on status -c

* remote: add index.intersection()

* remote: use index when making assumptions about remote .dir contents

* add missing func tests

* do not index standalone files

* cleanup index test fixture

* fix deepsource warnings

* update autocompletion scripts/cli help message for --clear-index

* tests: remove unused mocker

* push: only index successfully pushed dirs

- do not index files on partial push/upload

* remote: revert behavior to trust .dir files on remote

- skip unnecessary index check for dir contents if .dir file exists on
  the remote

* fix missing RemoteIndexNoop functions

* separate index validation from status dir exists query

* bugfix: include indexed checksums for all cache_exists cases

* review fix: cleanup debug messages

* tests: fix DS warning

* remove --clear-index

* remove unused index functions
2020-04-21 15:13:45 +03:00