Commit Graph
6 Commits
Author SHA1 Message Date
eb46c03c6c output/remote: get rid of NamedCache (#6008)
* objects: make HashInfo & HashFile hashable

* stage/output: collect objects (HashFiles) instead of NamedCache

* remote/data_cloud: use objs instead of NamedCache

* repo: use objects instead of NamedCache

* rerun black after rebase

* remove NamedCache usage from run-cache

* drop NamedCache

* update HashInfo.__hash__

* update unit tests

* revert remote_name change

* start cleaning up test failures

* fix tree obj expansion

* update missing cache logging

* fix external dict handling

* update naive odb.gc

* update state save behavior for pull

* update test_data_cloud

* update test_remote

* fix gc with multiple repos

* objects.tree: support filtering w/preserved hash_info

* fix stage cache restore

* drop fs.path_info from HashFile.__hash__

* objects: add optional name attribute

* revert missing cache logging name/suffix changes

* revert name changes in remote tests

* review fixes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update dvc/remote/base.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2021-06-02 22:19:58 +03:00
Ruslan Kuprieiev 89b40afee7 fs: make constructor closer to fsspec (#5911)
* fs: don't use repo for jobs/hash_jobs

* fs: local: exists: use fsspec's LocalFileSystem

* fs: make constructor comply with fsspec
2021-05-12 22:03:09 +03:00
Batuhan Taskaya a42e363217 objects: don't recompute hash on odb.add() if verify is False (#5860) 2021-04-22 13:50:24 +03:00
Batuhan TaskayaandRuslan Kuprieiev 68897aa0de update: only update the missing hashes with --to-remote (#5773)
* objects: merge transfer() with stage()

* objects: drop streamfile

* objects: preserve the signature of get_file_hash

* objects: override hash_jobs if jobs= passed

* objects: support no progress bar directive

* tests: spy on _get_file_hash

* objects: update=... directive

* update: only transfer missing caches with --to-remote

* tests: remove local_remote from the primary test (due to it moves)

* objects: add move=True/False option

* objects: implement _get_file_obj on top of get_file_hash

* objects: use threading on save()

* objects: pass jobs= on to-cache as well

* objects: use in place tqdm

* objects: check for FileExistsError

* Update dvc/objects/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-04-19 18:09:35 +03:00
Ruslan Kuprieiev 2db55032c4 objects: file: remove unused load() method (#5610) 2021-03-12 05:35:48 +02:00
Ruslan Kuprieiev 29ad87ebe2 objects: split into separate files (#5609) 2021-03-12 04:12:46 +02:00