Commit Graph
6379 Commits
Author SHA1 Message Date
Batuhan Taskaya f7d8761734 gdrive: use the normal file backend for saved credentials (#5554)
* gdrive: sync with pydrive's save backend changes

* bump pydrive version

* better temporary credentials managements for service accounts

* proper temp var management
2021-03-08 16:30:05 +02:00
Ruslan Kuprieiev 70c445d1ee utils: makedirs: ignore chmod errors (#5570)
Same as with `protect()`, these errors might happen if fs is immutable
or doesn't support chmod()-ing. We can safely ignore these.

Fixes #5552
2021-03-08 16:28:02 +02:00
Ruslan Kuprieiev 47bcb02ef1 fs: git: use pygit2 (#5559) 2021-03-06 02:54:13 +02:00
Ruslan Kuprieiev 4a75067f4c github: template: use dvc doctor instead of dvc version 2021-03-05 16:08:46 +02:00
Ruslan Kuprieiev 70f4c44ec6 dvc: bump to 2.0.3 2.0.3 2021-03-05 13:08:22 +02:00
af37527560 setup: rename azure.indentity to azure-identity (#5548)
* simple like that

* Update dvc/version.py

Co-authored-by: Marc-Antoine Belanger <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-03-05 01:43:32 +02:00
Ruslan Kuprieiev 71f8ffe13d dvc: bump to 2.0.2 2.0.2 2021-03-05 01:04:00 +02:00
Ruslan Kuprieiev 06aad3e2b3 dependency: repo: try to fetch before staging (#5549)
Related to https://github.com/iterative/dvc-bench/issues/234
2021-03-05 01:03:33 +02:00
Peter Rowlands (변기호) 1e7df0c27e dulwich: update to 0.20.20 (#5545) 2021-03-04 12:37:19 +02:00
Peter Rowlands (변기호) ce2d7c65db repro/exp run: use git status when reminding to git-track files (#5544)
* repro: skip DVC deps in SCM track stage

(DVC deps will be tracked when their own stage is repro'd)

* scm: use git status when reminding to git-track files

* sort names in reminder instead of random set order

* update unit test
2021-03-04 17:48:12 +09:00
Peter Rowlands (변기호) 2075a99233 exp diff: fix unresolved experiment names bug (#5543) 2021-03-04 15:39:21 +09:00
Ruslan Kuprieiev 98a1a92fa3 snap: debug failing tests 2021-03-03 23:11:19 +02:00
Ruslan Kuprieiev 16304b544f dvc: bump to 2.0.1 2.0.1 2021-03-03 17:38:26 +02:00
Ruslan Kuprieiev 803a100f32 setup: move google-cloud-storage to test requirements (#5540) 2021-03-03 17:11:30 +02:00
Ruslan Kuprieiev c3a0e8733e dvc: bump to 2.0.0 2.0.0 2021-03-03 12:21:03 +02:00
Peter Rowlands (변기호) 1542025afa exp: bash tab completion (#5532)
* exp list: add --names-only option

* completion: add exp name completion

* dvc exp: use exp name completion

* exp: fix bash completion
2021-03-03 15:46:11 +09:00
Batuhan Taskaya 8f6061f8cf update: clear error message for updating repo imports (#5520) 2.0.0a4 2021-03-02 20:34:59 +02:00
Paweł Redzyński 4a8cb806d2 live integration: get rid of dvc from live (#5466)
* live integration: get rid of dvc from live

* live: remove from api, add test for html generation during run

* stage: run: convert monitors to class context managers

* run: make monitors run in single thread

* run: move monitor logic out of run
2021-03-03 00:45:35 +09:00
Ruslan Kuprieiev e385e59a09 dvc: bump to 2.0.0a4 2021-03-02 13:54:16 +02:00
Batuhan Taskaya 2982e98c81 gs: Use rm() instead of delete() calls (#5535)
delete is a direct alias to rm(), though apparently it is kind of
broken on the latest release of gcsfs. This doesn't change any behavior
but only uses rm() directly instead of the given alias.
2021-03-02 13:53:44 +02:00
Batuhan Taskaya 3ebccda62c gs: use etag for the checksum parameter (#5534)
Switch from `md5` to `etag` as the checksum parameter. Due to `etag`
is not preserved while doing server side copies, this patch drops support
for external gs outputs.
2021-03-02 12:43:27 +02:00
Ruslan Kuprieiev 96092e5e27 dvc: bump to 2.0.0a3 2.0.0a3 2021-03-01 18:20:35 +02:00
Batuhan Taskaya b65dfe04f9 gs: Migrate Google Storage to FSSpec (#5526)
* azure: generalize fsspec functionality into FSSpecWrapper

* gs: migrate google storage to fsspec

* tests: disable empty_dir tests

* setup: add gcsfs to requirements

* gs: decode md5Hash into md5

* gs: speed up downloads via increasing chunk size
2021-03-01 17:15:31 +02:00
Paweł Redzyński 92b95bae9b plots: ignore binary plot files (#5523) 2021-03-01 15:00:42 +02:00
Ruslan Kuprieiev f2e0059392 dvc: bump to 2.0.0a2 2.0.0a2 2021-02-26 13:35:28 +02:00
Batuhan Taskaya a22101e0a6 azure: add account key to the config schema (#5522) 2021-02-25 19:53:47 +02:00
Peter Rowlands (변기호) fed6179992 exp run: if target is provided, only checkout target and dependencies (#5519)
* tests: test that only target and its deps are checked out on exp run

* exp run: if target is provided, only checkout target + deps
2021-02-25 15:12:27 +09:00
Peter Rowlands (변기호) b3a32809e9 exp show: fix bug where already collected exp results could be dropped (#5517) 2021-02-25 14:41:56 +09:00
Batuhan Taskaya e670644634 azure: support service principal in error messages as login_method (#5514) 2021-02-24 10:50:17 +02:00
Ruslan Kuprieiev a26f2995ef dir_info: only build trie when needed (#5512) 2021-02-24 09:15:20 +02:00
Ruslan Kuprieiev 502f0eecc9 tests: temporarily skip dvclive tests (#5513) 2021-02-24 08:52:33 +02:00
Ruslan Kuprieiev 8f568ab197 state: use diskcache (#5505)
* state: only load when needed

* state: don't use separate load/dump

* checkout: group link removal logic together

* state: use diskcache.Cache

* staging: use pbar and save to state

* state: don't pass the whole repo to __init__

* staging: explicitly pass state object

* fs: get rid of odb property

* state: don't close the connection to cache

* repo: close: close state
2021-02-23 20:51:34 +02:00
Ruslan Kuprieiev 2e01ce1974 dvc: bump to 2.0.0a1 2.0.0a1 2021-02-23 20:48:42 +02:00
Batuhan Taskaya d587d41d6c utils: proper file interface for HashedStreamReader (#5508) 2021-02-23 02:47:36 +02:00
Batuhan Taskaya 27c2401f59 azure: transfer bigger chunks of data (#5504) 2021-02-22 14:08:26 +02:00
Batuhan Taskaya 174621696d azure: migrate AzureFileSystem to fsspec (#5463)
* azure: migrate AzureFileSystem to fsspec

* client => fs

* principal auth

* better login error handling
2021-02-22 12:05:15 +02:00
Batuhan Taskaya e1a193f00b update: forbid updating repo-imports with --to-remote (#5475) 2021-02-22 11:55:00 +02:00
Saugat PachhaiandDave Berenbaum cccead3441 stage:list: update output and help text (#5499)
* stage:list: update output and help text

* Replaced 'Produces' with 'Outputs'
* Replaced 'Stage depends on' with just 'Depends on'
* Changes help text of --names-only and --fail

* Update dvc/command/stage.py

Co-authored-by: Dave Berenbaum <[email protected]>

* s/Generates/Reports

Co-authored-by: Dave Berenbaum <[email protected]>
2021-02-20 13:21:39 +05:45
Batuhan Taskaya 8102ede5cc add, import-url: always resolve the output in the local path for --to-remote (#5473)
* add, import-url: always resolve the output in the local path for --to-remote

* windows

* use the proper fixture
2021-02-19 15:22:59 +02:00
Saugat Pachhai 14944dd9e0 Minor refactoring stage:list command (#5498)
Moving text truncation to utils and adding tests for it
2021-02-19 18:27:37 +05:45
Paweł Redzyński 63f32936b2 live integration: turn on tests (#5487) 2021-02-19 09:25:28 +01:00
Peter Rowlands (변기호) 7d13d24a1f pygit/dulwich: implement create branch (#5497)
* pygit: implement create branch

* dulwich: implement create branch
2021-02-19 17:18:15 +09:00
Peter Rowlands (변기호) 9cb8187cb3 pygit: implement resolve_commit (#5496)
* git: return backend-agnostic object for resolve_commit

* use int timestamps instead of datetime

* make branch_revs Git method instead of backend

* pygit: implement resolve_commit
2021-02-19 17:00:22 +09:00
Peter Rowlands (변기호) 98590779eb pygit: implement get_refs_containing (#5495)
* git: make is_sha a Git class method instead of backend function

* pygit: implement get_refs_containing

* use merge base instead of parents
2021-02-19 16:42:04 +09:00
Batuhan Taskaya 9ae8bcdf52 gdrive: support service auth with json (#5485)
* gdrive: support service auth with json

* retry logic
2021-02-18 17:10:35 +02:00
Paweł Redzyński 14958ff85f plots: fix bug preventing usage from being displayed (#5492) 2021-02-18 14:24:47 +01:00
Peter Rowlands (변기호) 9909ee85a1 git: implement dulwich add --update (#5491)
* dulwich: implement add with --update

* add git backend test for add()

* fix list comprehension in dulwich status()

* allow adding single string path instead of list

* handle windows paths in add(update=...)
2021-02-18 17:56:56 +09:00
Saugat Pachhai 45477a2a81 dvc.lock: raise error if the file is gitignored (#5484) 2021-02-18 03:08:00 +02:00
Ruslan Kuprieiev 282e83c481 tests: temporarily exclude gdrivefs from extras (#5483)
When trying to publish to pypi, we get:

```
Invalid value for requires_dist. Error: Can't have direct dependency:
"gdrivefs @ git+https://github.com/intake/gdrivefs.git ; extra ==
'tests'"
```
2.0.0a0
2021-02-17 05:49:34 +02:00
Ruslan Kuprieiev 51988475a4 gha: packages: update for 2.0 2021-02-17 04:38:45 +02:00