gdrive: add workaround for downloading empty files (#4287)

* [WIP] gdrive: add workaround for downloading empty files

Better to solve it on PyDrive's side:
https://github.com/iterative/dvc/issues/4286

Fixes #4286

* deps: limit flatten_json

```
pylint...................................................................Failed
- hook id: pylint
- exit code: 2

Registered custom plugin. Some checks will be disabled for tests.
************* Module dvc.command.experiments
dvc/command/experiments.py:14:4: E0401: Unable to import 'flatten_json'
(import-error)
dvc/command/experiments.py:39:4: E0401: Unable to import 'flatten_json'
(import-error)

-----------------------------------
Your code has been rated at 9.99/10

Registered custom plugin. Some checks will be disabled for tests.
************* Module dvc.utils.diff
dvc/utils/diff.py:38:8: E0401: Unable to import 'flatten_json'
(import-error)
************* Module dvc.command.metrics
dvc/command/metrics.py:17:4: E0401: Unable to import 'flatten_json'
(import-error)

-----------------------------------
```
This commit is contained in:
Ruslan Kuprieiev
2020-07-29 01:25:06 +03:00
committed by GitHub
parent f4595a02b2
commit 5d1c1c418c
+10 -1
View File
@@ -381,6 +381,9 @@ class GDriveTree(BaseTree):
# it does not create a file on the remote
gdrive_file = self._drive.CreateFile(param)
gdrive_file.FetchMetadata(fields="fileSize")
size = gdrive_file["fileSize"]
with Tqdm(
desc=progress_desc,
disable=no_progress_bar,
@@ -388,7 +391,13 @@ class GDriveTree(BaseTree):
# explicit `bar_format` as `total` will be set by `update_to`
bar_format=Tqdm.BAR_FMT_DEFAULT,
) as pbar:
gdrive_file.GetContentFile(to_file, callback=pbar.update_to)
if size:
gdrive_file.GetContentFile(to_file, callback=pbar.update_to)
else:
# PyDrive doesn't like downloading empty files
# https://github.com/iterative/dvc/issues/4286
with open(to_file, "w"):
pass
@contextmanager
@_gdrive_retry