Files
dvc/dvc/scheme.py
T
Tomás Farías SantanaandRuslan Kuprieiev ca2e6a004c Support for WebHDFS (#4747)
* Add support for WebHDFS

* Style and linting fixes

* Implement abstract methods and disable no-member pylint errors

* Add init unit test

* Yield client and fix redefinition of hdfs

* Get dynamic hdfs Docker port

* Add support for open in WebHDFSTree

* Remove hdfscli_config fallbacks in WebHDFSTree

* Use port 50070 for webhdfs

* WebHDFSTree supports http URL and proper walking

* Fix failing unit test after refactoring

* Add overwrite argument to upload

* Add remaining func tests for webhdfs

* Remove print statements

* Allow dynamic attributes in pylint for hdfs library

* Comment error on instantiating HDFS client from config

* Remove depth argument from call to walk

* Update dvc/tree/webhdfs.py

* Remove unnecessary deque

* Check exception messages hdfs client failure from config

* Return client directly

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Split long line

* Add size to HashInfo

* Implement move via hdfs.rename

* Add missing line

* Make parent directories before moving file in WebHDFS

* Add total to progress bar

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-11-17 20:02:02 +02:00

15 lines
263 B
Python

class Schemes:
SSH = "ssh"
HDFS = "hdfs"
WEBHDFS = "webhdfs"
S3 = "s3"
AZURE = "azure"
HTTP = "http"
HTTPS = "https"
GS = "gs"
GDRIVE = "gdrive"
LOCAL = "local"
OSS = "oss"
WEBDAV = "webdav"
WEBDAVS = "webdavs"