mirror of
https://github.com/Stone-Red-Code/dvc.git
synced 2026-11-02 07:18:25 +01:00
Tomás Farías SantanaandRuslan Kuprieiev
ca2e6a004c
* Add support for WebHDFS * Style and linting fixes * Implement abstract methods and disable no-member pylint errors * Add init unit test * Yield client and fix redefinition of hdfs * Get dynamic hdfs Docker port * Add support for open in WebHDFSTree * Remove hdfscli_config fallbacks in WebHDFSTree * Use port 50070 for webhdfs * WebHDFSTree supports http URL and proper walking * Fix failing unit test after refactoring * Add overwrite argument to upload * Add remaining func tests for webhdfs * Remove print statements * Allow dynamic attributes in pylint for hdfs library * Comment error on instantiating HDFS client from config * Remove depth argument from call to walk * Update dvc/tree/webhdfs.py * Remove unnecessary deque * Check exception messages hdfs client failure from config * Return client directly Co-authored-by: Ruslan Kuprieiev <[email protected]> * Split long line * Add size to HashInfo * Implement move via hdfs.rename * Add missing line * Make parent directories before moving file in WebHDFS * Add total to progress bar Co-authored-by: Ruslan Kuprieiev <[email protected]>
23 lines
538 B
YAML
23 lines
538 B
YAML
---
|
|
version: '3.2'
|
|
services:
|
|
azurite:
|
|
image: mcr.microsoft.com/azure-storage/azurite:3.3.0-preview
|
|
ports:
|
|
- "10000"
|
|
oss:
|
|
image: rkuprieiev/oss-emulator
|
|
ports:
|
|
- "8880"
|
|
hdfs:
|
|
image: rkuprieiev/docker-hdfs
|
|
hostname: localhost
|
|
ports:
|
|
- "8020"
|
|
# NOTE: having this port as dynamic one will require modifying
|
|
# `dfs.datanode.address` in `hdfs-site.xml` and probably something
|
|
# else, so using default one for now.
|
|
- "50010:50010"
|
|
- "50075:50075"
|
|
- "50070"
|