Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-10 23:10:42 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,724 Commits 7 Branches 579 Tags
b294bca383547f92181b606cd7b45f17fd39c5e2
Commit Graph
17 Commits
Author SHA1 Message Date
Saugat Pachhai b9000ed926 pre-commit: update black to 21.5b1 (#6032) 
* pre-commit: update black to 21.5b1

also updated others, but black is more noticeable

* remove magic trailing comma
 2021-05-18 15:51:15 +00:00
Batuhan Taskaya d4a3df73df s3: fsspec migration (#5683) 
* s3: migrate to s3fs

* s3: strip etag

* s3: disable instance cache

* tests: use the proper remote:// form in test_checkout

* tests: move/drop some tests

* setup: depend on s3fs

* tests: normalize s3 tests with some cleanups

* Use _download/_upload with old implementation

* fsspec: cache .exists() results

* setup: pin the boto version

* setup: install s3fs[boto3]

* s3: proper exception management

* s3: config improvements

* setup: reference without quotes

* setup: install boto separately

* setup: install moto server

* s3: mocking adjustments for moto_server

* s3: mark some fixtures with scope=session

* s3: s3_fake_creds_file scopes to func

* fsspec: add some docs regarding exists() optimization

* fsspec: exists() fallback

* fsspec: revert exists() optimization

* tests: ssl_verify => verify

* setup: temporarily fetch from master

* s3: add boto as a requirement

* tests: do an extra check regarding race conditions

* tests: use motoserver docker image

* tests: wrap config deletion with try/finally

* tests: disable s3 by default

* setup: upgrade fsspec
 2021-05-05 11:36:10 +03:00
Ruslan Kuprieiev e0842ce01e dvc: rename cache -> odb (#5481) 2021-02-16 22:49:25 +02:00
Ruslan Kuprieiev d038bca05e tests: add basic [web]hdfs mockers (#5279) 2021-01-19 20:03:16 +02:00
Saugat Pachhai dcba0e8da4 pre-commit: update hooks (#4964) 2020-11-25 17:57:32 +05:45
Tomás Farías SantanaandRuslan Kuprieiev ca2e6a004c Support for WebHDFS (#4747) 
* Add support for WebHDFS

* Style and linting fixes

* Implement abstract methods and disable no-member pylint errors

* Add init unit test

* Yield client and fix redefinition of hdfs

* Get dynamic hdfs Docker port

* Add support for open in WebHDFSTree

* Remove hdfscli_config fallbacks in WebHDFSTree

* Use port 50070 for webhdfs

* WebHDFSTree supports http URL and proper walking

* Fix failing unit test after refactoring

* Add overwrite argument to upload

* Add remaining func tests for webhdfs

* Remove print statements

* Allow dynamic attributes in pylint for hdfs library

* Comment error on instantiating HDFS client from config

* Remove depth argument from call to walk

* Update dvc/tree/webhdfs.py

* Remove unnecessary deque

* Check exception messages hdfs client failure from config

* Return client directly

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Split long line

* Add size to HashInfo

* Implement move via hdfs.rename

* Add missing line

* Make parent directories before moving file in WebHDFS

* Add total to progress bar

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-11-17 20:02:02 +02:00
Saugat Pachhai 72eb4a5737 Add test server and tests for webdav (#4827) 
* Add test server and tests for webdav

* Fix webdav unittests
 2020-11-12 13:52:46 +05:45
Juliano Efson SalesandRuslan Kuprieiev 052b15a2a8 tests/remote: adding tests for real oss (#4735) 
* tests/remote: adding tests for real oss

the new test suggests #4633 is a bug in the emulator.

* Update tests/func/test_data_cloud.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update tests/remotes/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* tests: correcting small lint issue

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-10-18 16:21:51 +03:00
Paweł Redzyński b31bd1cb30 github actions tests: initial (#4469) 
* github actions tests: initial

* build: tests: add codecov action
 2020-09-22 23:05:06 +03:00
Saugat Pachhai 1cfccef22e update flake8 to latest version (#4403) 2020-08-17 14:26:10 +05:45
Ruslan Kuprieiev 102187be58 tests: check if docker is present (#4263) 
https://travis-ci.com/github/iterative/dvc/jobs/364061489#L3623
 2020-07-23 05:55:46 +03:00
Ruslan Kuprieiev 4c9245a3b3 tests: remove real SSH (#4200) 
* tests: remove real SSH

No longer needed, all tests are using mocked ssh already.

* travis: don't setup ssh
 2020-07-14 13:55:14 +03:00
Ruslan Kuprieiev 375998c664 tests: hdfs: use hadoop/docker fixtures (#4183) 
Fixed #4054
 2020-07-10 21:38:48 +03:00
Ruslan Kuprieiev 0899b277c0 tests: launch oss/azure containers from fixtures (#4178) 
* tests: launch oss/azure containers from fixtures

Related to #4054

* ci: hdfs: remove unneeded fix

We are using pyarrow 0.17.0 already.
 2020-07-08 19:54:17 +03:00
Ruslan Kuprieiev 30212f886b tests: create remote fixture (#4163) 
Usually we need it to parametrize tests with, so there is no
reason to duplicate the code. I've left `local_remote` fixture
just because it is so convenient and often used in a lot of tests.
 2020-07-03 22:30:56 +03:00
Ruslan Kuprieiev 07da9f1e5c tests: remotes: use TmpDir-like fixtures (#4140) 2020-07-02 04:46:40 +03:00
Ruslan Kuprieiev 51a8c782ae tests: create remote fixtures (#4063) 
* tests: use remote fixtures

* tests: use remote fixtures for data cloud tests

* tests: treat remotes as tmp dirs
 2020-06-18 01:36:13 +03:00