Files
dvc/tests/unit/remote/test_oss.py
T
GaoandRuslan Kuprieiev 93f36d9351 oss: migrate to ossfs (#6307)
* Migrate to ossfs

fix 6177

* Add verification to object stored on oss

* Change the oss remote's dependency.

pass all of the tests

* Performance optimaztion and ossfs version update

* Remove the argument to set the resumable get/put

1. Experiments shows resumable get/put has little influnence on performance.
2. Just use resumable ones as default

* Use a new api for performance optimization

* Without filesize progress bar version

* With progress bar version

* Bump and add comments

1. Bump ossfs from 2021.6.2 to 2021.7.0
2. add some comments to the verification trigger

* Change the name of kwrags following parent class.

* Bump to new ossfs and rewrite upload_fobj

Because some problem with oss-emulater
1. bump to ossfs 2021.7.1
2. rewrite upload_fobj

* Optimization on rm_files

* Some optimization follow the old version.

* Update dvc/fs/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/objects/db/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* bump into ossfs 2021.7.2

* Bump into ossfs 2021.7.3OC

* Switch to real_oss_server

* Switch to a real oss server

* Use the oss bucket located in US.

* Remove some emulator related code

* Add envs to ci

* remove some oss pytest XFAIL

Because we didn't use emulator, we can remove some pytest XFAILs

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-07-13 15:27:34 +03:00

22 lines
564 B
Python

from dvc.fs.oss import OSSFileSystem
bucket_name = "bucket-name"
endpoint = "endpoint"
key_id = "Fq2UVErCz4I6tq"
key_secret = "Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsu"
def test_init(dvc):
prefix = "some/prefix"
url = f"oss://{bucket_name}/{prefix}"
config = {
"url": url,
"oss_key_id": key_id,
"oss_key_secret": key_secret,
"oss_endpoint": endpoint,
}
fs = OSSFileSystem(**config)
assert fs.fs._endpoint == endpoint
assert fs.fs._auth.id == key_id
assert fs.fs._auth.secret == key_secret