1) use io for StringIO in python3
2) use builtins instead of __builtin__
3) use StringIO instead of BytesIO when mocking aws config
Othervise this test fails on python3.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
This patch introduces DataCloud class that used config to
determine proper cloud class(DataCloudAWS and DataCloudGCP
are awailable) to use in "data_sync" and "remove" commands.
This patch focuses on introducing the common class itself
saving original code and definetly needs some RFC later.
It also adds simple DataCloudLOCAL class that uses directory
as a StoragePath and cp as a sync method. Which makes testing
so much easier and covers local scenarios.
And it also introduces ability to run data sync in parallel
by collecting files first and then running sync in a number
of threads.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Dmitry Petrov
Ruslan Kuprieiev
Earl Hathaway