For the longest time we've had `repo.lock`, but were only using it from
CLI when running specific command, instead of simply properly taking it
on API operations. This patch takes care of that.
Fixes#2039
Required by #2012 to be able to fetch data through the API without
interuption from updater.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
- Uses a formatter for coloring output
- Move the `progress_aware` decorator logic to the logger itself
- Refactor modules so each one uses their own instance of a logger
Close#1753
* move man doc links to descriptions, fix some help copies
* minor init force message fix
* deal with ugly inheritance in lock base
* remove unnecessary super calls
* add return back to python lock/unlock run functions
1) split Project's methods into separate files;
2) rename Project into Repo(similar to gitpython's Repo)
Signed-off-by: Ruslan Kuprieiev <[email protected]>
In scenarios with millions of files, late shorcutting of
debug msgs results in massive overhead. In order to avoid
that, just shortcut logger messages ASAP if verbose mode
is not enabled.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
We already have 'sync' command, but users report it to be confusing
when both local and remote files exist, so lets introduce two direct
commands to implicitly push and pull from cloud.
Fixes#157.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
This will make the structure of dvc more linear
making it easier to read. Plus, these changes
are needed in preparation for libdvc, as they
move all cmdline parsing to one common place.
This patch also refactors our settings system
also make it more linear and remove abstraction
levels that are not needed and only cause confusion.
There is a lot more to be done in preparation for
dvc lib and RFC overall, but this a first step
in that long journey.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
It replaces POOL_SIZE constant and by default equals to the
number of cpus available.
map_progress is a helper for 'sync' and 'import' commands to
run in a multi-threaded mode.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
This patch introduces DataCloud class that used config to
determine proper cloud class(DataCloudAWS and DataCloudGCP
are awailable) to use in "data_sync" and "remove" commands.
This patch focuses on introducing the common class itself
saving original code and definetly needs some RFC later.
It also adds simple DataCloudLOCAL class that uses directory
as a StoragePath and cp as a sync method. Which makes testing
so much easier and covers local scenarios.
And it also introduces ability to run data sync in parallel
by collecting files first and then running sync in a number
of threads.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
These are cmdline shortcuts to change dvc's verbosity
without fiddling around with loglevel in dvc.conf.
Both are added and handled as a common options for
all dvc commands in base cmd class.