Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-07 07:18:27 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,611 Commits 7 Branches 579 Tags
0.14.2
Commit Graph
300 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 2363057165 metrics: add '-t|--type' and '-x|--xpath' options 
Fixes #914

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-23 00:34:03 +03:00
Ruslan Kuprieiev 9e576cd55b metrics: properly handle explicitly specified file 
Related to #865

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-22 21:34:03 +03:00
Ruslan Kuprieiev 6a6a4fe0b4 test: test unsupported chemes 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 23:05:41 +03:00
Ruslan Kuprieiev 07fa6442b8 test: add dvc status simple test 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 22:56:53 +03:00
Ruslan Kuprieiev 587cc08515 tests: test unsupported remote 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 22:56:53 +03:00
Ruslan Kuprieiev e598b0b4c7 test: repro: add test for failed stage in CLI 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:36:32 +03:00
Ruslan Kuprieiev 3be898797a test: remote: add test for failed write 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:36:32 +03:00
Ruslan Kuprieiev 41ed1bca94 tests: test dvc pipeline show --outs 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:24:55 +03:00
Ruslan Kuprieiev 90781900e2 tests: test CLI dvc metrics add 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:24:55 +03:00
Ruslan Kuprieiev bc4a5d78e3 tests: data cloud: don't forget to test fetch 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:24:55 +03:00
Ruslan Kuprieiev 637fc29b79 test: config: add test for failed write 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:24:55 +03:00
Ruslan Kuprieiev 0a3a124756 test: cover "Invalid option value" for config 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:24:55 +03:00
Ruslan Kuprieiev ff51b1ae04 test: add test for local config 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:24:55 +03:00
Ruslan Kuprieiev 9557c56a7e tests: add _find_root test 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 08:24:55 +03:00
Ruslan Kuprieiev ebd8468b1c tests: test CmdBase._set_loglevel 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-19 07:05:28 +03:00
Ruslan Kuprieiev 3f312a83c5 metrics: Show a proper message if repository does not have metrics and show is called 
Fixes #905

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-18 13:58:45 +03:00
Ruslan Kuprieiev 6c7bf390c6 metrics: check that specified path is a text file and not a directory or binary file 
Fixes #911

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-18 12:25:11 +03:00
Ruslan Kuprieiev 5565ece1f8 metrics: add support for csv and hcsv 
Fixes #910
Fixes #909

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-18 12:15:59 +03:00
Ruslan Kuprieiev ebecc9996b tests: use 0.3 sec as a max startup time 2018-07-18 11:39:24 +03:00
Ruslan Kuprieiev 3410f8048a dvc: optimize imports to speedup startup time 
Speeds up `time dvc` from 1.2 seconds to <0.2 seconds.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-18 11:20:05 +03:00
Ruslan Kuprieiev f54428d5ed test: properly check for hadoop availability 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-18 10:12:49 +03:00
Ivan Shcheklein 2d9b062fc4 minor: remove a temporary file in tests 2018-07-17 16:00:30 -07:00
Ruslan Kuprieiev 870d98bc18 test: fix expected number of stages 2018-07-17 15:57:31 +03:00
Ruslan Kuprieiev dc5fec6a75 test: remove unused add() call 2018-07-17 15:47:38 +03:00
Ruslan Kuprieiev 805eb908d0 import: support external urls 
Fixes #891

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-17 12:00:12 +03:00
Ruslan Kuprieiev f1a28c3812 dvc: support dvc add for external files 
Fixes #890

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-16 16:34:16 +03:00
Ruslan Kuprieiev 6d120f474e dvc: handle double dvc init 
Fixes #889

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-16 13:43:46 +03:00
Ruslan Kuprieiev fef8ea4602 dvc: add confirmation for dvc destroy 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-15 19:18:01 +03:00
Ruslan Kuprieiev 71d9fb44fc dvc: add dvc pipeline show 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-14 10:49:32 +03:00
Ruslan Kuprieiev c636581699 test: gc: catch OSError 2018-07-12 10:41:34 +03:00
Ruslan Kuprieiev 4668c10e6e test: gs: catch WinError 2018-07-12 10:32:34 +03:00
Ruslan Kuprieiev 9be161de54 test: use IOError instead of FileNotFoundError 2018-07-12 10:25:40 +03:00
Ruslan Kuprieiev 79826b21a4 test: gcp: handle FileNotFoundError 2018-07-12 10:14:15 +03:00
Ruslan Kuprieiev f476143c52 test: check if we need to test RemoteAzure with _should_test_azure 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-12 09:13:07 +03:00
Clemens Wolff 7897dd8c45 Make tests more verbose 
This helps verify in the CI that the environment variables for the
integration tests are set up correctly since we can see exactly which
tests are being executed and which ones are being skipped.
 2018-07-10 16:44:29 -04:00
Clemens Wolff 97dfc88647 Flag skipped tests during test run 2018-07-10 16:44:28 -04:00
Clemens Wolff fcbc6d115c Wire-up Azure in TestDataCloud 2018-07-10 16:33:52 -04:00
Clemens Wolff 1b1ca68b64 Fix inspections 2018-07-10 16:33:52 -04:00
Clemens Wolff 9724bd94aa Replace os.system with subprocess.check_output 
It is generally recommended to use the subprocess module instead of the
os module to run subshell commands given since the former makes it
easier to comunicate with the process, has an improved security model,
etc. More information is available in [1].

Note that this change uses `check_output` instead of `check_call` to
prevent the command output from being printed to stdout.

This change also introduces two additional enhancements:
- Skip the hadoop test if the hadoop binary is not available.
- Skip the ssh test if the ssh binary is not available.

[1] https://docs.python.org/3/library/subprocess.html#replacing-os-system
 2018-07-10 16:33:14 -04:00
Clemens Wolff 581132bbe0 Fix environment variable name 2018-07-10 13:54:49 -04:00
Clemens Wolff 6882a781a4 Add Azure Blob Storage remote 
This change adds a new cloud remote based on Azure Blob Storage [1] and
the azure-storage-blob SDK [2].

The implementation closely follows the patterns laid out by the existing
cloud remotes (AWS and GCP). Notable differences include:

- No use of a lock in the progress-bar callback and no requirement to
  look up total transfer size ahead of time: the Azure Storage SDK
  provides the total transfer bytes and current transfer offset in its
  progress callback.

- When `exists` is called, check the existence of every blob instead of
  listing the entire container. Container list operations can be
  expensive so unless the number of checked files is very large, making
  separate requests is likely going to be more efficient.

- For increased flexibility, access credentials for the Azure Storage
  backend can be provided either connection-string style via the remote
  URL or via environment variables. The Azure Storage backend can be
  emulated via Azurite [3].

[1] https://azure.microsoft.com/en-us/services/storage/blobs/
[2] https://github.com/Azure/azure-storage-python
[3] https://github.com/Azure/azurite

This change was created together with @EricSchles.
 2018-07-09 14:47:40 -04:00
Ruslan Kuprieiev b952e599bf debug: add a few more sleep()s to test_repro 2018-07-03 02:43:01 +03:00
Ruslan Kuprieiev 71d03b5a43 test: add sleep() calls 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-03 01:33:13 +03:00
Ruslan Kuprieiev f471f7714b dvc: uptimize add/checkout/state 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-03 00:33:05 +03:00
Ruslan Kuprieiev d37828c4e3 state: save collected dir to cache 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-02 20:41:35 +03:00
Ruslan Kuprieiev 67e0c24ced tests: add test_updater 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 23:29:07 +03:00
Ruslan Kuprieiev f634bc1916 test: remote: add basic test for remove() and checkout() 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 22:34:48 +03:00
Ruslan Kuprieiev f274a745a0 tests: checkout: test CLI for single files 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 21:02:43 +03:00
Ruslan Kuprieiev e8818ef67a test: add basic test from dvc import cmd 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 21:02:43 +03:00
Ruslan Kuprieiev 449d4880d1 tests: add test for double install 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 21:02:43 +03:00