Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-08 07:18:27 +01:00
Code Issues Packages Projects Releases Wiki Activity
1,494 Commits 7 Branches 579 Tags
1bcd7408ec3b09f78bfb540dd48dc3609459e4e4
Commit Graph
1494 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev 1bcd7408ec Merge pull request #868 from CatalystCode/azure-blob-storage-remote 
Add Azure Blob Storage remote
 2018-07-09 23:01:07 +03:00
Clemens Wolff 6882a781a4 Add Azure Blob Storage remote 
This change adds a new cloud remote based on Azure Blob Storage [1] and
the azure-storage-blob SDK [2].

The implementation closely follows the patterns laid out by the existing
cloud remotes (AWS and GCP). Notable differences include:

- No use of a lock in the progress-bar callback and no requirement to
  look up total transfer size ahead of time: the Azure Storage SDK
  provides the total transfer bytes and current transfer offset in its
  progress callback.

- When `exists` is called, check the existence of every blob instead of
  listing the entire container. Container list operations can be
  expensive so unless the number of checked files is very large, making
  separate requests is likely going to be more efficient.

- For increased flexibility, access credentials for the Azure Storage
  backend can be provided either connection-string style via the remote
  URL or via environment variables. The Azure Storage backend can be
  emulated via Azurite [3].

[1] https://azure.microsoft.com/en-us/services/storage/blobs/
[2] https://github.com/Azure/azure-storage-python
[3] https://github.com/Azure/azurite

This change was created together with @EricSchles.
 2018-07-09 14:47:40 -04:00
Ruslan Kuprieiev 50ea36e740 dvc: bump to 0.10.2 0.10.2 2018-07-08 01:13:21 +03:00
Ruslan Kuprieiev c7ed95f5ba Merge pull request #867 from efiop/master 
metrics bugfixes
 2018-07-08 01:11:03 +03:00
Ruslan Kuprieiev 5d3a009886 metrics: require ply>=3.9 to avoid bug on binary windows package 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-08 01:02:26 +03:00
Ruslan Kuprieiev 6683f13b4e metric: print error if unable to read 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-08 01:02:26 +03:00
Ruslan Kuprieiev 3f73ef057a dvc: bump to 0.10.1 0.10.1 2018-07-07 03:43:16 +03:00
Ruslan Kuprieiev 961bf8b34c Merge pull request #866 from efiop/master 
metrics: show: don't print branch if all_branches=False
 2018-07-07 03:41:39 +03:00
Ruslan Kuprieiev 11de88fcd4 metrics: show: don't print branch if all_branches=False 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-07 03:31:42 +03:00
Ivan Shcheklein e83dd1c972 fix README tutorial link 2018-07-05 20:51:25 -07:00
Ruslan Kuprieiev 62cd68af77 Merge pull request #864 from efiop/master 
updater: don't forget to convert to int
 2018-07-06 06:31:45 +03:00
Ruslan Kuprieiev 715cc717aa updater: don't forget to convert to int 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-06 05:30:14 +03:00
Ruslan Kuprieiev d5c015ff63 dvc: bump to 0.10.0 0.10.0 2018-07-06 04:56:30 +03:00
Ruslan Kuprieiev db374b6553 Merge pull request #863 from efiop/master 
brancher: don't checkout if no branches is specified
 2018-07-06 04:12:05 +03:00
Ruslan Kuprieiev 01efbac5a3 brancher: don't checkout if no branches is specified 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-06 04:05:27 +03:00
Ruslan Kuprieiev 0f940f9167 appveyor: pull images before deploy 2018-07-06 03:39:47 +03:00
Ruslan Kuprieiev e85368c8b8 Merge pull request #862 from efiop/master 
dvc: use heuristics to detect binary files
 2018-07-06 01:39:13 +03:00
Ruslan Kuprieiev fba1d8c181 dvc: use heuristics to detect binary files 
Fixes #859

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-06 01:32:14 +03:00
Ruslan Kuprieiev 6e82c6c748 Merge pull request #861 from efiop/master 
dvc: properly save symlinks
 2018-07-05 20:41:24 +03:00
Ruslan Kuprieiev 468f4beee7 dvc: properly save symlinks 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-05 20:00:58 +03:00
Ruslan Kuprieiev 08b3273fcc Merge pull request #860 from efiop/master 
dvc: don't forget to add state and state.lock to gitignore
 2018-07-05 16:59:24 +03:00
Ruslan Kuprieiev b7b07d393a dvc: don't forget to add state and state.lock to gitignore 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-05 16:48:23 +03:00
Ivan Shcheklein 2542ceca0d update README, documenation links, flow image 2018-07-04 23:11:49 -07:00
Ruslan Kuprieiev 9b032b72cb Merge pull request #858 from efiop/master 
docs: remove from the core repository
 2018-07-04 20:31:16 +03:00
Ruslan Kuprieiev 0a173c4731 docs: remove from the core repository 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-04 20:23:20 +03:00
Ruslan Kuprieiev 65b5dcb601 Merge pull request #854 from efiop/master 
link: properly iterate cache types
 2018-07-03 03:25:24 +03:00
Ruslan Kuprieiev 425e44bac8 link: properly iterate cache types 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-03 03:12:32 +03:00
Ruslan Kuprieiev b952e599bf debug: add a few more sleep()s to test_repro 2018-07-03 02:43:01 +03:00
Ruslan Kuprieiev adc34eb312 Merge pull request #851 from efiop/master 
Misc fixes
 2018-07-03 01:41:05 +03:00
Ruslan Kuprieiev 71d03b5a43 test: add sleep() calls 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-03 01:33:13 +03:00
Ruslan Kuprieiev db5cb125fb add: fix error msg 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-03 01:23:03 +03:00
Ruslan Kuprieiev 45fc17a074 Merge pull request #849 from efiop/master 
dvc: misc improvements
 2018-07-03 00:45:52 +03:00
Ruslan Kuprieiev e6bcdd3979 link: exclude unsupported link types after trying once 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-03 00:33:05 +03:00
Ruslan Kuprieiev f471f7714b dvc: uptimize add/checkout/state 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-03 00:33:05 +03:00
Ruslan Kuprieiev d37828c4e3 state: save collected dir to cache 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-02 20:41:35 +03:00
Ruslan Kuprieiev d9b1308447 state: store collected dir info 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-02 19:54:17 +03:00
Ruslan Kuprieiev ecf8045887 state: store cache state on per-repo basis 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-02 19:04:40 +03:00
Ruslan Kuprieiev 8d6813a112 checkout: use link_state properly 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-02 18:54:33 +03:00
Ruslan Kuprieiev 35d318f0f0 Merge pull request #847 from yustoris/feature/add-aws-remote-url 
RFP: Add endpoint url config for AWS
 2018-07-02 16:15:30 +03:00
kenji.yamauchi 0f8288fd9a Add endpoint url config for AWS 2018-07-02 12:44:33 +09:00
Ruslan Kuprieiev 5100e860fc Merge pull request #846 from efiop/master 
coverage: exclude 'pass' and 'NotImplementedError'
 2018-07-02 00:28:21 +03:00
Ruslan Kuprieiev 8e06d9d8ca coverage: exclude 'pass' and 'NotImplementedError' 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-02 00:19:41 +03:00
Ruslan Kuprieiev 52083c2bc5 Merge pull request #845 from efiop/master 
tests: add test_updater
 2018-07-01 23:44:24 +03:00
Ruslan Kuprieiev 67e0c24ced tests: add test_updater 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 23:29:07 +03:00
Ruslan Kuprieiev c6c6a0521c Merge pull request #844 from efiop/master 
Misc tests
 2018-07-01 22:59:44 +03:00
Ruslan Kuprieiev 459b21e0cf output: local: don't add/remove to/from gitignore if file is external 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 22:34:48 +03:00
Ruslan Kuprieiev f634bc1916 test: remote: add basic test for remove() and checkout() 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 22:34:48 +03:00
Ruslan Kuprieiev 3405a1ec47 remote: ssh: remove unused code 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 21:19:24 +03:00
Ruslan Kuprieiev aca3abff26 Merge pull request #843 from efiop/master 
Misc tests
 2018-07-01 21:09:52 +03:00
Ruslan Kuprieiev f274a745a0 tests: checkout: test CLI for single files 
Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2018-07-01 21:02:43 +03:00