Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
5,095 Commits 7 Branches 579 Tags
2734f49b9ed8ec16364b027c48ae91608e0c09fc
Commit Graph
201 Commits
Author SHA1 Message Date
Max Risuhin 7b0e44e293 gdrive: support corpora and driveId 2020-01-28 11:50:21 -08:00
Ruslan Kuprieiev 2f1224b27b setup: update pydrive2 version 2020-01-24 09:47:52 +02:00
Ruslan Kuprieiev 1eb82ac707 dvc: replace flatten_dict with flatten_json 
flatten_json has a conda package, while flatten_dict doesn't and I'm not
willing to maintain yet another conda package :)
 2020-01-21 00:14:22 +02:00
Ruslan Kuprieiev 6ebd053c0f Merge pull request #3165 from mroutis/remove-py2-compat 
setup: remove py2 backports
 2020-01-17 04:28:22 +02:00
Ruslan Kuprieiev 74bbc3bc04 Merge pull request #3135 from rxxg/master 
Perf improvement for file copies
 2020-01-16 18:46:58 +02:00
Mr. Outis 48326620ec setup: remove py2 backports 2020-01-15 19:13:01 -06:00
Ruslan Kuprieiev 97a29a1f7e dvc: use pydrive2 instead of pydrive 
PyDrive2 is our maintained fork.
 2020-01-15 19:13:29 +02:00
rxxg e170852e97 Merge branch 'master' into master 2020-01-15 12:53:40 +01:00
rxxg 8fca139246 Use released version of speedcopy library 2020-01-15 12:50:43 +01:00
Ruslan Kuprieiev 9cafef1fdd Merge pull request #3051 from efiop/2995 
metrics: introduce `diff`
 2020-01-14 14:18:40 +02:00
Ruslan Kuprieiev f18bfdc674 setup: fix typo 2020-01-14 11:22:29 +02:00
Ruslan Kuprieiev ebb0243e87 setup: rollback pydrive dep 
pypi doesn't allow direct github deps https://travis-ci.com/iterative/dvc/jobs/275282583
 2020-01-14 11:15:11 +02:00
rxxg 674541f922 Use pyfastcopy for non-Windows systems using Python < 3.8 2020-01-14 09:32:09 +01:00
rxxg 811fc5317c Use speedcopy module with bugfix 2020-01-14 09:32:09 +01:00
rxxg 09223442f5 Use speedcopy module for faster copies across network boundaries 2020-01-14 09:32:09 +01:00
Ruslan Kuprieiev e393e29ef5 metrics: introduce diff 
This first implementation is based on existing `dvc metrics show`
functionality, but is also able to auto-detect json to properly
show diff for it.
 2020-01-13 06:45:48 +02:00
Max Risuhin 57111a7788 gdrive: add support for teamDriveId 2020-01-11 07:41:12 -08:00
Ruslan Kuprieiev c064865c80 build: use build-requirements.txt 2020-01-11 04:55:34 +02:00
Ruslan Kuprieiev ba45c9cd5b setup: get rid of py2 reqs 2020-01-08 02:16:06 +02:00
Ruslan Kuprieiev 381b279e53 Merge pull request #3022 from mroutis/remove-future-imports 
py3: remove future imports
 2019-12-31 04:08:22 +02:00
Mr. Outis 94a0914bba py3: get rid of future imports 2019-12-30 19:35:48 -06:00
Mr. Outis c61254dc75 py3: get rid of py2 on CI scripts 2019-12-30 19:16:10 -06:00
Mr. Outis d3d6f679be py3: modify python_requires to support 3.5 onwards 2019-12-30 13:10:28 -06:00
Ruslan Kuprieiev f1b7f86dbf setup: fix inflect version requirements 2019-12-25 16:04:23 +02:00
Ruslan Kuprieiev 25807f2690 dvc: make flufl.lock opt-in and use zc.lockfile 
As it turned out (see issue numbers down below), we can't really take
hardlinks for granted, so `flufl.lock` is not a panacea for all
filesystems. Considering that the vast majority of filesystems that our
users use support `zc.lockfile`(flock-based) and it has benefits like
more reliable mechanism, auto-delete when process exits, more sturdy
implementation, etc, it makes more sense to bring it back and use by
default again. For filesystems that don't support `flock()`, users will
be able to manually enable `flufl.lock` use through the config option.
It would be ideal if we could auto-detect that flock is not supported,
but in the real world, it turned out to be non-trivial, as it might hang
forever in a kernel context, which makes the implementation way too
complex for our purposes. So what we're doing instead is showing a
message before locking with `zc.lockfile` that, under normal
circumstances will disappear once the lock is taken or failed, otherwise
it will point users to the related documentation where they can learn
about how to opt-in for `flufl.lock`.

Fixes #2831
Fixes #2897
Related #2860
 2019-12-13 02:25:57 +02:00
Ruslan Kuprieiev 6296e12f48 setup: update pyarrow version 
https://github.com/apache/arrow/pull/4907
 2019-12-11 06:24:49 +02:00
Ruslan Kuprieiev 82dd1ac7c1 Merge pull request #2815 from xliiv/2807-use-pager 
dvc/dagascii: Use pager instead of AsciiCanvas._do_draw
 2019-12-07 01:16:16 +02:00
Mr. Outis 2237ab6a51 setup: fix only mayor minor version for pyyaml 2019-12-05 17:47:01 -06:00
Mr. Outis 4fd701317b setup: fix compatibility errors between dependencies 
fix #2570
 2019-12-05 13:19:23 -06:00
Tymoteusz Jankowski c17cf2b2f7 dvc/dagascii: Use less instead of AsciiCanvas._do_draw 
Uses less CLI tool to paging the output in the interactive mode while
doing e.g.  `dvc pipeline show ..`.

Fixes #2807
 2019-12-01 13:23:09 +01:00
Casper da Costa-Luis ecdc608e13 clean up gs progress using Tqdm.wrapattr 
- fixes #2829
 2019-12-01 02:22:04 +00:00
Max Risuhin e7227898e4 Support GDrive as remote 2019-11-29 01:21:10 -08:00
Ruslan Kuprieiev 510ff8a9da Merge pull request #2863 from Suor/locks 
remote: protect all remote client/session creation code with locks
 2019-11-29 11:18:56 +02:00
Ruslan Kuprieiev 593ab17d25 setup: don't forget contextlib2 2019-11-29 02:14:25 +02:00
Alexander Schepanovski 73eac3b6cc remote: protect all remote client/session creation code with locks 
With @cached_property last write wins, so all threads will use the same
session. However, several might be created concurrently wasting
resources and even potentially asking something from the user.

This change guarantees only one client instance will be created for a
remote.
 2019-11-28 21:05:20 +01:00
Saugat Pachhai 461a79e1e8 Fix typos 2019-11-26 20:07:20 +05:45
Ruslan Kuprieiev a1433ba356 Merge branch 'master' into opt-validate 2019-11-20 11:00:28 +02:00
Ruslan Kuprieiev 7fae34ee5d test: freeze pytest-mock version 
To address failing tests.
 2019-11-19 18:52:10 +02:00
Alexander Schepanovski 3bffe9dbff dvc: drop schema from requirements 2019-11-18 17:24:40 +01:00
Alexander Schepanovski 78fad73f2b perf: switch schema validation library 
voluptuous is compiling schema to python function, which makes it
faster.
 2019-11-14 22:33:28 +01:00
Alexander Schepanovski 58c9e5557c perf: optimize yaml parsing for stages 2019-11-11 13:41:50 +01:00
Casper da Costa-Luis 2e1360a34f non-blocking intermediate progress IO 
- fixes #2766
 2019-11-09 01:43:47 +00:00
Ruslan Kuprieiev ee70822bd4 Merge pull request #2736 from efiop/master 
dvc: automatically reorder imports
 2019-11-05 03:02:32 +02:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Ruslan Kuprieiev b718b839bf system: win: use win32file for _getdirinfo 
Current implementation is triggering some non-obvious gc bug, that
causes memory leaks. Using win32file simplifies the code and gets rid of
this bug.

To reproduce it, run:

```
from dvc.system import System

while True:
    System.inode(__file__)
```

and see your Windows machine run out of memory pretty quickly.
 2019-11-05 01:46:21 +02:00
Ruslan Kuprieiev e75c52a170 setup: freeze awscli version 2019-11-03 15:09:29 +02:00
Ruslan Kuprieiev 3514cc5b5a test: freeze moto version 2019-11-03 14:49:36 +02:00
Alexander Schepanovski 77fcd49695 dvc: fix tqdm dep (#2694) 
tqdm==4.35.0 was throwing KeyError: 'percentage'.
 2019-10-30 16:43:18 +02:00
Ruslan Kuprieiev 24ed639901 setup: add upper limit for tqdm version 
https://github.com/iterative/dvc/issues/2659#issuecomment-545940739
 2019-10-24 17:29:53 +03:00
Ruslan Kuprieiev 8d067c8805 setup: update project url 2019-10-24 15:19:04 +03:00