Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,719 Commits 7 Branches 579 Tags
67b42ebe4f9474d71a2d5b195014046775a08401
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev 97809661bd setup: pathspec <0.9.0 (#6331) 2021-07-18 18:24:28 +03:00
Ruslan Kuprieiev 52047214b7 fs: repo: handle directories with multiple artifacts (#6278) 
Related to #5677
 2021-07-12 02:17:07 +03:00
Ruslan Kuprieiev 67c46b98dd dvc: bump to 2.5.4 2021-07-08 23:02:16 +03:00
Ruslan Kuprieiev bd5809572e setup: relax requirements for importlib-metadata 2021-07-08 23:01:48 +03:00
Ruslan Kuprieiev 05832c81fc dvc: bump to 2.5.3 2021-07-08 19:34:26 +03:00
Ruslan KuprieievandSaugat Pachhai 86df89e3a8 pkg: don't build universal wheels (#6299) 
Kudos @skshetry for discovering that we've been publishing py2.py3
wheels on pypi and researching the cause.

Co-authored-by: Saugat Pachhai <[email protected]>

Co-authored-by: Saugat Pachhai <[email protected]>
 2021-07-08 19:33:52 +03:00
Ruslan Kuprieiev c33776eaee dvc: bump to 2.5.2 2021-07-08 18:30:11 +03:00
Ruslan Kuprieiev ae478ccdc0 dvc: bump to 2.5.1 2021-07-08 17:48:15 +03:00
Ruslan Kuprieiev 247b9c8be8 dvc: bump to 2.5.0 2021-07-03 22:34:11 +03:00
Ruslan Kuprieiev 07f5f6b633 ssh: don't use path_info for connections (#6259) 2021-07-03 21:54:14 +03:00
Ruslan Kuprieiev aa255a1102 output: optimize obj collection (#6277) 
* output: str only once

Calling `__str__` takes a long time and if we are dealing with a large
dataset, it will result in a very substantial time. For example,
locally, for 700K objects, it took 20sec more.

Related to #6276

* output: don't use os.path.join

Takes around 2sec for 700K objects.

* output: use fs.sep

* output: remove unused sep
 2021-07-03 21:53:55 +03:00
Ruslan Kuprieiev 87cdbf8ace dvc: bump to 2.4.3 2021-06-29 17:27:21 +03:00
Ruslan Kuprieiev d1b36e36ca dvc: bump to 2.4.2 2021-06-28 20:19:42 +03:00
Ruslan Kuprieiev 4e792ae61c setup: update fsspec to 2021.6.1 (#6219) 
* setup: update fsspec to 2021.6.1

* fs: local: remove temp workaround
 2021-06-24 10:06:42 +05:45
Ruslan Kuprieiev 53e9460c65 staging: remove redundant exists() call (#6217) 2021-06-24 00:19:06 +03:00
Ruslan Kuprieiev 09946847d2 dvc: bump to 2.4.1 2021-06-23 16:08:10 +03:00
Ruslan Kuprieiev 6779d3d431 dvc: bump to 2.4.0 2021-06-22 20:14:46 +03:00
Ruslan Kuprieiev 429d54e146 stage: monitor: don't forget to unprotect outs (#6134) 
Fixes #6083
 2021-06-10 10:53:51 +02:00
Ruslan Kuprieiev 8b9b2f6907 ls: specify rev as kwarg (#6121) 
Fixes #5865
 2021-06-07 18:36:35 +03:00
Ruslan Kuprieiev 5fb181fcf8 odb: don't use fs.path_info (#6059) 
* odb: don't use fs.path_info

* fs: get rid of fs.path_info
 2021-06-02 20:43:51 +03:00
Ruslan Kuprieiev 5e4de841d1 dvc: bump to 2.3.0 2021-06-02 15:08:23 +03:00
Ruslan Kuprieiev 488a6f8076 fs: don't automatically create buckets (#6086) 
* fs: azure: don't create bucket

* fs: oss: don't create bucket
 2021-06-01 01:29:17 +03:00
Ruslan Kuprieiev 04349a50eb dvc: bump to 2.2.0 2021-05-31 13:43:57 +03:00
Ruslan Kuprieiev 8a7ccab82f fs: deps/outs: get rid of types (#6054) 
Pre-requisite for removing fs.path_info and switching to regular strings
as paths in filesystems.

With new filesystems we no longer need the zoo of different dependency
and output types, as most of them are now just handled by the underlying
filesystem. Though deps/outs are still due for a refactor in other
aspects.
 2021-05-25 19:25:09 +03:00
Ruslan Kuprieiev aa672c5108 setup: bump dulwich to 0.20.23 (#6053) 
Fixes #6052
 2021-05-25 00:30:22 +03:00
Ruslan Kuprieiev c6b4c94fde dvcfs: raise FileNotFoundError on missing version info (#6017) 2021-05-18 19:12:35 +03:00
Ruslan Kuprieiev a5b080645c dvcignore: refactor from_file(s) (#6002) 2021-05-13 12:23:58 +02:00
Ruslan Kuprieiev 6d941e6cb5 fs: get rid of fs_root (#6000) 
* dvcignore: don't use fs_root

* fs: get rid of fs_root
 2021-05-13 04:15:56 +03:00
Ruslan Kuprieiev da0f0406fb fs: fix typo 2021-05-12 22:28:58 +03:00
Ruslan Kuprieiev 89b40afee7 fs: make constructor closer to fsspec (#5911) 
* fs: don't use repo for jobs/hash_jobs

* fs: local: exists: use fsspec's LocalFileSystem

* fs: make constructor comply with fsspec
 2021-05-12 22:03:09 +03:00
Ruslan Kuprieiev 0ad1408da7 plots: handle NoRemoteError for dirs (#5976) 2021-05-07 14:35:35 +03:00
Ruslan Kuprieiev 3db25ea67d dvc: bump to 2.1.0 2021-05-03 16:29:49 +03:00
Ruslan Kuprieiev c9a748421a dependabot: add "maintenance" label (#5898) 2021-04-29 16:59:53 +03:00
Ruslan Kuprieiev cab4126ed4 output: relative paths outside of repo (#5893) 
* output: relative paths outside of repo

Fixes #5870

* scm: enhance FileNotInRepo message
 2021-04-28 16:13:01 +03:00
Ruslan Kuprieiev 3472cb1a6a tests: don't set sensitive env secrets for all steps (#5823) 
Due to https://about.codecov.io/security-update/ , there is a potential that we've leaked these. This PR takes a more granular approach when setting secret env vars so that this doesn't happen in the future.

We'll change these secrets after this PR is merged.
 2021-04-19 16:39:37 +03:00
Ruslan Kuprieiev d02ab4bafd dvc: bump to 2.0.18 2021-04-15 23:19:04 +03:00
Ruslan Kuprieiev 853d304ce2 repo: handle rev without scm (#5822) 
This part of logic used to be only used by external repo for cloned
repos, but since we now handle local repos without cloning, we don't
always pass scm instance. This PR automatically creates scm instance
for such cases.

Fixes #5590
 2021-04-15 23:16:01 +03:00
Ruslan Kuprieiev 4f8cfc025b gha: remove leftovers 2021-04-06 14:19:25 +03:00
Ruslan Kuprieiev cd98761e63 Delete debian directory 
Closes #2800
 2021-04-06 14:06:09 +03:00
Ruslan Kuprieiev 14f05073ad gha: packages: don't build snap (#5774) 
* Delete snap directory

* gha: packages: remove snap
 2021-04-06 13:59:09 +03:00
Ruslan Kuprieiev 2049a9dbdd dvc: bump to 2.0.17 2021-04-02 15:57:17 +03:00
Ruslan Kuprieiev ba38bb300c sign: increase timeout to 180 sec 2021-04-02 15:56:56 +03:00
Ruslan Kuprieiev 585ff36486 dvc: bump to 2.0.16 2021-04-01 21:34:52 +03:00
Ruslan Kuprieiev b2503d5178 pyinstaller: hooks: bring back hook-googleapiclient.model.py 
Got deleted by mistake in https://github.com/iterative/dvc/pull/5682/files
 2021-04-01 21:34:26 +03:00
Ruslan Kuprieiev c4622c6311 sign: increase timeout to 90 2021-04-01 02:11:28 +03:00
Ruslan Kuprieiev 2f759d60be sign: increase timeout 2021-04-01 01:41:36 +03:00
Ruslan Kuprieiev 6be2efc9cf dvc: bump to 2.0.15 2021-03-31 00:06:33 +03:00
Ruslan Kuprieiev 09cb495423 dvc: bump to 2.0.14 2021-03-30 15:44:20 +03:00
Ruslan Kuprieiev 7d1a35388e dvc: bump to 2.0.13 2021-03-29 16:56:06 +03:00
Ruslan Kuprieiev 98edde06a4 sign: increase timeout 2021-03-29 16:27:05 +03:00
Ruslan Kuprieiev 30f9ad4b54 sign: add timeout and verbosity 2021-03-29 15:54:32 +03:00
Ruslan Kuprieiev 1a25ebe3bd osxpkg: notarize: fix typo 2021-03-29 01:07:06 +03:00
Ruslan Kuprieiev 813f4d1f07 dvc: bump to 2.0.12 2021-03-27 00:13:30 +02:00
Ruslan Kuprieiev 4ffa0675b4 build: fix --notarize 2021-03-26 03:47:34 +02:00
Ruslan Kuprieiev c0671a2d9b gha: don't upload pkg artifacts 2021-03-26 03:18:29 +02:00
Ruslan Kuprieiev f8498d6340 gha: don't build packages for PRs 2021-03-26 03:17:22 +02:00
Ruslan Kuprieiev 27b589f1b9 dvc: bump to 2.0.11 2021-03-26 03:06:36 +02:00
Ruslan Kuprieiev 49cdc2bea3 build: add osx signing/notarization (#5695) 
* build: add osx signing/notarization

* gha: temporarily enable pkg builds
 2021-03-26 02:31:32 +02:00
Ruslan Kuprieiev a3a55339ab dvc: bump to 2.0.10 2021-03-25 02:21:52 +02:00
Ruslan Kuprieiev 8abffbf68b scripts: fpm: install zsh competion to install / "share" 
Currently we create /usr/share/... which is not allowed by macos:

> the package is trying to install content to the system volume
 2021-03-25 02:21:11 +02:00
Ruslan Kuprieiev 2e76dabbaa dvc: bump to 2.0.9 2021-03-24 20:30:48 +02:00
Ruslan Kuprieiev 8bebed93dd build: osxpkg: build lxml (#5687) 
lxml wheels are built with an old SDK, which breaks osxpkg notarization:

```
	* error for path "dvc-2.0.7.pkg/dvc-2.0.7.pkg Contents/Payload/usr/local/lib/dvc/lxml/etree.cpython-37m-darwin.so": The binary uses an SDK older than the 10.9 SDK.
```
 2021-03-24 20:30:03 +02:00
Ruslan Kuprieiev 58a5442d71 dvc: bump to 2.0.8 2021-03-24 19:20:04 +02:00
Ruslan Kuprieiev f2ba67aece build: upgrade pyinstaller to 4.2 (#5682) 
* build: upgrade pyinstaller to 4.2

* pyinstaller: hooks: remove hook-googleapiclient.model.py

https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py

* gha: use python 3.8 for pyinstaller
 2021-03-24 09:43:05 +02:00
Ruslan Kuprieiev 19cdef1675 build: separate building and packaging (#5675) 
* build: separate pyinstaller and fpm/innosetup

* gha: upload deb/rpm/osxpkg/exe artifacts

* build: dynamically generate dvc version
 2021-03-23 19:55:44 +02:00
Ruslan Kuprieiev 619c55d399 dvc: bump to 2.0.7 2021-03-22 15:29:52 +02:00
Ruslan Kuprieiev dcc55aa645 dvc: bump to 2.0.6 2021-03-17 00:01:10 +02:00
Ruslan Kuprieiev 2db55032c4 objects: file: remove unused load() method (#5610) 2021-03-12 05:35:48 +02:00
Ruslan Kuprieiev 29ad87ebe2 objects: split into separate files (#5609) 2021-03-12 04:12:46 +02:00
Ruslan Kuprieiev 4af3b3d20d dvc: refactor objects (#5607) 
* dvc: get rid of DirInfo

DirInfo was created before we had proper objects, as a way to have a
structure that represents a staged directory. Right now we have a
dedicated Tree object that is specifically meant for that.

As a result, HashInfo no longer needs to have dir_info and we no longer
need `get_hash/get_dir_hash` helpers.

* objects: move Tree into tree.py
 2021-03-12 01:10:34 +02:00
Ruslan Kuprieiev cf1070a531 dvc: bump to 2.0.5 2021-03-10 11:33:47 +02:00
Ruslan Kuprieiev 8433b36266 dvc: bump to 2.0.4 2021-03-09 17:59:26 +02:00
Ruslan Kuprieiev b4232bdc7c state: use pickle v4 (#5571) 
Kudos @skshetry for the discovery, research and fix.

Fixes #5563
 2021-03-08 17:00:40 +02:00
Ruslan Kuprieiev 70c445d1ee utils: makedirs: ignore chmod errors (#5570) 
Same as with `protect()`, these errors might happen if fs is immutable
or doesn't support chmod()-ing. We can safely ignore these.

Fixes #5552
 2021-03-08 16:28:02 +02:00
Ruslan Kuprieiev 47bcb02ef1 fs: git: use pygit2 (#5559) 2021-03-06 02:54:13 +02:00
Ruslan Kuprieiev 4a75067f4c github: template: use dvc doctor instead of dvc version 2021-03-05 16:08:46 +02:00
Ruslan Kuprieiev 70f4c44ec6 dvc: bump to 2.0.3 2021-03-05 13:08:22 +02:00
Ruslan Kuprieiev 71f8ffe13d dvc: bump to 2.0.2 2021-03-05 01:04:00 +02:00
Ruslan Kuprieiev 06aad3e2b3 dependency: repo: try to fetch before staging (#5549) 
Related to https://github.com/iterative/dvc-bench/issues/234
 2021-03-05 01:03:33 +02:00
Ruslan Kuprieiev 98a1a92fa3 snap: debug failing tests 2021-03-03 23:11:19 +02:00
Ruslan Kuprieiev 16304b544f dvc: bump to 2.0.1 2021-03-03 17:38:26 +02:00
Ruslan Kuprieiev 803a100f32 setup: move google-cloud-storage to test requirements (#5540) 2021-03-03 17:11:30 +02:00
Ruslan Kuprieiev c3a0e8733e dvc: bump to 2.0.0 2021-03-03 12:21:03 +02:00
Ruslan Kuprieiev e385e59a09 dvc: bump to 2.0.0a4 2021-03-02 13:54:16 +02:00
Ruslan Kuprieiev 96092e5e27 dvc: bump to 2.0.0a3 2021-03-01 18:20:35 +02:00
Ruslan Kuprieiev f2e0059392 dvc: bump to 2.0.0a2 2021-02-26 13:35:28 +02:00
Ruslan Kuprieiev a26f2995ef dir_info: only build trie when needed (#5512) 2021-02-24 09:15:20 +02:00
Ruslan Kuprieiev 502f0eecc9 tests: temporarily skip dvclive tests (#5513) 2021-02-24 08:52:33 +02:00
Ruslan Kuprieiev 8f568ab197 state: use diskcache (#5505) 
* state: only load when needed

* state: don't use separate load/dump

* checkout: group link removal logic together

* state: use diskcache.Cache

* staging: use pbar and save to state

* state: don't pass the whole repo to __init__

* staging: explicitly pass state object

* fs: get rid of odb property

* state: don't close the connection to cache

* repo: close: close state
 2021-02-23 20:51:34 +02:00
Ruslan Kuprieiev 2e01ce1974 dvc: bump to 2.0.0a1 2021-02-23 20:48:42 +02:00
Ruslan Kuprieiev 282e83c481 tests: temporarily exclude gdrivefs from extras (#5483) 
When trying to publish to pypi, we get:

```
Invalid value for requires_dist. Error: Can't have direct dependency:
"gdrivefs @ git+https://github.com/intake/gdrivefs.git ; extra ==
'tests'"
```
 2021-02-17 05:49:34 +02:00
Ruslan Kuprieiev 51988475a4 gha: packages: update for 2.0 2021-02-17 04:38:45 +02:00
Ruslan Kuprieiev e0842ce01e dvc: rename cache -> odb (#5481) 2021-02-16 22:49:25 +02:00
Ruslan Kuprieiev a814a49181 fs: detach state from fs (#5479) 
We used to think this way, but state is not really an fs concept
(fsspec's dircache is though), as it is a cache of hashes of objects
that we are staging, and the hash doesn't depend on fs, as it is usually
computed by us by reading from fs.

Pre-requisite for diskcache-based state.
 2021-02-16 17:42:50 +02:00
Ruslan Kuprieiev ae8efdeb0f state: remove unused files (#5474) 
They've been used before we've moved state to .dvc/tmp
 2021-02-16 14:48:48 +02:00
Ruslan Kuprieiev 53ba51b5fe objects: separate staging (#5470) 2021-02-16 02:21:54 +02:00
Ruslan Kuprieiev e90c5fdd19 fs: remove unused use_state decorator (#5469) 
* fs: remove unused use_state decorator

* pylint: limit astroid version
 2021-02-16 02:00:55 +02:00
Ruslan Kuprieiev a72cea67f1 build: require git >= 1.7.0 (#5467) 2021-02-15 20:54:07 +02:00
Ruslan Kuprieiev 242d346f15 gs: ls: add etag (#5465) 2021-02-15 17:06:16 +02:00
Ruslan Kuprieiev bb24131530 fs/outs/deps: remove supported() logic (#5462) 2021-02-13 22:59:20 +02:00