Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,812 Commits 7 Branches 579 Tags
73c493500b1a4d3c81ac854541e92ea385d02ccc
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev 2b63cedfcd dvc: bump to 2.7.1 2021-09-07 21:16:49 +03:00
Ruslan Kuprieiev cbd49365c6 dvc: bump to 2.7.0 2021-09-06 20:21:55 +03:00
Ruslan Kuprieiev 6287b13ba3 gha: allow running tests on other branches (#6520) 
Simplifies development for all contributors.
 2021-09-02 17:55:54 +03:00
Ruslan Kuprieiev 5d7476a539 gha: packages: don't forget terraform 2021-08-27 23:43:28 +03:00
Ruslan Kuprieiev 341ab4fe9b gha: packages: don't forget terraform 2021-08-27 23:38:21 +03:00
Ruslan Kuprieiev 37717a7ef3 hdfs: don't read the whole file into memory (#6498) 
* hdfs: don't read the whole file into memory

Oversight from #4973 🤦

* hdfs: use default block size of 128MB
 2021-08-27 23:19:47 +03:00
Ruslan Kuprieiev 4382b80d3b dvcfile: support remote per output (#6486) 
Related to #2095
 2021-08-27 12:45:25 +03:00
Ruslan Kuprieiev 95ba68d181 ref: use fs.checksum instead of get_mtime_and_size (#6495) 2021-08-27 12:37:08 +09:00
Ruslan Kuprieiev 13d1276a12 dvc: bump to 2.6.4 2021-08-20 20:00:55 +03:00
Ruslan Kuprieiev 7da20a7c74 setup: upgrade s3fs 
Fixes https://github.com/iterative/dvc/issues/6466
 2021-08-20 19:43:14 +03:00
Ruslan Kuprieiev 82e559a51a dvc: bump to 2.6.3 2021-08-17 15:17:48 +03:00
Ruslan Kuprieiev 33c6a9b19b analytics: introduce DVC_NO_ANALYTICS (#6453) 
Replacing DVC_ANALYTICS. This is following our other env conventions and
will not have effect if set to false/no/etc, and will only work if set
to true/yes/y/etc. Will disable analytics even if it is explicitly
enabled in our configs.

Kudos @skshetry
 2021-08-17 15:16:40 +03:00
Ruslan Kuprieiev 9fef38e778 dvc: bump to 2.6.2 2021-08-17 02:52:07 +03:00
Ruslan Kuprieiev c9b3d9dbae dvc: bump to 2.6.1 2021-08-17 01:49:06 +03:00
Ruslan Kuprieiev c2bf0dc713 analytics: add DVC_ANALYTICS env var (#6448) 2021-08-17 01:48:38 +03:00
Ruslan Kuprieiev 0aec0e3249 updater: only notify in dvc doctor (#6247) 
We've used updater for years now to notify our users ASAP about new
releases. These days dvc is more stable and we've actually been
receiving numerous complains about updater being too annoying and
even introduced a config flag to turn it off, which a lot of people do
and which defeats its purpose. Instead of introducing some complex logic
that would notify more rarely and once for the user instead of once for
every dvc project they have - this PR just turns it off for everything,
except `dvc doctor/version`, which is the command that people use when
they are running into issues and that will give them a hint to just try
out the newest version first, as it might be already fixed.

This PR is a more of a compromise, to get the most out of updater while
removing a pain point of it being too annoying, without investing much
time into it and not having to maintain complex unnecessary logic. We
might get back to introducing something more intricate in the future,
if the need arises.

Fixes #5912
 2021-08-16 17:28:49 +05:45
Ruslan Kuprieiev 13c1314099 dvc: bump to 2.6.0 2021-08-15 17:39:36 +03:00
Ruslan Kuprieiev 27b54198a4 pyinstaller: build: check dvc doctor output (#6425) 2021-08-15 17:30:17 +03:00
Ruslan Kuprieiev e99e6cd367 repofs: use underlying fs.download to download files (#6401) 2021-08-10 13:12:56 +03:00
Ruslan Kuprieiev d97a19e97d refodb: cache ref objects (#6392) 
* refodb: cache ref objects

* odb: ignore FileNotFoundError when removing
 2021-08-09 13:53:38 +03:00
Ruslan Kuprieiev 8d71c3861f fs: raise RemoteNotFoundError (#6375) 2021-08-02 21:09:55 +03:00
Ruslan Kuprieiev 15de826266 setup: limit python-benedict version (#6377) 
Temporary fix for #6374 Kudos @skshetry
 2021-08-02 20:44:03 +03:00
Ruslan Kuprieiev e3a541aa00 setup: relax aiobotocore requirements (#6367) 
Fixes #6366
 2021-07-28 16:19:16 +03:00
Ruslan Kuprieiev 1e43c303a7 output: cache tree objects (#6301) 2021-07-24 23:07:43 +03:00
Ruslan Kuprieiev 97809661bd setup: pathspec <0.9.0 (#6331) 2021-07-18 18:24:28 +03:00
Ruslan Kuprieiev 52047214b7 fs: repo: handle directories with multiple artifacts (#6278) 
Related to #5677
 2021-07-12 02:17:07 +03:00
Ruslan Kuprieiev 67c46b98dd dvc: bump to 2.5.4 2021-07-08 23:02:16 +03:00
Ruslan Kuprieiev bd5809572e setup: relax requirements for importlib-metadata 2021-07-08 23:01:48 +03:00
Ruslan Kuprieiev 05832c81fc dvc: bump to 2.5.3 2021-07-08 19:34:26 +03:00
Ruslan KuprieievandSaugat Pachhai 86df89e3a8 pkg: don't build universal wheels (#6299) 
Kudos @skshetry for discovering that we've been publishing py2.py3
wheels on pypi and researching the cause.

Co-authored-by: Saugat Pachhai <[email protected]>

Co-authored-by: Saugat Pachhai <[email protected]>
 2021-07-08 19:33:52 +03:00
Ruslan Kuprieiev c33776eaee dvc: bump to 2.5.2 2021-07-08 18:30:11 +03:00
Ruslan Kuprieiev ae478ccdc0 dvc: bump to 2.5.1 2021-07-08 17:48:15 +03:00
Ruslan Kuprieiev 247b9c8be8 dvc: bump to 2.5.0 2021-07-03 22:34:11 +03:00
Ruslan Kuprieiev 07f5f6b633 ssh: don't use path_info for connections (#6259) 2021-07-03 21:54:14 +03:00
Ruslan Kuprieiev aa255a1102 output: optimize obj collection (#6277) 
* output: str only once

Calling `__str__` takes a long time and if we are dealing with a large
dataset, it will result in a very substantial time. For example,
locally, for 700K objects, it took 20sec more.

Related to #6276

* output: don't use os.path.join

Takes around 2sec for 700K objects.

* output: use fs.sep

* output: remove unused sep
 2021-07-03 21:53:55 +03:00
Ruslan Kuprieiev 87cdbf8ace dvc: bump to 2.4.3 2021-06-29 17:27:21 +03:00
Ruslan Kuprieiev d1b36e36ca dvc: bump to 2.4.2 2021-06-28 20:19:42 +03:00
Ruslan Kuprieiev 4e792ae61c setup: update fsspec to 2021.6.1 (#6219) 
* setup: update fsspec to 2021.6.1

* fs: local: remove temp workaround
 2021-06-24 10:06:42 +05:45
Ruslan Kuprieiev 53e9460c65 staging: remove redundant exists() call (#6217) 2021-06-24 00:19:06 +03:00
Ruslan Kuprieiev 09946847d2 dvc: bump to 2.4.1 2021-06-23 16:08:10 +03:00
Ruslan Kuprieiev 6779d3d431 dvc: bump to 2.4.0 2021-06-22 20:14:46 +03:00
Ruslan Kuprieiev 429d54e146 stage: monitor: don't forget to unprotect outs (#6134) 
Fixes #6083
 2021-06-10 10:53:51 +02:00
Ruslan Kuprieiev 8b9b2f6907 ls: specify rev as kwarg (#6121) 
Fixes #5865
 2021-06-07 18:36:35 +03:00
Ruslan Kuprieiev 5fb181fcf8 odb: don't use fs.path_info (#6059) 
* odb: don't use fs.path_info

* fs: get rid of fs.path_info
 2021-06-02 20:43:51 +03:00
Ruslan Kuprieiev 5e4de841d1 dvc: bump to 2.3.0 2021-06-02 15:08:23 +03:00
Ruslan Kuprieiev 488a6f8076 fs: don't automatically create buckets (#6086) 
* fs: azure: don't create bucket

* fs: oss: don't create bucket
 2021-06-01 01:29:17 +03:00
Ruslan Kuprieiev 04349a50eb dvc: bump to 2.2.0 2021-05-31 13:43:57 +03:00
Ruslan Kuprieiev 8a7ccab82f fs: deps/outs: get rid of types (#6054) 
Pre-requisite for removing fs.path_info and switching to regular strings
as paths in filesystems.

With new filesystems we no longer need the zoo of different dependency
and output types, as most of them are now just handled by the underlying
filesystem. Though deps/outs are still due for a refactor in other
aspects.
 2021-05-25 19:25:09 +03:00
Ruslan Kuprieiev aa672c5108 setup: bump dulwich to 0.20.23 (#6053) 
Fixes #6052
 2021-05-25 00:30:22 +03:00
Ruslan Kuprieiev c6b4c94fde dvcfs: raise FileNotFoundError on missing version info (#6017) 2021-05-18 19:12:35 +03:00
Ruslan Kuprieiev a5b080645c dvcignore: refactor from_file(s) (#6002) 2021-05-13 12:23:58 +02:00
Ruslan Kuprieiev 6d941e6cb5 fs: get rid of fs_root (#6000) 
* dvcignore: don't use fs_root

* fs: get rid of fs_root
 2021-05-13 04:15:56 +03:00
Ruslan Kuprieiev da0f0406fb fs: fix typo 2021-05-12 22:28:58 +03:00
Ruslan Kuprieiev 89b40afee7 fs: make constructor closer to fsspec (#5911) 
* fs: don't use repo for jobs/hash_jobs

* fs: local: exists: use fsspec's LocalFileSystem

* fs: make constructor comply with fsspec
 2021-05-12 22:03:09 +03:00
Ruslan Kuprieiev 0ad1408da7 plots: handle NoRemoteError for dirs (#5976) 2021-05-07 14:35:35 +03:00
Ruslan Kuprieiev 3db25ea67d dvc: bump to 2.1.0 2021-05-03 16:29:49 +03:00
Ruslan Kuprieiev c9a748421a dependabot: add "maintenance" label (#5898) 2021-04-29 16:59:53 +03:00
Ruslan Kuprieiev cab4126ed4 output: relative paths outside of repo (#5893) 
* output: relative paths outside of repo

Fixes #5870

* scm: enhance FileNotInRepo message
 2021-04-28 16:13:01 +03:00
Ruslan Kuprieiev 3472cb1a6a tests: don't set sensitive env secrets for all steps (#5823) 
Due to https://about.codecov.io/security-update/ , there is a potential that we've leaked these. This PR takes a more granular approach when setting secret env vars so that this doesn't happen in the future.

We'll change these secrets after this PR is merged.
 2021-04-19 16:39:37 +03:00
Ruslan Kuprieiev d02ab4bafd dvc: bump to 2.0.18 2021-04-15 23:19:04 +03:00
Ruslan Kuprieiev 853d304ce2 repo: handle rev without scm (#5822) 
This part of logic used to be only used by external repo for cloned
repos, but since we now handle local repos without cloning, we don't
always pass scm instance. This PR automatically creates scm instance
for such cases.

Fixes #5590
 2021-04-15 23:16:01 +03:00
Ruslan Kuprieiev 4f8cfc025b gha: remove leftovers 2021-04-06 14:19:25 +03:00
Ruslan Kuprieiev cd98761e63 Delete debian directory 
Closes #2800
 2021-04-06 14:06:09 +03:00
Ruslan Kuprieiev 14f05073ad gha: packages: don't build snap (#5774) 
* Delete snap directory

* gha: packages: remove snap
 2021-04-06 13:59:09 +03:00
Ruslan Kuprieiev 2049a9dbdd dvc: bump to 2.0.17 2021-04-02 15:57:17 +03:00
Ruslan Kuprieiev ba38bb300c sign: increase timeout to 180 sec 2021-04-02 15:56:56 +03:00
Ruslan Kuprieiev 585ff36486 dvc: bump to 2.0.16 2021-04-01 21:34:52 +03:00
Ruslan Kuprieiev b2503d5178 pyinstaller: hooks: bring back hook-googleapiclient.model.py 
Got deleted by mistake in https://github.com/iterative/dvc/pull/5682/files
 2021-04-01 21:34:26 +03:00
Ruslan Kuprieiev c4622c6311 sign: increase timeout to 90 2021-04-01 02:11:28 +03:00
Ruslan Kuprieiev 2f759d60be sign: increase timeout 2021-04-01 01:41:36 +03:00
Ruslan Kuprieiev 6be2efc9cf dvc: bump to 2.0.15 2021-03-31 00:06:33 +03:00
Ruslan Kuprieiev 09cb495423 dvc: bump to 2.0.14 2021-03-30 15:44:20 +03:00
Ruslan Kuprieiev 7d1a35388e dvc: bump to 2.0.13 2021-03-29 16:56:06 +03:00
Ruslan Kuprieiev 98edde06a4 sign: increase timeout 2021-03-29 16:27:05 +03:00
Ruslan Kuprieiev 30f9ad4b54 sign: add timeout and verbosity 2021-03-29 15:54:32 +03:00
Ruslan Kuprieiev 1a25ebe3bd osxpkg: notarize: fix typo 2021-03-29 01:07:06 +03:00
Ruslan Kuprieiev 813f4d1f07 dvc: bump to 2.0.12 2021-03-27 00:13:30 +02:00
Ruslan Kuprieiev 4ffa0675b4 build: fix --notarize 2021-03-26 03:47:34 +02:00
Ruslan Kuprieiev c0671a2d9b gha: don't upload pkg artifacts 2021-03-26 03:18:29 +02:00
Ruslan Kuprieiev f8498d6340 gha: don't build packages for PRs 2021-03-26 03:17:22 +02:00
Ruslan Kuprieiev 27b589f1b9 dvc: bump to 2.0.11 2021-03-26 03:06:36 +02:00
Ruslan Kuprieiev 49cdc2bea3 build: add osx signing/notarization (#5695) 
* build: add osx signing/notarization

* gha: temporarily enable pkg builds
 2021-03-26 02:31:32 +02:00
Ruslan Kuprieiev a3a55339ab dvc: bump to 2.0.10 2021-03-25 02:21:52 +02:00
Ruslan Kuprieiev 8abffbf68b scripts: fpm: install zsh competion to install / "share" 
Currently we create /usr/share/... which is not allowed by macos:

> the package is trying to install content to the system volume
 2021-03-25 02:21:11 +02:00
Ruslan Kuprieiev 2e76dabbaa dvc: bump to 2.0.9 2021-03-24 20:30:48 +02:00
Ruslan Kuprieiev 8bebed93dd build: osxpkg: build lxml (#5687) 
lxml wheels are built with an old SDK, which breaks osxpkg notarization:

```
	* error for path "dvc-2.0.7.pkg/dvc-2.0.7.pkg Contents/Payload/usr/local/lib/dvc/lxml/etree.cpython-37m-darwin.so": The binary uses an SDK older than the 10.9 SDK.
```
 2021-03-24 20:30:03 +02:00
Ruslan Kuprieiev 58a5442d71 dvc: bump to 2.0.8 2021-03-24 19:20:04 +02:00
Ruslan Kuprieiev f2ba67aece build: upgrade pyinstaller to 4.2 (#5682) 
* build: upgrade pyinstaller to 4.2

* pyinstaller: hooks: remove hook-googleapiclient.model.py

https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py

* gha: use python 3.8 for pyinstaller
 2021-03-24 09:43:05 +02:00
Ruslan Kuprieiev 19cdef1675 build: separate building and packaging (#5675) 
* build: separate pyinstaller and fpm/innosetup

* gha: upload deb/rpm/osxpkg/exe artifacts

* build: dynamically generate dvc version
 2021-03-23 19:55:44 +02:00
Ruslan Kuprieiev 619c55d399 dvc: bump to 2.0.7 2021-03-22 15:29:52 +02:00
Ruslan Kuprieiev dcc55aa645 dvc: bump to 2.0.6 2021-03-17 00:01:10 +02:00
Ruslan Kuprieiev 2db55032c4 objects: file: remove unused load() method (#5610) 2021-03-12 05:35:48 +02:00
Ruslan Kuprieiev 29ad87ebe2 objects: split into separate files (#5609) 2021-03-12 04:12:46 +02:00
Ruslan Kuprieiev 4af3b3d20d dvc: refactor objects (#5607) 
* dvc: get rid of DirInfo

DirInfo was created before we had proper objects, as a way to have a
structure that represents a staged directory. Right now we have a
dedicated Tree object that is specifically meant for that.

As a result, HashInfo no longer needs to have dir_info and we no longer
need `get_hash/get_dir_hash` helpers.

* objects: move Tree into tree.py
 2021-03-12 01:10:34 +02:00
Ruslan Kuprieiev cf1070a531 dvc: bump to 2.0.5 2021-03-10 11:33:47 +02:00
Ruslan Kuprieiev 8433b36266 dvc: bump to 2.0.4 2021-03-09 17:59:26 +02:00
Ruslan Kuprieiev b4232bdc7c state: use pickle v4 (#5571) 
Kudos @skshetry for the discovery, research and fix.

Fixes #5563
 2021-03-08 17:00:40 +02:00
Ruslan Kuprieiev 70c445d1ee utils: makedirs: ignore chmod errors (#5570) 
Same as with `protect()`, these errors might happen if fs is immutable
or doesn't support chmod()-ing. We can safely ignore these.

Fixes #5552
 2021-03-08 16:28:02 +02:00
Ruslan Kuprieiev 47bcb02ef1 fs: git: use pygit2 (#5559) 2021-03-06 02:54:13 +02:00
Ruslan Kuprieiev 4a75067f4c github: template: use dvc doctor instead of dvc version 2021-03-05 16:08:46 +02:00