Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
8,058 Commits 7 Branches 579 Tags
8f82a421a0608bef96f06ecbbcf4150c5333536b
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev 8f82a421a0 index: support multiple targets within output in IndexView 
stage_infos might contain the same stage multiple times, but with different
filter_info, so we shouldn't convert it to dict, as that makes us lose all
but one filter_info's.
 2022-10-25 13:37:44 +03:00
Ruslan Kuprieiev 23534063ca push: use build_tree instead of tree_from_index 2022-10-21 17:02:33 +03:00
Ruslan Kuprieiev d56957416e index: data: don't forget to set entry.key 2022-10-21 17:02:33 +03:00
Ruslan Kuprieiev 6711b653a5 tests: add tests for dvcignore.walk 2022-10-10 21:43:44 +03:00
Ruslan Kuprieiev faef8eec2c tests: ignore: use dvcignore.find directly 2022-10-10 21:43:44 +03:00
Ruslan Kuprieiev 11766c8839 ignore: walk: support detail=True 2022-10-10 21:43:44 +03:00
Ruslan Kuprieiev 890fbf3a6e deps: bump dvc-data to 0.14.0 2022-10-01 21:56:49 +03:00
Ruslan Kuprieiev 8c484645fa cloud-versioning: better handling for directories 2022-09-26 02:54:17 +03:00
Ruslan Kuprieiev 10cfa0d5d7 fetch: use md5() before save() 2022-09-23 15:12:19 +03:00
Ruslan Kuprieiev d2a31eb660 push: use update_pipeline=False for cloud versioning 2022-09-19 17:21:43 +03:00
Ruslan Kuprieiev 625b735502 deps: bump dvc-s3 2022-09-19 14:57:35 +03:00
Ruslan Kuprieiev 35e6f082a4 analytics: use iterative-telemetry for user_id lookup 2022-09-17 00:45:45 +03:00
Ruslan Kuprieiev 4531fec2b7 deps: bump dvc-data to 0.10.0 2022-09-16 22:56:56 +03:00
Ruslan Kuprieiev 3cbf319e0d loader: don't forget to set meta.isdir 
Leftover from #8273

Fixes #8300
 2022-09-15 22:53:48 +03:00
Ruslan Kuprieiev 477a6e177f index: use tree obj if using inline files 
Leftover from #8273
 2022-09-15 21:25:28 +03:00
Ruslan Kuprieiev 7e2248bbe5 deps: bump dvc-data to 0.7.1 2022-09-07 23:07:35 +03:00
Ruslan Kuprieiev 6cada6dc12 dvc: cloud versioning POC 2022-09-07 21:43:34 +03:00
Ruslan Kuprieiev 89d4461908 deps: temporarily limit dulwich to <0.20.46 
Looks like the new release broke some data status logic.
 2022-09-07 02:56:14 +03:00
Ruslan Kuprieiev a124d7e866 deps: bump dvc-data to 0.6.3 2022-09-07 02:23:29 +03:00
Ruslan Kuprieiev 628be2ae49 deps: bump dvc-data to 0.6.1 2022-09-06 20:30:20 +03:00
Ruslan Kuprieiev aff65d42bb deps: bump dvc-azure and dvc-s3 to 2.20.0 2022-09-01 13:00:47 +03:00
Ruslan Kuprieiev e307a62dc0 deps: bump dvc-data to 0.4.0 2022-09-01 02:59:24 +03:00
Ruslan Kuprieiev e7daeb1c3e deps: bump dvc-data to 0.3.0 2022-08-30 22:46:27 +03:00
Ruslan Kuprieiev 9cb242c8a5 deps: bump dvc-http and dvc-gs 2022-08-28 23:13:18 +03:00
Ruslan Kuprieiev 11802426de pyinstaller: don't forget dvc_* plugins 
Fixes #8194
 2022-08-28 21:05:25 +03:00
Ruslan Kuprieiev 3e183d62ed deps: bump dvc-* plugins to 2.19.0 
These are the same packages but with correct versions to sync up with existing packages that we generated in conda over the years.
 2022-08-25 14:44:13 +03:00
Ruslan Kuprieiev 460d08057c Create hook-fsspec.py 2022-08-24 23:39:46 +03:00
Ruslan Kuprieiev fc509d4e4a Delete hook-googleapiclient.model.py 
This is already provided by pyinstaller's standard hooks https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py
 2022-08-18 21:07:57 +03:00
Ruslan Kuprieiev a67fea2235 doctor: ignore missing plugins 2022-08-18 21:02:41 +03:00
Ruslan Kuprieiev a79ea15827 tests: webhdfs: fix linter 2022-08-18 12:43:12 +03:00
Ruslan Kuprieiev 8db9a2f70f deps: bump dvc-data to 0.1.19 2022-08-18 12:43:12 +03:00
Ruslan Kuprieiev 3fc9f323a7 fs: don't import SSHFileSystem 2022-08-18 00:23:40 +03:00
Ruslan Kuprieiev bf0aa5e8aa repo: data: ignore pylint false-positives 2022-08-17 23:02:22 +03:00
Ruslan Kuprieiev 3b3866323e deps: bump dvc-data to 0.1.18 2022-08-17 23:02:22 +03:00
Ruslan Kuprieiev 3c3cbee7fd deps: bump dvc-http to 0.0.2 2022-08-16 22:31:05 +03:00
Ruslan Kuprieiev 4a95020155 gs: add endpointurl 2022-08-14 13:39:03 +03:00
Ruslan Kuprieiev 3a429a869e tests: lockfile: use fs_path 2022-08-13 23:09:21 +03:00
Ruslan Kuprieiev 99465b25b5 deps: bump dvc-data to 0.1.17 2022-08-13 22:43:46 +03:00
Ruslan Kuprieiev fcdad4d229 deps: bump dvc-data to 0.1.16 2022-08-13 22:25:24 +03:00
Ruslan Kuprieiev 2e8b463795 deps: bump dvc-data 2022-08-12 11:58:44 +03:00
Ruslan Kuprieiev 704191b15a output: resolve relative to stage.wdir only on the same fs 
Leftover from https://github.com/iterative/dvc/pull/8091
 2022-08-04 19:25:31 +03:00
Ruslan Kuprieiev 0ce4e769f1 repo: make lock/rwlock kinda work on non local fs 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev 2c3655f7ff repo: use fs.makedirs instead of os.makedirs 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev 52bdd90840 repo: support basic workflow on s3fs/adlfs/etc 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev fcc7240617 odb: use odb.path instead of odb.cache_dir 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev ecd98bdf7d odbmgr: separate repo and local odbs 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev e525d9bedb deps: bump dvc-data to 0.1.11 2022-08-03 21:10:28 +03:00
Ruslan Kuprieiev 496599518a deps: bump dvc-data to 0.1.9 2022-08-02 23:57:34 +03:00
Ruslan Kuprieiev 6a7b55fdf7 datafs: use dvc-data's datafs 2022-08-02 03:13:06 +03:00
Ruslan Kuprieiev 4b965c3f9f fs: gdrive: add gdrive_acknowledge_abuse 
Fixes https://github.com/iterative/dvc/issues/7832
 2022-07-22 14:58:45 +03:00
Ruslan Kuprieiev 7a4886386f deps: bump dvc-data to 0.1.4 2022-07-21 22:53:26 +03:00
Ruslan Kuprieiev ae143505e4 deps: bump dvc-data to 0.1.3 2022-07-21 19:44:38 +03:00
Ruslan Kuprieiev b9b2e5bbd9 Delete test_gdrive.py 
Moved to dvc-gdrive.
 2022-07-21 19:34:04 +03:00
Ruslan Kuprieiev 66420439c9 deps: bump dvc-data to 0.1.2 2022-07-21 19:16:21 +03:00
Ruslan Kuprieiev 94eec50d27 deps: bump dvc-data to 0.1.1 2022-07-19 14:55:28 +03:00
Ruslan Kuprieiev dd32ff6206 deps: bump dvc-data to 0.0.31 2022-07-19 04:54:34 +03:00
Ruslan Kuprieiev 52a9c58cc0 deps: bump dvc-data to 0.0.30 2022-07-19 04:36:41 +03:00
Ruslan Kuprieiev 01855b7dcf deps: bump dvc-data to 0.0.28 2022-07-18 21:46:22 +03:00
Ruslan Kuprieiev 38f10884a7 deps: bump dvc-data to 0.0.27 2022-07-18 19:15:02 +03:00
Ruslan Kuprieiev 9e6d2163df deps: bump dvc-data to 0.0.26 2022-07-18 01:03:06 +03:00
Ruslan Kuprieiev d8be68436f deps: bump dvc-data to 0.0.23 2022-07-10 18:10:45 +03:00
Ruslan Kuprieiev f6adc49089 deps: bump dvc-data to 0.0.22 2022-07-10 17:05:26 +03:00
Ruslan Kuprieiev 8b3020aaf3 deps: bump dvc-data to 0.0.20 2022-07-08 21:37:38 +03:00
Ruslan Kuprieiev bd93d85d47 deps: bump dvc-data 2022-07-05 21:29:34 +03:00
Ruslan Kuprieiev eed6a8483c data_cloud: remove logger check 
Per https://github.com/iterative/dvc/pull/7961/files#r913635687
 2022-07-05 13:22:23 +03:00
Ruslan Kuprieiev 0d71194741 dvc: normalize targets before entering brancher 
Fixes #7888
 2022-07-04 16:34:49 +03:00
Ruslan Kuprieiev f42beb4433 pyinstaller: add note about frozen version 2022-06-30 15:25:24 +03:00
Ruslan Kuprieiev d448eb5fc1 setup: bump dvc-data 2022-06-26 19:40:31 +03:00
Ruslan Kuprieiev 286fab4aab setup: bump dvc-data to 0.0.12 2022-06-22 00:41:29 +03:00
Ruslan Kuprieiev ea1c54f48e setup: bump dvc-data to 0.0.11 2022-06-20 11:55:50 +03:00
Ruslan Kuprieiev 1253e6cc46 setup: bump dvc-data to 0.0.10 2022-06-19 21:16:08 +03:00
Ruslan Kuprieiev 19f2186c8b setup: bump dvc-data to 0.0.9 2022-06-18 22:34:25 +03:00
Ruslan Kuprieiev 2e21c92c4f dvc: use dvc-data 0.0.8 2022-06-17 22:08:33 +03:00
Ruslan Kuprieiev b9c04a247a dvc: upgrade to dvc-data 0.0.5 2022-06-13 18:03:53 +05:45
Ruslan Kuprieiev ef689c5593 dvc: upgrade to dvc-objects 0.0.4 2022-06-13 18:03:53 +05:45
Ruslan Kuprieiev 9ac2f49b57 Revert "fix dvc-objects/data imports" 
This reverts commit 677da2b22c.
 2022-06-10 12:01:07 +03:00
Ruslan Kuprieiev 9b2adfd3cc Revert "deps: bump dvc-data to 0.0.3" 
This reverts commit 6dd3f1e053.
 2022-06-10 12:01:07 +03:00
Ruslan Kuprieiev 8fa0e40669 setup: set upper bound on networkx 2022-06-06 13:31:59 +03:00
Ruslan Kuprieiev a3d6b1299a brancher: use scm.root_dir to determine relative cwd 
Fixes #7776
 2022-06-06 12:42:22 +03:00
Ruslan Kuprieiev 2b6aa01392 setup: bump scmrepo to 0.0.24 
Fixes #7824
 2022-06-01 20:51:08 +03:00
Ruslan Kuprieiev 844f77b4f9 dvc: drop python 3.7 
Also upgraded with `pyupgrade --py38-plus`

Fixes #7708
 2022-05-30 16:51:35 +03:00
Ruslan Kuprieiev 71a4b4823a fs: path: don't overuse parts 
These days we have `self.flavour` that is more efficient and more simple.
 2022-05-17 22:29:34 +03:00
Ruslan Kuprieiev 01cc97f38a fs: path: use flavour.basename 2022-05-17 20:21:46 +03:00
Ruslan Kuprieiev 005d56eedf fs: rename RepoFileSystem to DvcFileSystem 2022-05-12 00:39:41 +03:00
Ruslan Kuprieiev b6d25825c8 fs: rename DvcFileSystem to DataFileSystem 
Current naming for `DvcFileSystem` and `RepoFileSystem` is confusing, because
dvc is not only about data and it is much more natural to expect something
named `dvcfs` to include complete view for your whole repo (including git
files), which is what `repofs` is doing right now.

`DvcFileSystem` can provide a view not only for the repo data, but also for
external data (e.g. local external outputs). So it makes more sense to call
it a `DataFileSystem`, since it really represents the data management side
of dvc.
 2022-05-11 21:49:57 +03:00
Ruslan Kuprieiev a80a85e6bd dvc: detach gitfs from local paths 
Our gitfs paths are currently tied to the local path in which the cloned
repository resides in, which results in a lot of hacky assumptions throughout
the code.

This PR detaches gitfs from local paths completely, making dvc work with a
proper fsspec gitfs implementation.

Part of #7543
 2022-05-11 18:17:39 +03:00
Ruslan Kuprieiev b14a972cd0 repofs: use / as root_marker 
This makes dvcfs/repofs much easier to work with and allows it to support
isabs/abspath/etc correctly, as previously any full path (with root_marker "")
would be impossible to distinguish from a relative one.

This also detaches external workspaces from the main repo workspace in
DvcFileSystem, which was necessary to be able to distinguish dvcfs abspaths
from local external output paths. Note that currently we still have limited
support for those (e.g. get/api supports them, but import doesn't) and this
change keeps it that way.
 2022-05-08 17:19:53 +03:00
Ruslan Kuprieiev 58d960a47c Revert "fs.utils: add upload that can be conditionally atomic" 
This reverts commit dd187df667.
 2022-04-30 17:34:39 +03:00
Ruslan Kuprieiev 8cdcf9b8bb utils: remove PathStringTrie 2022-04-27 22:40:38 +03:00
Ruslan Kuprieiev 55a4532f1d ignore: use Trie instead of PathStringTrie 2022-04-27 22:40:38 +03:00
Ruslan Kuprieiev 43b9cab01b repofs: use Trie instead of PathStringTrie 2022-04-27 22:40:38 +03:00
Ruslan Kuprieiev fd440df5b3 dvcignore: use fs.path instead of os.path 2022-04-20 12:27:26 +03:00
Ruslan Kuprieiev 55219e9089 fs: get rid of CallbackMixin in azure/gs/hdfs/webhdfs 
No longer needed, as all of those filesystems already support callbacks.
 2022-04-18 17:34:06 +03:00
Ruslan Kuprieiev 517f45d45f repofs: migrate to fsspec 2022-04-18 14:17:20 +03:00
Ruslan Kuprieiev 53c7f44480 repofs: rename _main_repo -> repo 2022-04-18 14:17:20 +03:00
Ruslan Kuprieiev 9d75e8261b repofs: use repo paths 2022-04-18 14:17:20 +03:00
Ruslan Kuprieiev b3b0ec1fba repofs: unify ls/walk/info/exists/etc behaviour 2022-04-18 14:17:20 +03:00
Ruslan Kuprieiev 0a3f5cbdd5 repofs: base walk on ls 2022-04-18 14:17:20 +03:00
Ruslan Kuprieiev bdd21fd481 repofs: use fs.ls 2022-04-18 14:17:20 +03:00
Ruslan Kuprieiev 394118ef3d repofs: don't assume repofs paths are the same as underlying fs paths 2022-04-18 14:17:20 +03:00