Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-08 23:09:48 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,782 Commits 7 Branches 579 Tags
0.81.2
Commit Graph
295 Commits
Author SHA1 Message Date
Mr. Outis 2c9b763906 version: decode correctly commands output 2020-01-13 12:36:47 -06:00
Saugat Pachhai 141f308a6a test: refactor ssh related functions in SSH and SSHMocked test helpers 2020-01-13 14:23:42 +05:45
Max Risuhin 57111a7788 gdrive: add support for teamDriveId 2020-01-11 07:41:12 -08:00
Ruslan Kuprieiev f6a00f35ae ssh: fix bug in remove() 2020-01-11 03:14:45 +02:00
Aman Sharma 74850fe728 Move walk_files to fs.py 2020-01-09 14:55:49 +05:30
Aman Sharma a79816920d Move copyfile to fs.py 2020-01-09 04:57:53 +05:30
Aman Sharma eca5d2dde4 Move makedirs to fs.py 2020-01-09 04:30:51 +05:30
Ruslan Kuprieiev 049367e475 Merge pull request #3034 from mroutis/remove-py2-comments 
py3: remove code related to py2 limitations
 2020-01-08 20:14:40 +02:00
Ruslan Kuprieiev ee0a8fa92d Merge pull request #3075 from efiop/assert_stage 
rwlock: check that .dvc/lock is locked
 2020-01-08 00:44:32 +02:00
Mr. Outis 71b15d259a py3: remove code related to py2 limitations 2020-01-06 23:03:33 -06:00
Ruslan Kuprieiev 11fa9204f8 Merge pull request #3058 from efiop/too_many_files 
ui: don't show exception for "Too many files" error
 2020-01-07 06:51:29 +02:00
Aman Sharma 13ca5bdf28 Write test for walk_files to check if it accepts Path-like and str objects 2020-01-07 05:50:48 +05:30
Ruslan Kuprieiev 1dcce1d8b0 rwlock: check that .dvc/lock is locked 
Mostly a sanity check, but we did have a potential bug in `destroy`,
whcih this PR fixes along the way.

Kudos @shcheklein for catching this one.
 2020-01-06 22:34:34 +02:00
Ruslan Kuprieiev 40449c49b1 Merge pull request #3070 from pared/2896_http_imp 
http/ imp tests - migrate do dir helpers
 2020-01-06 20:38:37 +02:00
pawel e1176bf42d imp/imp_url cmd test: remove unnecessary dvc fixture 2020-01-06 16:40:15 +01:00
pawel a620658884 http/ imp tests - migrate do dir helpers 2020-01-06 16:18:01 +01:00
pawel bb72ce8a50 analytics: tests: migrate to dir helpers 2020-01-06 15:49:26 +01:00
Ruslan Kuprieiev 6411a3bb32 ui: don't show exception for "Too many files" error 
https://github.com/iterative/dvc.org/pull/875#discussion_r362920851

Stops showing the exception in non-verbose mode, but shows full
traceback in verbose mode.

Introducing our custom parameter `tb_only` through standard `extra`
logging parameter.
 2020-01-06 01:02:59 +02:00
Mr. Outis 5611ccb43a exceptions: remove cause 2020-01-02 20:43:24 -06:00
Aman Sharma f08c90fc6e Ensure relpath accepts str and Path-like objects 2020-01-02 02:17:02 +05:30
Mr. Outis 086acf6404 py3: dvc/utils/compat.py -> dvc/compat.py 2019-12-31 23:42:07 -06:00
Mr. Outis 27f581cf4e py3: remove imports from utils.compat 2019-12-31 23:16:11 -06:00
Mr. Outis b844aea42c py3: remove cast_bytes method 2019-12-31 20:14:25 -06:00
Mr. Outis 6d36b85736 py3: remove string compat related code 2019-12-30 23:49:19 -06:00
Mr. Outis 94a0914bba py3: get rid of future imports 2019-12-30 19:35:48 -06:00
pawel cf2d349bd7 tree: move CleanTree to ignore 2019-12-30 14:19:43 +01:00
pawel d8c0ae6909 remove most occurences of dvc_walk 2019-12-30 14:19:43 +01:00
pawel f4ee01cffe no need for dvcignore in walk args 2019-12-30 14:19:43 +01:00
Ruslan Kuprieiev ef6018fc2f Merge pull request #3002 from efiop/2458 
dvc: support granularity for fetch/pull/push/status/checkout
 2019-12-30 07:26:57 +02:00
Ruslan Kuprieiev c81795c047 dvc: support granularity for fetch/pull/push/status/checkout 
Using output path or a subdir/subfile path within an output now works
with `dvc push/pull/fetch/status -c` commands. Other commands don't
support the same logic for now, as there are some questions about what
should commands like `dvc remove` do when given a specific output path.

Example:
    dvc add data
    dvc pull data/subdir # will only pull files within data/subdir

Related to #2458
 2019-12-29 01:31:08 +02:00
Max Risuhin 4727cb7893 Improve Google Drive auth failure error reporting 2019-12-28 05:22:37 -08:00
Ruslan Kuprieiev 477488ea0a Merge pull request #2950 from mroutis/tests-repo 
tests: use dir_helpers on repo
 2019-12-17 22:20:31 +02:00
Ruslan Kuprieiev 825e9fc74c Merge pull request #2952 from mroutis/tests-updater 
tests: use dir helpers on updater
 2019-12-17 22:18:18 +02:00
Mr. Outis e519464aca 💅 black 2019-12-17 12:21:50 -06:00
Mr. Outis 8a14700c12 tests/updater: fix problems with .check 2019-12-17 12:18:43 -06:00
Mr. Outis 1bd66fa859 tests/repo: old params in is_dvc_internal 2019-12-17 12:03:35 -06:00
Mr. Outis 08b335d2b5 tests/repo: move unit tests to func tests 2019-12-16 08:04:28 -06:00
Mr. Outis 4b833f58e5 tests/updater: use skipif for .check 2019-12-15 13:09:23 -06:00
Max Risuhin 4d4a3fc716 Standardize gdrive tests with others remotes 2019-12-15 05:31:52 -08:00
Mr. Outis df9f3de231 tests/repo: stringify stage.relpath 2019-12-13 19:15:25 -06:00
Mr. Outis 299fb9022f 💅 black 2019-12-13 18:04:15 -06:00
Mr. Outis 37a2844a74 tests/repo: move .stages to unit 2019-12-13 18:01:11 -06:00
Mr. Outis b3a52515a3 tests/repo: move .collect to unit 2019-12-13 17:54:05 -06:00
Mr. Outis 655b79bf0c tests/repo: move .destroy to unit test 2019-12-13 17:51:54 -06:00
Mr. Outis c178e77ab7 tests/repo: use pytest 2019-12-13 17:51:54 -06:00
Ruslan Kuprieiev da49badbfe Merge branch 'master' into relink 2019-12-13 20:35:02 +02:00
Ruslan Kuprieiev 3c835909f6 Merge pull request #2931 from rbvesb/dvc-s3-acl 
remote: Extend S3 remote support for custom ACL
 2019-12-13 19:31:44 +02:00
Veselin Borisov cd461aec4f fixup! remote: Extend S3 remote support for custom ACL 2019-12-13 10:43:38 +02:00
Ruslan Kuprieiev e384685715 dvc: release locks when running a command 
Currently only one instance of dvc could run in a particular repo
because of the `.dvc/lock`. That gets furstrating when you are running
some long-running command and blocked from doing anything else.
This PR solved that issue for `dvc run` by releasing the repo lock
while running the command. To protect against conflicts, we now use
rwlock, which keeps the list of readers and writers for particular
paths. The process looks like this:

1) aquire read locks for deps;
2) aquire write locks for outs;
3) release repo lock;
4) run cmd;
5) aquire repo lock;
6) release write locks for outs;
7) release read locks for outs;

We also take rwlocks for operations such as `import`, `add` and `checkout`
to ensure that we won't hit the same files that are being used by
another `dvc run` in the background.

We don't aquire rwlocks for the stage file itself, as we don't really
care about anyone overwriting a stage file, as there is a very little
difference here between parallel runs and sequential runs that overwrite
the same stage file.

Next steps might be to release repo lock on `import`(when dowloading
stuff) and when computing checksums or copying files.

Fixes #755
 2019-12-13 10:38:10 +02:00
Mr. Outis d0f3559d96 tests: updater, use dir helpers 2019-12-12 18:59:52 -06:00