Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-05 23:09:32 +01:00
Code Issues Packages Projects Releases Wiki Activity
4,492 Commits 7 Branches 579 Tags
0.78.0
Commit Graph
1202 Commits
Author SHA1 Message Date
Jorge Orpinel 654525cdde add: fix test with ` around DVC command 2019-12-20 19:29:29 -07:00
Ruslan Kuprieiev 477488ea0a Merge pull request #2950 from mroutis/tests-repo 
tests: use dir_helpers on repo
 2019-12-17 22:20:31 +02:00
Ruslan Kuprieiev 825e9fc74c Merge pull request #2952 from mroutis/tests-updater 
tests: use dir helpers on updater
 2019-12-17 22:18:18 +02:00
Mr. Outis e519464aca 💅 black 2019-12-17 12:21:50 -06:00
Mr. Outis 8a14700c12 tests/updater: fix problems with .check 2019-12-17 12:18:43 -06:00
Mr. Outis 7dedd7f518 tests/repo: bring back the old branch logic 2019-12-17 12:12:57 -06:00
Mr. Outis d2469f7a1f tests/repo: str -> fspath 2019-12-17 12:04:37 -06:00
Mr. Outis 1bd66fa859 tests/repo: old params in is_dvc_internal 2019-12-17 12:03:35 -06:00
Mr. Outis a83e440b32 tests/repo: remove unnecessary os.path.join 2019-12-17 11:56:48 -06:00
Mr. Outis 0de183de26 tests/repo: reload cache properly 2019-12-17 11:55:47 -06:00
pawel ffe0071647 import: test: refactor to dir helpers 2019-12-17 13:57:46 +01:00
Ruslan Kuprieiev 85b1721de8 Merge pull request #2866 from pared/2473 
remote: s3: adjust jobs number basing on file descriptors number
 2019-12-17 14:20:36 +02:00
Ruslan Kuprieiev 032d094239 Merge pull request #2889 from Baranowski/import_regulars.squashed 
import: allow downloading regular files/dirs tracked by git
 2019-12-17 14:09:58 +02:00
Paweł Redzyński 53163366b6 remote: s3: revert jobs adjustment, pass OSError to main 2019-12-17 11:41:10 +01:00
pawel 6ba4aba419 remote: base: [down/up]load: raise on too many open files 2019-12-17 11:41:03 +01:00
pawel d253f1319d remote: upload/download: handle OSError 2019-12-17 11:41:03 +01:00
pawel a0eda4c884 remote: s3: reading ulimit/handles limit: support windows 2019-12-17 11:41:03 +01:00
pawel 710c4bd32e s3: test: mocker instead of mock annotations 2019-12-17 11:41:03 +01:00
pawel c167eb2675 remote: s3: adjust jobs only on default 2019-12-17 11:41:03 +01:00
Ruslan Kuprieiev bdfeba8f9b Merge pull request #2958 from MaxRis/gdrive-tests 
Standardize gdrive tests with others remotes
 2019-12-16 20:52:46 +02:00
Mr. Outis 5194543007 tests/repo: windows compat 2019-12-16 10:25:07 -06:00
Mr. Outis 59072f4d18 💅 revert black upgrade 2019-12-16 09:44:59 -06:00
Mr. Outis a7525fca64 💅 black and flake 2019-12-16 08:13:54 -06:00
Mr. Outis 6ed61efc05 tests/repo: refactor some tests 2019-12-16 08:08:23 -06:00
Mr. Outis 08b335d2b5 tests/repo: move unit tests to func tests 2019-12-16 08:04:28 -06:00
prd d40588aefa temporary_windows_drive fixture: move to dir_helpers 2019-12-16 13:15:53 +01:00
Ruslan Kuprieiev 39a9fd8d79 Merge pull request #2951 from mroutis/tests-analytics 
tests: use dir helpers on analytics
 2019-12-16 14:41:28 +02:00
Mr. Outis f209e1eb6f tests/analytics: explicitly set content for foo 2019-12-15 13:11:40 -06:00
Mr. Outis 4b833f58e5 tests/updater: use skipif for .check 2019-12-15 13:09:23 -06:00
Wojciech Baranowski ad98f0fb50 Use fspath() in tests 2019-12-15 18:21:11 +02:00
Max Risuhin 4d4a3fc716 Standardize gdrive tests with others remotes 2019-12-15 05:31:52 -08:00
Wojciech Baranowski 59156021de Revert "test_status_imported_git_file" 
This reverts commit 4d59bcddfd.
 2019-12-14 19:39:05 +02:00
Wojciech Baranowski 4d59bcddfd test_status_imported_git_file 2019-12-14 19:38:57 +02:00
Wojciech Baranowski 11d890a4fa Wrap tmp_dir in str() 2019-12-14 14:08:38 +02:00
Wojciech Baranowski c8c285536f os.path.isdir(str(...)) 2019-12-14 14:08:38 +02:00
Wojciech Baranowski 5ceaf5a6f0 os.path.isfile(str(...)) 2019-12-14 14:08:38 +02:00
Wojciech Baranowski 0ee6998a29 New tests use tmp_dir 2019-12-14 14:08:38 +02:00
Wojciech Baranowski c04f5a0f9d import: allow downloading regular files/dirs tracked by git 
Fixes #2862
 2019-12-14 14:08:38 +02:00
Mr. Outis df9f3de231 tests/repo: stringify stage.relpath 2019-12-13 19:15:25 -06:00
Mr. Outis 299fb9022f 💅 black 2019-12-13 18:04:15 -06:00
Mr. Outis 37a2844a74 tests/repo: move .stages to unit 2019-12-13 18:01:11 -06:00
Mr. Outis b3a52515a3 tests/repo: move .collect to unit 2019-12-13 17:54:05 -06:00
Mr. Outis 655b79bf0c tests/repo: move .destroy to unit test 2019-12-13 17:51:54 -06:00
Mr. Outis c178e77ab7 tests/repo: use pytest 2019-12-13 17:51:54 -06:00
Ruslan Kuprieiev e88856dc16 Merge pull request #2943 from efiop/2942 
remote: make `copy` links atomic for local/hdfs/ssh
 2019-12-13 20:57:46 +02:00
Ruslan Kuprieiev da49badbfe Merge branch 'master' into relink 2019-12-13 20:35:02 +02:00
Ruslan Kuprieiev 3c835909f6 Merge pull request #2931 from rbvesb/dvc-s3-acl 
remote: Extend S3 remote support for custom ACL
 2019-12-13 19:31:44 +02:00
Veselin Borisov cd461aec4f fixup! remote: Extend S3 remote support for custom ACL 2019-12-13 10:43:38 +02:00
Ruslan Kuprieiev e384685715 dvc: release locks when running a command 
Currently only one instance of dvc could run in a particular repo
because of the `.dvc/lock`. That gets furstrating when you are running
some long-running command and blocked from doing anything else.
This PR solved that issue for `dvc run` by releasing the repo lock
while running the command. To protect against conflicts, we now use
rwlock, which keeps the list of readers and writers for particular
paths. The process looks like this:

1) aquire read locks for deps;
2) aquire write locks for outs;
3) release repo lock;
4) run cmd;
5) aquire repo lock;
6) release write locks for outs;
7) release read locks for outs;

We also take rwlocks for operations such as `import`, `add` and `checkout`
to ensure that we won't hit the same files that are being used by
another `dvc run` in the background.

We don't aquire rwlocks for the stage file itself, as we don't really
care about anyone overwriting a stage file, as there is a very little
difference here between parallel runs and sequential runs that overwrite
the same stage file.

Next steps might be to release repo lock on `import`(when dowloading
stuff) and when computing checksums or copying files.

Fixes #755
 2019-12-13 10:38:10 +02:00
Ruslan Kuprieiev b8bcff5e06 remote: make copy links atomic for local/hdfs/ssh 
Fixes #2942
 2019-12-13 09:37:47 +02:00