pawel
6ba4aba419
remote: base: [down/up]load: raise on too many open files
2019-12-17 11:41:03 +01:00
pawel
d253f1319d
remote: upload/download: handle OSError
2019-12-17 11:41:03 +01:00
pawel
a0eda4c884
remote: s3: reading ulimit/handles limit: support windows
2019-12-17 11:41:03 +01:00
pawel
710c4bd32e
s3: test: mocker instead of mock annotations
2019-12-17 11:41:03 +01:00
pawel
c167eb2675
remote: s3: adjust jobs only on default
2019-12-17 11:41:03 +01:00
Ruslan Kuprieiev
bdfeba8f9b
Merge pull request #2958 from MaxRis/gdrive-tests
...
Standardize gdrive tests with others remotes
2019-12-16 20:52:46 +02:00
Mr. Outis
5194543007
tests/repo: windows compat
2019-12-16 10:25:07 -06:00
Mr. Outis
59072f4d18
💅 revert black upgrade
2019-12-16 09:44:59 -06:00
Mr. Outis
a7525fca64
💅 black and flake
2019-12-16 08:13:54 -06:00
Mr. Outis
6ed61efc05
tests/repo: refactor some tests
2019-12-16 08:08:23 -06:00
Mr. Outis
08b335d2b5
tests/repo: move unit tests to func tests
2019-12-16 08:04:28 -06:00
prd
d40588aefa
temporary_windows_drive fixture: move to dir_helpers
2019-12-16 13:15:53 +01:00
Ruslan Kuprieiev
39a9fd8d79
Merge pull request #2951 from mroutis/tests-analytics
...
tests: use dir helpers on analytics
2019-12-16 14:41:28 +02:00
Mr. Outis
f209e1eb6f
tests/analytics: explicitly set content for foo
2019-12-15 13:11:40 -06:00
Mr. Outis
4b833f58e5
tests/updater: use skipif for .check
2019-12-15 13:09:23 -06:00
Wojciech Baranowski
ad98f0fb50
Use fspath() in tests
2019-12-15 18:21:11 +02:00
Max Risuhin
4d4a3fc716
Standardize gdrive tests with others remotes
2019-12-15 05:31:52 -08:00
Wojciech Baranowski
59156021de
Revert "test_status_imported_git_file"
...
This reverts commit 4d59bcddfd .
2019-12-14 19:39:05 +02:00
Wojciech Baranowski
4d59bcddfd
test_status_imported_git_file
2019-12-14 19:38:57 +02:00
Wojciech Baranowski
11d890a4fa
Wrap tmp_dir in str()
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
c8c285536f
os.path.isdir(str(...))
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
5ceaf5a6f0
os.path.isfile(str(...))
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
0ee6998a29
New tests use tmp_dir
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
c04f5a0f9d
import: allow downloading regular files/dirs tracked by git
...
Fixes #2862
2019-12-14 14:08:38 +02:00
Mr. Outis
df9f3de231
tests/repo: stringify stage.relpath
2019-12-13 19:15:25 -06:00
Mr. Outis
299fb9022f
💅 black
2019-12-13 18:04:15 -06:00
Mr. Outis
37a2844a74
tests/repo: move .stages to unit
2019-12-13 18:01:11 -06:00
Mr. Outis
b3a52515a3
tests/repo: move .collect to unit
2019-12-13 17:54:05 -06:00
Mr. Outis
655b79bf0c
tests/repo: move .destroy to unit test
2019-12-13 17:51:54 -06:00
Mr. Outis
c178e77ab7
tests/repo: use pytest
2019-12-13 17:51:54 -06:00
Ruslan Kuprieiev
e88856dc16
Merge pull request #2943 from efiop/2942
...
remote: make `copy` links atomic for local/hdfs/ssh
2019-12-13 20:57:46 +02:00
Ruslan Kuprieiev
da49badbfe
Merge branch 'master' into relink
2019-12-13 20:35:02 +02:00
Ruslan Kuprieiev
3c835909f6
Merge pull request #2931 from rbvesb/dvc-s3-acl
...
remote: Extend S3 remote support for custom ACL
2019-12-13 19:31:44 +02:00
Veselin Borisov
cd461aec4f
fixup! remote: Extend S3 remote support for custom ACL
2019-12-13 10:43:38 +02:00
Ruslan Kuprieiev
e384685715
dvc: release locks when running a command
...
Currently only one instance of dvc could run in a particular repo
because of the `.dvc/lock`. That gets furstrating when you are running
some long-running command and blocked from doing anything else.
This PR solved that issue for `dvc run` by releasing the repo lock
while running the command. To protect against conflicts, we now use
rwlock, which keeps the list of readers and writers for particular
paths. The process looks like this:
1) aquire read locks for deps;
2) aquire write locks for outs;
3) release repo lock;
4) run cmd;
5) aquire repo lock;
6) release write locks for outs;
7) release read locks for outs;
We also take rwlocks for operations such as `import`, `add` and `checkout`
to ensure that we won't hit the same files that are being used by
another `dvc run` in the background.
We don't aquire rwlocks for the stage file itself, as we don't really
care about anyone overwriting a stage file, as there is a very little
difference here between parallel runs and sequential runs that overwrite
the same stage file.
Next steps might be to release repo lock on `import`(when dowloading
stuff) and when computing checksums or copying files.
Fixes #755
2019-12-13 10:38:10 +02:00
Ruslan Kuprieiev
b8bcff5e06
remote: make copy links atomic for local/hdfs/ssh
...
Fixes #2942
2019-12-13 09:37:47 +02:00
Mr. Outis
c09c3aec0c
remove empty file
2019-12-12 19:05:31 -06:00
Mr. Outis
d0f3559d96
tests: updater, use dir helpers
2019-12-12 18:59:52 -06:00
Mr. Outis
7426a62baa
tests: analytics, use dir helpers
2019-12-12 18:16:10 -06:00
Ruslan Kuprieiev
6ab2322cde
Merge pull request #2938 from Suor/fix-dir-helpers
...
test: a couple of dir helpers fixes
2019-12-12 23:20:46 +02:00
Alexander Schepanovski
1143a8c57f
test: add checkout parse-cli-api test
2019-12-12 20:16:17 +07:00
Veselin Borisov
4c2bc6bf48
fixup! remote: Extend S3 remote support for custom ACL
2019-12-12 12:41:38 +02:00
Ruslan Kuprieiev
fd62ba840d
Merge pull request #2934 from pared/2896_checkout
...
checkout: test: refactor to dir helpers
2019-12-12 04:42:31 +02:00
pawel
5f826dd2e6
commit: test: refactor to dir helpers
2019-12-11 14:23:44 +01:00
Alexander Schepanovski
b8cf906edb
test: stop erepo_dir from undoing parent chdir
2019-12-11 20:19:38 +07:00
Alexander Schepanovski
a1643d576d
test: fix run_copy fixture when used with scm
2019-12-11 20:19:02 +07:00
pawel
26ef77cf96
checkout: test: refactor to dir helpers
2019-12-11 13:54:44 +01:00
Ruslan Kuprieiev
53c33a22d9
Merge pull request #2915 from maykulkarni/import-external-repo-fix
...
use cache dir as a default remote when importing from local repo
2019-12-11 08:45:22 +02:00
Ruslan Kuprieiev
d2f40703cd
Merge pull request #2926 from pared/2896_test_move
...
test: move: convert to dir helpers
2019-12-11 02:44:14 +02:00
Ruslan Kuprieiev
6bf8d487da
Merge pull request #2921 from pared/2896_add_3
...
add: test: convert to dir helpers 3
2019-12-10 19:58:15 +02:00