Jorge Orpinel
654525cdde
add: fix test with ` around DVC command
2019-12-20 19:29:29 -07:00
Ruslan Kuprieiev
477488ea0a
Merge pull request #2950 from mroutis/tests-repo
...
tests: use dir_helpers on repo
2019-12-17 22:20:31 +02:00
Ruslan Kuprieiev
825e9fc74c
Merge pull request #2952 from mroutis/tests-updater
...
tests: use dir helpers on updater
2019-12-17 22:18:18 +02:00
Mr. Outis
7dedd7f518
tests/repo: bring back the old branch logic
2019-12-17 12:12:57 -06:00
Mr. Outis
d2469f7a1f
tests/repo: str -> fspath
2019-12-17 12:04:37 -06:00
Mr. Outis
a83e440b32
tests/repo: remove unnecessary os.path.join
2019-12-17 11:56:48 -06:00
Mr. Outis
0de183de26
tests/repo: reload cache properly
2019-12-17 11:55:47 -06:00
pawel
ffe0071647
import: test: refactor to dir helpers
2019-12-17 13:57:46 +01:00
Ruslan Kuprieiev
85b1721de8
Merge pull request #2866 from pared/2473
...
remote: s3: adjust jobs number basing on file descriptors number
2019-12-17 14:20:36 +02:00
Ruslan Kuprieiev
032d094239
Merge pull request #2889 from Baranowski/import_regulars.squashed
...
import: allow downloading regular files/dirs tracked by git
2019-12-17 14:09:58 +02:00
Paweł Redzyński
53163366b6
remote: s3: revert jobs adjustment, pass OSError to main
2019-12-17 11:41:10 +01:00
pawel
6ba4aba419
remote: base: [down/up]load: raise on too many open files
2019-12-17 11:41:03 +01:00
Mr. Outis
5194543007
tests/repo: windows compat
2019-12-16 10:25:07 -06:00
Mr. Outis
59072f4d18
💅 revert black upgrade
2019-12-16 09:44:59 -06:00
Mr. Outis
a7525fca64
💅 black and flake
2019-12-16 08:13:54 -06:00
Mr. Outis
6ed61efc05
tests/repo: refactor some tests
2019-12-16 08:08:23 -06:00
Mr. Outis
08b335d2b5
tests/repo: move unit tests to func tests
2019-12-16 08:04:28 -06:00
prd
d40588aefa
temporary_windows_drive fixture: move to dir_helpers
2019-12-16 13:15:53 +01:00
Ruslan Kuprieiev
39a9fd8d79
Merge pull request #2951 from mroutis/tests-analytics
...
tests: use dir helpers on analytics
2019-12-16 14:41:28 +02:00
Mr. Outis
f209e1eb6f
tests/analytics: explicitly set content for foo
2019-12-15 13:11:40 -06:00
Wojciech Baranowski
ad98f0fb50
Use fspath() in tests
2019-12-15 18:21:11 +02:00
Wojciech Baranowski
59156021de
Revert "test_status_imported_git_file"
...
This reverts commit 4d59bcddfd
2019-12-14 19:39:05 +02:00
Wojciech Baranowski
4d59bcddfd
test_status_imported_git_file
2019-12-14 19:38:57 +02:00
Wojciech Baranowski
11d890a4fa
Wrap tmp_dir in str()
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
c8c285536f
os.path.isdir(str(...))
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
5ceaf5a6f0
os.path.isfile(str(...))
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
0ee6998a29
New tests use tmp_dir
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
c04f5a0f9d
import: allow downloading regular files/dirs tracked by git
...
Fixes #2862
2019-12-14 14:08:38 +02:00
Mr. Outis
299fb9022f
💅 black
2019-12-13 18:04:15 -06:00
Mr. Outis
37a2844a74
tests/repo: move .stages to unit
2019-12-13 18:01:11 -06:00
Mr. Outis
b3a52515a3
tests/repo: move .collect to unit
2019-12-13 17:54:05 -06:00
Mr. Outis
655b79bf0c
tests/repo: move .destroy to unit test
2019-12-13 17:51:54 -06:00
Ruslan Kuprieiev
e88856dc16
Merge pull request #2943 from efiop/2942
...
remote: make `copy` links atomic for local/hdfs/ssh
2019-12-13 20:57:46 +02:00
Ruslan Kuprieiev
da49badbfe
Merge branch 'master' into relink
2019-12-13 20:35:02 +02:00
Ruslan Kuprieiev
e384685715
dvc: release locks when running a command
...
Currently only one instance of dvc could run in a particular repo
because of the `.dvc/lock`. That gets furstrating when you are running
some long-running command and blocked from doing anything else.
This PR solved that issue for `dvc run` by releasing the repo lock
while running the command. To protect against conflicts, we now use
rwlock, which keeps the list of readers and writers for particular
paths. The process looks like this:
1) aquire read locks for deps;
2) aquire write locks for outs;
3) release repo lock;
4) run cmd;
5) aquire repo lock;
6) release write locks for outs;
7) release read locks for outs;
We also take rwlocks for operations such as `import`, `add` and `checkout`
to ensure that we won't hit the same files that are being used by
another `dvc run` in the background.
We don't aquire rwlocks for the stage file itself, as we don't really
care about anyone overwriting a stage file, as there is a very little
difference here between parallel runs and sequential runs that overwrite
the same stage file.
Next steps might be to release repo lock on `import`(when dowloading
stuff) and when computing checksums or copying files.
Fixes #755
2019-12-13 10:38:10 +02:00
Ruslan Kuprieiev
b8bcff5e06
remote: make copy links atomic for local/hdfs/ssh
...
Fixes #2942
2019-12-13 09:37:47 +02:00
Mr. Outis
c09c3aec0c
remove empty file
2019-12-12 19:05:31 -06:00
Mr. Outis
d0f3559d96
tests: updater, use dir helpers
2019-12-12 18:59:52 -06:00
Mr. Outis
7426a62baa
tests: analytics, use dir helpers
2019-12-12 18:16:10 -06:00
Ruslan Kuprieiev
fd62ba840d
Merge pull request #2934 from pared/2896_checkout
...
checkout: test: refactor to dir helpers
2019-12-12 04:42:31 +02:00
pawel
5f826dd2e6
commit: test: refactor to dir helpers
2019-12-11 14:23:44 +01:00
pawel
26ef77cf96
checkout: test: refactor to dir helpers
2019-12-11 13:54:44 +01:00
Ruslan Kuprieiev
d2f40703cd
Merge pull request #2926 from pared/2896_test_move
...
test: move: convert to dir helpers
2019-12-11 02:44:14 +02:00
Ruslan Kuprieiev
6bf8d487da
Merge pull request #2921 from pared/2896_add_3
...
add: test: convert to dir helpers 3
2019-12-10 19:58:15 +02:00
Paweł Redzyński
267ac9b3ff
test: move: convert to dir helpers
2019-12-09 19:27:28 +01:00
Alexander Schepanovski
9607e5f873
get: refactor, clean up and fix dvc get implementation
...
Things changed:
- both no output and no git file result into PathMissingError now
- fail on missing cache or cache download error loudly
- removed sqlite bypass, since we use nolock now anyway
Refactoring:
- streamlined logic and made it similar to Repo.open() one
- moved exclusive exception classes to dvc.repo.get
2019-12-09 21:02:49 +03:00
Ruslan Kuprieiev
26a9702a4a
Merge pull request #2837 from danihodovic/feat/2515
...
get: copy/download files tracked by Git
2019-12-09 18:57:52 +02:00
Ruslan Kuprieiev
8890daf703
Merge pull request #2826 from mroutis/refactor-analytics
...
analytics: refactor into a module
2019-12-09 17:59:51 +02:00
pawel
c4565ce2ac
add: test: convert to dir helpers 3
...
TestAddUnsupportedFile
TestAdd
TestAddDirectory
2019-12-09 16:44:56 +01:00
Ruslan Kuprieiev
24fc8beaeb
Update test_ignore.py
2019-12-09 15:27:54 +02:00
Jorge Orpinel
Ruslan Kuprieiev
Mr. Outis
pawel
Wojciech Baranowski
Alexander Schepanovski