Mr. Outis
4a25612282
💅 refactor
2019-12-25 01:04:52 -06:00
Mr. Outis
75ad14aaa2
💅 black
2019-12-25 00:43:33 -06:00
Mr. Outis
0658622ca2
summon: pair-programming adjustments
2019-12-25 00:42:55 -06:00
Mr. Outis
1e65c76ef1
summon: raise SummonError appropiately
2019-12-24 20:20:03 -06:00
Mr. Outis
fbb7aebd31
tests/summon: use different args for non-canon.yml
2019-12-24 19:50:43 -06:00
Mr. Outis
1c22cd5af4
summon: adjust comments
2019-12-24 02:20:08 -06:00
Mr. Outis
addc56c232
summon: allow different YAML file names
2019-12-18 23:38:52 -06:00
Mr. Outis
c852d3316d
💅 black
2019-12-18 23:34:43 -06:00
Mr. Outis
4273ff49b3
summon: params -> args; use summon key in yaml
2019-12-18 23:32:28 -06:00
Mr. Outis
29339d29a2
summon: file & method -> call
2019-12-18 23:25:45 -06:00
Mr. Outis
1b3c771df5
summon: artifacs.yaml -> dvcsummon.yaml; artifact -> object
2019-12-18 22:42:55 -06:00
Mr. Outis
32f35ea686
summon: remove scm.add, *_gen already handles it
2019-12-18 22:29:44 -06:00
Mr. Outis
5d66a572d4
summon: support overriding parameters
2019-12-18 17:20:07 -06:00
Mr. Outis
d0344ce975
summon: grimoire/spells -> artifacts
2019-12-18 17:00:39 -06:00
Mr. Outis
789db9169a
summon: support checking out dependencies
2019-12-18 16:41:03 -06:00
Mr. Outis
b921730f37
summon: safely get a module; change naming
2019-12-18 13:32:23 -06:00
Mr. Outis
3d133186d4
summon: ugly hack
2019-12-18 00:48:10 -06:00
pawel
ffe0071647
import: test: refactor to dir helpers
2019-12-17 13:57:46 +01:00
Ruslan Kuprieiev
85b1721de8
Merge pull request #2866 from pared/2473
...
remote: s3: adjust jobs number basing on file descriptors number
2019-12-17 14:20:36 +02:00
Ruslan Kuprieiev
032d094239
Merge pull request #2889 from Baranowski/import_regulars.squashed
...
import: allow downloading regular files/dirs tracked by git
2019-12-17 14:09:58 +02:00
Paweł Redzyński
53163366b6
remote: s3: revert jobs adjustment, pass OSError to main
2019-12-17 11:41:10 +01:00
pawel
6ba4aba419
remote: base: [down/up]load: raise on too many open files
2019-12-17 11:41:03 +01:00
prd
d40588aefa
temporary_windows_drive fixture: move to dir_helpers
2019-12-16 13:15:53 +01:00
Ruslan Kuprieiev
39a9fd8d79
Merge pull request #2951 from mroutis/tests-analytics
...
tests: use dir helpers on analytics
2019-12-16 14:41:28 +02:00
Mr. Outis
f209e1eb6f
tests/analytics: explicitly set content for foo
2019-12-15 13:11:40 -06:00
Wojciech Baranowski
ad98f0fb50
Use fspath() in tests
2019-12-15 18:21:11 +02:00
Wojciech Baranowski
59156021de
Revert "test_status_imported_git_file"
...
This reverts commit 4d59bcddfd
2019-12-14 19:39:05 +02:00
Wojciech Baranowski
4d59bcddfd
test_status_imported_git_file
2019-12-14 19:38:57 +02:00
Wojciech Baranowski
11d890a4fa
Wrap tmp_dir in str()
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
c8c285536f
os.path.isdir(str(...))
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
5ceaf5a6f0
os.path.isfile(str(...))
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
0ee6998a29
New tests use tmp_dir
2019-12-14 14:08:38 +02:00
Wojciech Baranowski
c04f5a0f9d
import: allow downloading regular files/dirs tracked by git
...
Fixes #2862
2019-12-14 14:08:38 +02:00
Ruslan Kuprieiev
e88856dc16
Merge pull request #2943 from efiop/2942
...
remote: make `copy` links atomic for local/hdfs/ssh
2019-12-13 20:57:46 +02:00
Ruslan Kuprieiev
da49badbfe
Merge branch 'master' into relink
2019-12-13 20:35:02 +02:00
Ruslan Kuprieiev
e384685715
dvc: release locks when running a command
...
Currently only one instance of dvc could run in a particular repo
because of the `.dvc/lock`. That gets furstrating when you are running
some long-running command and blocked from doing anything else.
This PR solved that issue for `dvc run` by releasing the repo lock
while running the command. To protect against conflicts, we now use
rwlock, which keeps the list of readers and writers for particular
paths. The process looks like this:
1) aquire read locks for deps;
2) aquire write locks for outs;
3) release repo lock;
4) run cmd;
5) aquire repo lock;
6) release write locks for outs;
7) release read locks for outs;
We also take rwlocks for operations such as `import`, `add` and `checkout`
to ensure that we won't hit the same files that are being used by
another `dvc run` in the background.
We don't aquire rwlocks for the stage file itself, as we don't really
care about anyone overwriting a stage file, as there is a very little
difference here between parallel runs and sequential runs that overwrite
the same stage file.
Next steps might be to release repo lock on `import`(when dowloading
stuff) and when computing checksums or copying files.
Fixes #755
2019-12-13 10:38:10 +02:00
Ruslan Kuprieiev
b8bcff5e06
remote: make copy links atomic for local/hdfs/ssh
...
Fixes #2942
2019-12-13 09:37:47 +02:00
Mr. Outis
7426a62baa
tests: analytics, use dir helpers
2019-12-12 18:16:10 -06:00
Ruslan Kuprieiev
fd62ba840d
Merge pull request #2934 from pared/2896_checkout
...
checkout: test: refactor to dir helpers
2019-12-12 04:42:31 +02:00
pawel
5f826dd2e6
commit: test: refactor to dir helpers
2019-12-11 14:23:44 +01:00
pawel
26ef77cf96
checkout: test: refactor to dir helpers
2019-12-11 13:54:44 +01:00
Ruslan Kuprieiev
d2f40703cd
Merge pull request #2926 from pared/2896_test_move
...
test: move: convert to dir helpers
2019-12-11 02:44:14 +02:00
Ruslan Kuprieiev
6bf8d487da
Merge pull request #2921 from pared/2896_add_3
...
add: test: convert to dir helpers 3
2019-12-10 19:58:15 +02:00
Paweł Redzyński
267ac9b3ff
test: move: convert to dir helpers
2019-12-09 19:27:28 +01:00
Alexander Schepanovski
9607e5f873
get: refactor, clean up and fix dvc get implementation
...
Things changed:
- both no output and no git file result into PathMissingError now
- fail on missing cache or cache download error loudly
- removed sqlite bypass, since we use nolock now anyway
Refactoring:
- streamlined logic and made it similar to Repo.open() one
- moved exclusive exception classes to dvc.repo.get
2019-12-09 21:02:49 +03:00
Ruslan Kuprieiev
26a9702a4a
Merge pull request #2837 from danihodovic/feat/2515
...
get: copy/download files tracked by Git
2019-12-09 18:57:52 +02:00
Ruslan Kuprieiev
8890daf703
Merge pull request #2826 from mroutis/refactor-analytics
...
analytics: refactor into a module
2019-12-09 17:59:51 +02:00
pawel
c4565ce2ac
add: test: convert to dir helpers 3
...
TestAddUnsupportedFile
TestAdd
TestAddDirectory
2019-12-09 16:44:56 +01:00
Ruslan Kuprieiev
24fc8beaeb
Update test_ignore.py
2019-12-09 15:27:54 +02:00
Ruslan Kuprieiev
302b016d65
Merge pull request #2913 from Suor/migrate-test-ignore
...
test: migrate test_ignore
2019-12-09 15:25:46 +02:00
Mr. Outis
pawel
Ruslan Kuprieiev
Wojciech Baranowski
Alexander Schepanovski