Commit Graph
3688 Commits
Author SHA1 Message Date
Alexander Schepanovski 293a50d75c perf: use fast entry points on non-wheel install
This avoids importing `pkg_resources` on startup, which saves around 200
ms.

NOTE: wheel installer uses its own entry points template, which is the same
fastentrypoints.py script provides, so no gain there.
2019-09-12 22:33:59 +07:00
Paweł RedzyńskiandRuslan Kuprieiev 69f0de0090 google-resumable-media build problems (#2449)
* google-resumable-media hook for pyinstaller

* working wtih g-r-m build with -e

* update google-cloud-storage

* google-cloud-storage hook: remove hiddenimports

* Update setup.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
2019-09-12 12:12:37 +03:00
Ruslan Kuprieiev da6a0ca933 lock: remove unused lifetime argument 2019-09-11 16:45:09 +03:00
Ruslan Kuprieiev 05adcf9a4f Merge pull request #2461 from Suor/nfs
dvc: switch to flufl lock
2019-09-11 16:43:40 +03:00
Ruslan Kuprieiev 8338d4507c Merge pull request #2488 from pared/2444
remote: checkout: reuse _checkout_file functionality
2019-09-11 15:22:18 +03:00
pawel d4c9c71a2f remote: checkout: reuse _checkout_file functionality 2019-09-11 11:26:30 +02:00
Alexander Schepanovski 3c3378bdca dvc: make lock lifetime long by default 2019-09-11 14:21:12 +07:00
Alexander Schepanovski d9d85f3cb0 dvc: fix locks in Python2.7/Windows
Return to zc.lockfile lock in Python 2, NFS is not supported there
anyway since sqlite3 doesn't support URIs.
2019-09-11 14:17:21 +07:00
Alexander Schepanovski f29f07ca88 dvc: do not lock sqlite db 2019-09-11 14:17:16 +07:00
Ruslan Kuprieiev 09f6b1c7b9 Merge pull request #2485 from pared/2477
move: use relpath to find target path
2019-09-10 16:23:27 +03:00
pawel a16dc58ae3 move: use relpath to find target path 2019-09-10 14:03:55 +02:00
Ruslan Kuprieiev 9fa6bee13a Merge pull request #2482 from efiop/master
test: repro: fix link types
2019-09-10 03:10:29 +03:00
Ruslan Kuprieiev 384fd4cd9a test: repro: fix link types 2019-09-10 02:12:28 +03:00
Ruslan Kuprieiev 2c003fdc04 Merge pull request #2426 from mroutis/close-1652
ssh: support linking
2019-09-09 20:48:22 +03:00
Ruslan Kuprieiev af8a7aa25d dvc: bump to 0.59.2 0.59.2 2019-09-09 20:28:37 +03:00
Mr. Outis 3c5bcbd463 local: mark static methods 2019-09-09 12:26:48 -05:00
Ruslan Kuprieiev b0b958e14b Merge pull request #2481 from efiop/2476
scm: remove `pkg install` leftovers
2019-09-09 20:16:01 +03:00
Ruslan Kuprieiev a529f7d8ad Merge pull request #2479 from efiop/always_reproduce
run: introduce --always-changed
2019-09-09 19:45:35 +03:00
Ruslan Kuprieiev ce27444dd2 scm: remove pkg install leftovers
Leftover code was causing problems when working with symlinks, because
it tried to use `realpath` on symlinks to further use it to determine
parent directory.

Fixes #2476
2019-09-09 19:43:10 +03:00
Mr. Outis 2a847ef25c tests: ignore ssh's hardlink and copy tests on windows 2019-09-09 11:20:29 -05:00
Mr. Outis e0fd808ab0 setup: iterative/mock-ssh-server -> pypi mock-ssh-server 2019-09-09 11:15:49 -05:00
Ruslan Kuprieiev 9638b454d7 Merge pull request #2478 from pared/2443
stage: prevent writing default wdir into stage file
2019-09-09 18:23:36 +03:00
Ruslan Kuprieiev 357ce82f28 run: introduce --always-changed
This is an extension of our callback stages (no deps, so always
considered as changed) for stages with dependencies, so that these
stages could be used properly in the middle of the DAG. See attached
issue for more info.

Related to #2378
2019-09-09 18:03:28 +03:00
pawel 90738fab9c stage: prevent writing default wdir into stage file 2019-09-09 16:20:33 +02:00
Ruslan Kuprieiev b3e4ff7cba test: install: don't forget to adjust hook tests 2019-09-09 05:10:30 +03:00
Ruslan Kuprieiev 79cd3d670f dvc: bump to 0.59.1 2019-09-09 00:26:54 +03:00
Ruslan Kuprieiev 2d2ac67e50 hooks: make syntax compatible with bare sh
`==` is not part of the posix and not all shells (e.g. dash) support it, making hooks break.
2019-09-09 00:26:04 +03:00
Mr. OutisandRuslan Kuprieiev 5103dfd4aa tests: fix skipif on ssh/test_connection.py
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
2019-09-07 21:38:50 -05:00
Mr. Outis b4507444d5 version: adjust get_fs_type to discover all partitions 2019-09-06 19:27:23 -05:00
Ruslan Kuprieiev 2ee9da61f0 dvc: bump to 0.59.0 0.59.0 2019-09-07 00:13:05 +03:00
Ruslan Kuprieiev b61c97aa68 Merge pull request #2475 from efiop/2471
erepo: fix environment before running git
2019-09-07 00:12:35 +03:00
Ruslan Kuprieiev 60d8e8fbec erepo: fix environment before running git
Fixes #2471
2019-09-06 23:42:46 +03:00
Paweł Redzyński 0a214ae15a test: add: remove TestDvcGit (#2467) 2019-09-06 23:15:26 +03:00
Mr. Outis 190fbaa63b get: no relpath on output not found (#2435)
* get: no relpath on output not found

related #2430

* get: catch output not found
2019-09-06 23:14:07 +03:00
Mr. Outis 05cbe0c922 tests: support windows 2019-09-06 14:17:44 -05:00
Mr. Outis 737f1e141d tests: use repo_dir instead of custom fixtures 2019-09-06 13:28:48 -05:00
Mr. Outis b66996e0a1 ssh: support reflinks in Darwin 2019-09-06 13:24:25 -05:00
Mr. Outis b903fe8023 get: show error message when path is a DVC-file (#2470)
* get: show error message when path is a DVC-file

Fix #2468

* exceptions: rename output for data artifact

* get: use method to indetify if file is a DVC-file

* exceptions: change GetDVCFileError message
2019-09-06 21:11:23 +03:00
Mr. Outis dd1d5bdbb0 tests: PosixPath to string in chdir_tmp fixture 2019-09-06 12:57:06 -05:00
Mr. Outis 73133e8c7a tests: [re] fix hardlink stuff with HDFS 2019-09-06 12:35:38 -05:00
Mr. Outis 23f2650b24 ssh: remove write_through in TextIOWrapper 2019-09-06 12:07:22 -05:00
Mr. Outis f97c33178f tests: use copy as cache type for HDFS 2019-09-06 12:05:53 -05:00
Mr. Outis 6eb26de856 fix tests 2019-09-06 11:32:31 -05:00
Mr. Outis 38d4cbf5c1 fix broken merge 2019-09-06 09:42:56 -05:00
Mr. Outis 42168d34f3 Merge branch 'master' into close-1652 2019-09-06 09:29:54 -05:00
5ec1d274ae remote: base: don't checkout existing file (#2358)
* remote: base: don't checkout existing file

* remote: _checkout_file: split to smaller methods

* state: local: verify if file needs checkout considering link type and access rights

* tests: checkout: test avoiding unnecessary copying

* efiop review refactor

* remote: local: refactor _needs_checkout NOTE

* remote: local: remove bool cast for `protected`, wrap linking in try

* test: checkout: test relinking upon existing cache

* stage: remove obsolete unprotecing from Stage.create

* test: checkout: refactor setting link type

* test: checkout: parametrize relinking test

* remote: local: don't remove cache test file on cache_copy test

* remote: base: remove force from _needs_checkout arguments

* Update dvc/remote/local/__init__.py

Co-Authored-By: Alexander Schepanovski <[email protected]>

* suor review refactor

* remote: base: re-add note why do we unprotect on _checkout_file

* test: checkout: tests for relinking and resetting target protection

* remove checkout behavior change

* remove unused imports

* remote: base: reuse __checkout_file in __checkout_dir

* initial dir

* checkout: test: should relink on repeated add

* remote: cleanup commented lines

* remote: base/local: add save_link flag to _checkout_file

* remote: local: add allow_copy flag to unprotect method

* Update dvc/remote/local/__init__.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* remote: local: remove unnecessary removal warning on _checkout_file

* remote: local: proper cache link type detection

* test: checkout: move repeated add tests

* Update dvc/remote/local/__init__.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* remote: base: remove _needs_checkout

* Update dvc/remote/base.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* remote: base: remove wrappers from _checkout_file
2019-09-06 17:20:32 +03:00
Mr. Outis 739236eab0 ssh: accept only wb mode for open 2019-09-06 08:48:22 -05:00
Alexander Schepanovski 6aeccfcc07 dvc: fix long lock filenames under Windows 2019-09-06 16:29:23 +07:00
Alexander Schepanovski 1bdaca0d6c dvc: switch to flufl lock
This locks employs hardlink trick, which works on NFS and alike unlike
fcntl/open locks.

Closes #1918.
2019-09-06 16:29:09 +07:00
Ruslan Kuprieiev 74bd7d59cf Merge pull request #2469 from pared/2465
remote: base: check progress_callback when skipping on checkout
2019-09-06 00:47:45 +03:00