Commit Graph
3504 Commits
Author SHA1 Message Date
Alexander Schepanovski 6010ce54cd remote: manually close all ssh pools at the end
Python 2 fails to do it automatically occasionally and shows weird error
messages.

This was harmless but confusing.
2019-07-17 22:20:43 +07:00
Andrew Grigorev 7d3e9808b5 ci: unify pip cache location for all platforms (#2286) 2019-07-17 15:42:38 +03:00
Nabanita Dash 89541b73fa Autocompletion of commands and files (#2226)
* changed " to '

* autocompletion of commands and files

* Revert "autocompletion of commands and files"

This reverts commit 0b394ec841c68357cec6eb1cfa2cb20b636d3739.

revert the changes made

* changed

* bash complete

* modified

* shows subcommands but not files

* modified
2019-07-17 00:29:09 +03:00
Georgy Dyuldin 840318dd22 git: Fix submodule names issue (#2274)
GitPython creates submodules nodes during tree iterate without explicit
passing name attribute. This path replaces such submodules with new
instances with filled name attribute.

Fixes: #1898
2019-07-16 16:38:42 +03:00
Alexander Schepanovski db19957b8a Download refactor (#2275)
* remote: drop not working http resume

Closes #2136

* remote: drop @transfer_context()

It is not needed for ssh anymore and GS/S3 are switched to client
instances in cached properties.

* remote: drop upload/download chunking
2019-07-16 16:27:44 +03:00
Alexander Schepanovski 9766d34cec remote: support older linuxes as ssh remotes (#2276)
* remote: support older linuxes as ssh remotes

Work around md5sum not having --tag in Red Hat 6.3 and friends.
Fixes #2242.

* remote: optimize consequent ssh md5 calls
2019-07-16 14:15:28 +03:00
Ruslan Kuprieiev fc5d3c182c dvc: bump to 0.52.1 0.52.1 2019-07-15 20:33:54 +03:00
Ruslan Kuprieiev a59fdfdd35 stage: don't ignore SIGINT in the child (#2271)
Signal mask is inherited by the child, so when we set it up before
Popen, our child ignores SIGINT, which is wrong. What we should've
done ideally, is to setup the signal handler after fork() but before
exec(). That is pretty much what preexec_fn is used for in Popen, but
unfortunately for us, it is not available on windows, so we can't use
it. What we will do instead is just setup signal handler after Popen()
but before communicate(), which would leave a slight gap where we might
also get affected by SIGINTS meant for the child, but that is the best
we can do.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-15 20:17:10 +03:00
Alexander Schepanovski b5a26e5853 remote: fix collect dir progress (#2270)
We were resolving futures after ThreadPoolExecutor exited and finished
everything.
2019-07-15 18:31:19 +03:00
Paweł Redzyński 8641773992 run: use kwargs instead list of explicit arguments (#2260) 2019-07-15 16:19:11 +03:00
Ruslan Kuprieiev e215364e6d test: http: use random port (#2269)
Fixes #1878

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-15 13:18:10 +03:00
Ruslan Kuprieiev 8c303512e8 dvc: bump to 0.52.0 0.52.0 2019-07-15 11:41:30 +03:00
Ivan Shcheklein 587687fda7 readme: add blog link, fix Twitter link 2019-07-14 14:29:38 -07:00
Alexander Schepanovski b9a3c7c038 remote: use ssh connections pool (#2264)
* remote: use ssh connections pool

* dvc: never use paramiko DEBUG mode

It's ridiculously verbose, makes logs unreadable.
2019-07-14 23:30:24 +03:00
mindsbackyard 5976b5e466 disable analytics warning message on init if disabled globally/system… (#2268)
* disable analytics warning message on init if disabled globally/system-wide

* rename Analytics._is_enabled
Analytics.is_enabled

* refactor trying reduce cognitive complexity metric

* fix missing imports

* fix mistaken reference of self
2019-07-14 23:29:30 +03:00
Ruslan Kuprieiev 10121411e0 import: don't forget to add outs to gitignore (#2266)
Fixes #2263

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-14 21:35:23 +03:00
Ruslan Kuprieiev 94fe86afe8 tests: import-url: rewrite and move error msg test (#2262)
This one was failing randomly.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-14 03:09:08 +03:00
Paweł Redzyński e8623a0c69 run: rerun deterministic callback (#2254)
* run: rerun deterministic callback

* test_run: refactor deterministic run tests to use pytest
2019-07-14 02:25:13 +03:00
Ruslan Kuprieiev 2c2f573897 import: remove beta warnings (#2257)
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-13 23:29:36 +03:00
Paweł RedzyńskiandRuslan Kuprieiev 393d926b74 state: use dvcignore when accessing metadata (#2161)
* state: use ignore file handler when accessing metadata

* fs: don't add dir size to overall size for dir

* ignore: use posix-like path on windows

* ignore: remove tree from dvc_walk arguments

* ignore: handle DvcIgnoreFilter creation outside of DVC repo

* ignore: throw on dvcignore file under output dir, but not dependency dir

* ignore: move ignore initialization to Repo init

* ignore: work in progress

* ignore: work in progress 2

* ignore: throw on .dvcignore upon collecting dir

* ignore: refactor

* ignore: loading upper level ignores in stages()

* Update dvc/repo/__init__.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* dvcignore review refactor
2019-07-12 11:50:22 +03:00
Paweł Redzyński 217f2c4dbd imp_url: normalize path for default out name (#2248) 2019-07-12 09:20:47 +03:00
Ruslan Kuprieiev 95110a865c dvc: bump to 0.51.2 0.51.2 2019-07-11 12:36:35 +03:00
Ruslan Kuprieiev eaab968036 build: windows: don't forget to pull images 2019-07-11 08:52:22 +03:00
Ruslan Kuprieiev 72a61b5599 build: windows: don't use abs path for iscc 2019-07-10 20:59:04 +03:00
Ruslan Kuprieiev 375dc261d0 build: windows: use call to call refreshenv 2019-07-10 18:03:15 +03:00
Ruslan Kuprieiev ccb0d7d1b3 travis: build_package: use which 2019-07-10 15:46:51 +03:00
Ruslan Kuprieiev d5c473acd4 travis: fix build_package.sh 2019-07-10 15:41:52 +03:00
Ruslan Kuprieiev 9eb06fee3b travis: before_install: don't set -x,-e in env.sh 2019-07-10 15:22:22 +03:00
Ruslan Kuprieiev c02137d7ba travis: adjust job names 2019-07-10 14:20:48 +03:00
Ruslan Kuprieiev 1edd2ffaec dvc: bump to 0.51.1 2019-07-10 13:03:49 +03:00
Ruslan Kuprieiev fa89e84baa travis: use stages (#2249)
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-10 12:12:11 +03:00
Ruslan Kuprieiev 99065bd42e dvc: networkx 2.3 doesn't support py2 anymore (#2250)
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-10 05:58:23 +03:00
Ruslan Kuprieiev 8e8dc921a3 travis: try build packages on regular builds (#2244)
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-10 03:25:52 +03:00
Alexander Schepanovski 732fd01cc5 test: drop a reimport (#2247) 2019-07-10 03:24:59 +03:00
Alexander Schepanovski 90b1a80240 test: test ssh dir output (#2246) 2019-07-10 03:24:35 +03:00
Alexander Schepanovski 16c16d334b dvc: skip unused checks in repro -f (#2245) 2019-07-10 03:22:07 +03:00
Ruslan Kuprieiev 0ab55f84ba travis: don't uninstall awscli on win 2019-07-08 21:42:08 +03:00
Ruslan Kuprieiev fcc8938b33 test: repro: fix formatting 2019-07-08 21:27:32 +03:00
Ruslan Kuprieiev e8b0d2d987 test: disable repro s3 test on windows 2019-07-08 21:24:23 +03:00
Ruslan Kuprieiev a8bd68c848 travis: windows: fix typo in choco command 2019-07-08 21:05:02 +03:00
Ruslan Kuprieiev 5ac6509ffd travis: win: install awscli using choco 2019-07-08 18:21:49 +03:00
Alexander Schepanovski 14ce9d6d4a dvc: stop trying to fetch from remote when looking for git object (#2241)
Closes #2213
2019-07-08 17:57:25 +03:00
Ruslan Kuprieiev d0299b3edf travis: debug windows s3 test 2019-07-08 17:47:54 +03:00
Ruslan Kuprieiev f3c4aff6c5 test: don't use python to call awscli 2019-07-08 17:44:18 +03:00
Ruslan Kuprieiev 912ec6b425 dvc: bump to 0.51.0 2019-07-08 17:19:48 +03:00
Ruslan Kuprieiev 7ad3c56865 travis: don't override before_install (#2239) 2019-07-07 15:57:08 +03:00
Andrew Grigorev a4720d07c7 travis: add $HOME/.cache/pip to cache explicitly (#2240)
TravisCI doesn't support Python on Windows yet, so `cache.pip` option
doesn't work.
2019-07-07 02:16:37 +03:00
Andrew Grigorev 40c10bd916 ci: pydocstyle==4.0.0 breaks flake8 2019-07-06 21:55:33 +03:00
Ruslan Kuprieiev af34867d96 test: try calling awscli through python 2019-07-06 02:49:08 +03:00
Ruslan Kuprieiev 932b0d7af9 introduce dvc update for ext repos (#2218)
* erepo: dvc: implement `update` for imports from ext repos

Fixes #1774
Fixes #2139
Fixes #2201

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* deps: repo: make  private

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* update: only support repo imports

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-06 02:28:35 +03:00