3368 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 32425e9069 dvc: bump to 0.53.2 0.53.2 2019-07-29 00:55:05 +03:00
Alex JozandRuslan Kuprieiev 2500ce64a6 Restrict ruamel.yaml version due import error (#2335)
* restrict ruamel.yaml version due import error

* Update setup.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
2019-07-29 00:49:29 +03:00
Ruslan Kuprieiev 386793ae14 Merge pull request #2317 from mroutis/fix-1475
scm: reset tracked files after every operation
2019-07-28 02:39:47 +03:00
Mr. OutisandRuslan Kuprieiev dc5dd1e443 change set.update to set.add
created a set with `{}` but resulted in a dictionary, making me think about `set` not supporting the `add` method.

Thanks for the correction, @efiop !

Co-Authored-By: Ruslan Kuprieiev <[email protected]>
2019-07-27 17:44:45 -05:00
Mr. Outis 405dfde23f use set instead of list for deduplication 2019-07-27 16:34:22 -05:00
Ruslan Kuprieiev a4aef6751e Merge pull request #2304 from pared/2285
metric: show: xpath should override stage file xpath
2019-07-25 18:43:29 +03:00
pared f5e0024137 metrics: show: add comment clarifying metric type and xpath usage 2019-07-25 16:33:49 +02:00
pawel 7c52f65736 metric: show: xpath should override xpath set after run 2019-07-25 10:07:02 +02:00
Ruslan Kuprieiev 7fea91387e dvc: bump to 0.53.1 0.53.1 2019-07-25 03:49:29 +03:00
Ruslan Kuprieiev 25a782e538 Merge pull request #2320 from efiop/2284
version: temporarily make psutil optional
2019-07-25 03:48:44 +03:00
Ruslan Kuprieiev 4a4ddf078c version: temporarily make psutil optional
Workaround for https://github.com/iterative/dvc/issues/2284

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-25 01:52:11 +03:00
Ruslan Kuprieiev 590c2dcbfc Merge pull request #2293 from pared/2258
remote: base: try to retrieve path_info checksum from database upon dir checksum calculation
2019-07-24 16:43:50 +03:00
Mr. Outis 9ca5452603 scm: reset tracked files after every operation
Fix #1475
2019-07-23 15:10:48 -05:00
Ruslan Kuprieiev 507879e841 Merge pull request #2315 from Suor/ask
ssh: do not ask for same password repeatedly
2019-07-23 19:54:08 +03:00
Ruslan Kuprieiev 12461cf356 Merge pull request #2314 from Suor/meta-md5
dvc: ignore Stage.meta in md5 calculations plus some polish
2019-07-23 19:53:23 +03:00
Alexander Schepanovski 59eb929516 ssh: do not ask for same password repeatedly 2019-07-23 16:25:40 +03:00
Alexander Schepanovski 35a8d69c4f dvc: ignore Stage.meta in md5 calculations
Meta change surely should not trigger a repro run. Closes #2209.
2019-07-23 16:00:22 +03:00
Alexander Schepanovski f00a325114 dvc: refactor dict_md5()
- stop using mutable default value
- use list and dict comprehensions to speed it up and make nicer
- employ exhaustive if..elif..else
2019-07-23 16:00:12 +03:00
Ruslan Kuprieiev 6b84dbb985 Merge pull request #2303 from Suor/push-jobs
remote: fixed Repo.push() to default to 1 job
2019-07-23 14:44:54 +03:00
Ruslan Kuprieiev 93f0062482 setup: freeze pyarrow version 2019-07-23 14:27:51 +03:00
pawel e78afefc97 remote: base: refactor _collect_dir 2019-07-23 13:03:08 +02:00
pared 7885b0404f remote: base: try to retrieve path_info checksum from database upon dir checksum calculation 2019-07-23 13:03:08 +02:00
Alexander Schepanovski 20f1976e03 remote: fixed Repo.push/pull/fetch/status() to default to 1 job 2019-07-23 12:23:43 +03:00
Alexander Schepanovski f37ab9431b perf: stop on first changed thing in Stage.changed() 2019-07-23 15:29:49 +07:00
Ruslan Kuprieiev f346300a13 cache: don't forget about -q and -v options
https://github.com/iterative/dvc/issues/1924#issuecomment-496755449
2019-07-23 03:13:02 +03:00
Ruslan Kuprieiev ecb28a285c Merge pull request #2307 from efiop/2238
import-url: lock by-default
2019-07-23 01:44:04 +03:00
Ruslan Kuprieiev ffc828df6f dvc: bump to 0.53.0 2019-07-23 01:38:37 +03:00
Ruslan Kuprieiev 641edacef6 import-url: lock by-default
Fixes #2238

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-22 20:24:00 +03:00
Ruslan Kuprieiev 33511902c3 Merge pull request #2306 from efiop/2252
import: only download specific output
2019-07-22 15:55:16 +03:00
Ruslan Kuprieiev 4a0b3a807d import: only donwload specific output
Fixes #2252

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-22 15:04:10 +03:00
Ruslan Kuprieiev a1143c532c Merge pull request #2289 from Suor/exists-jobs
remote: honor --jobs for status collection and other fixes
2019-07-21 19:24:30 +03:00
Alexander Schepanovski 1f90b426fb remote: prettify get_connection/pool() signatures 2019-07-21 13:41:04 +07:00
Alexander Schepanovski 6afc2a38ba test: close pools on tests cleanup
This avoids weird errors as pools are collected by gc in broken env.
2019-07-21 13:40:29 +07:00
Alexander Schepanovski 31f1ded315 remote: don't close sftp connections until parent ssh conn closed
Since this is multiplexing over the same TCP connection no system
resources are withheld, so this should be a no-brainer optimization.

As a side-effect this should fix "open max - close all - try open -
Administratively prohobited" error @darabi experiencing in #2280.
2019-07-21 13:38:44 +07:00
Alexander Schepanovski 624ccfc744 remote: honor --jobs for status collection 2019-07-21 12:50:35 +07:00
Ruslan Kuprieiev b6bc65fcbf Merge pull request #2302 from shcheklein/fix-2259
url info: fix infinite recustion on deepcopy
2019-07-21 04:32:30 +03:00
Ruslan Kuprieiev bd13d6f9d6 Merge pull request #2299 from jorgeorpinel/master
output: remove a couple more "(s)" left
2019-07-21 02:17:42 +03:00
Ivan Shcheklein ef013c8472 url info: minor fix typo in a comment 2019-07-20 15:51:11 -07:00
Ivan Shcheklein 164c2e8dc1 url info: fix infinite recustion on deepcopy 2019-07-20 15:26:00 -07:00
Jorge Orpinel 80c30cebea update: better short desc in command help outut
per https://github.com/iterative/dvc.org/pull/490#discussion_r305584059
2019-07-20 13:47:00 -06:00
Jorge Orpinel 0575104c20 update: small autocomplete scripts reformat 2019-07-20 13:44:36 -06:00
Jorge Orpinel c7ef6592fe output: remove a couple more "(s)" left
Per https://github.com/iterative/dvc/pull/2281#issuecomment-513053199
2019-07-20 13:44:36 -06:00
Ruslan Kuprieiev df28309403 Merge pull request #2300 from efiop/moto
tests: use moto pre-release
2019-07-20 17:12:38 +03:00
Ruslan Kuprieiev f901ba5c10 tests: use moto pre-release
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-20 16:09:44 +03:00
Ruslan Kuprieiev c2d25f455f travis: hadoop: fix typo 2019-07-20 03:00:39 +03:00
Ruslan Kuprieiev 3ccb80466a travis: hadoop: add openjdk-7 ppa 2019-07-20 02:45:38 +03:00
Ruslan Kuprieiev a4e461fc02 travis: hadoop: use openjdk-7 2019-07-20 02:32:49 +03:00
Ruslan Kuprieiev 3becd5c20b Merge pull request #2297 from efiop/hdfs
remote: hdfs: use pyarrow and connection pool
2019-07-20 02:14:56 +03:00
Ruslan Kuprieiev 68bebe7944 remote: hdfs: use pyarrow and connection pool
Related to #1629

Speeds up HDFS tests significantly. E.g. ExternalHDFS test goes from 600
sec to 190sec. test_open_external[HDFS] from 160sec to 3sec.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-20 01:12:17 +03:00
Ruslan Kuprieiev 1e9f1d0619 Merge pull request #2281 from jorgeorpinel/master
help: finish pluralizing "the pipeline" term in output
2019-07-19 12:55:10 +03:00