Commit Graph
5258 Commits
Author SHA1 Message Date
Ivan Shcheklein 4489f98dc3 readme: update flow chart link 2020-03-18 21:06:41 -07:00
Ivan Shcheklein de19d15332 readme: fix banner link after some dvc.org changes 2020-03-18 16:09:49 -07:00
Ruslan Kuprieiev a1fe6c6f44 dvc: bump to 0.90.2 0.90.2 2020-03-18 23:11:17 +02:00
Ruslan Kuprieiev 7edee656da pipeline: fix --outs output (#3507)
Fixes #3497
2020-03-18 23:03:41 +02:00
Ruslan Kuprieiev 498ee509da dvc: use consistent messages for InvalidArgumentError (#3506)
Per https://discordapp.com/channels/485586884165107732/563406153334128681/689896480898810034
2020-03-18 21:34:46 +02:00
Ruslan Kuprieiev 55f86df5b1 dvc: bump to 0.90.1 0.90.1 2020-03-18 19:20:22 +02:00
Ruslan Kuprieiev eae432818a git: tree: use item.path instead of item.name (#3505)
We are currently using this [0] hack to work around the fact that
item.name
is not always a basename. Even though `IndexObject` has it defined that
way
[1], but `Submodule` - doesn't [2]. So using `item.name` might cause
issues
such as [3], because GitPython doesn't pass name parameter when simply
going through objects [4]. In other places, when working specifically
with
submodules, it sets `_name` attribute explicitly [5].

Note that we still don't walk into submodules when walking through a git
repo. Might want to reconsider that later as for now we haven't heard
anything about it from actual users.

Fixes #3481

[0] https://github.com/iterative/dvc/blob/0.90.0/dvc/scm/git/tree.py#L15
[1]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/base.py#L170
[2]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/submodule/base.py#L1123
[3] https://github.com/gitpython-developers/GitPython/issues/597
[4]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/tree.py#L237
[5]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/submodule/base.py#L357
2020-03-18 19:18:58 +02:00
Ruslan Kuprieiev aef3dfba13 dvc: bump to 0.90.0 0.90.0 2020-03-18 01:24:19 +02:00
Ruslan Kuprieiev 42c8fab47f version: add supported remotes (#3498)
Since our remotes have optional dependencies, it is really useful to be
able to see what is currently supported. Also helps debug issues with
packages such as brew/conda/etc.
2020-03-18 01:14:59 +02:00
Casper da Costa-Luis f19d800f43 import-url: allow queries in URL (#3432)
* import-url: allow queries in URL

Fixes #3424

* Revert "import-url: allow queries in URL"

This reverts commit 48495d027963073fe260f596d85c3c179fee037f.

* path_info: add HTTPURLInfo base

* path_info: HTTPURLInfo complete

* remote: http(s): support params, queries, fragments

Fixes #3424
Closes #3432

* tests: path_info: HTTPURLInfo

* lint

* tests: path_info: HTTPURLInfo

Slightly more thorough testing
2020-03-17 19:23:56 +02:00
Casper da Costa-LuisandJorge Orpinel 137373c165 template: pr update (#3495)
* template: pr update

* Update .github/PULL_REQUEST_TEMPLATE.md

* Update .github/PULL_REQUEST_TEMPLATE.md

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Jorge Orpinel <[email protected]>
2020-03-17 19:18:25 +02:00
Ruslan Kuprieiev 6559b9d222 dvc: use protected mode by default (#3472)
* dvc: use protected mode by default

Fixes #3261
Fixes #2041

* unprotect: adjust help message

https://github.com/iterative/dvc.org/pull/1058/files/21aab371a487acf6f6e6201b29cd832e7c55ed23#r393324668
2020-03-17 19:16:21 +02:00
Saugat Pachhai 4b5d4d759a gc: do not work without specifier when using -c flag (#3493)
* gc: ask for specifier when using -c flag

* tests: add -w flag with gc -c
2020-03-17 18:12:42 +02:00
Saugat Pachhai e1334448ea completion: add --summary flag on checkout (#3491) 2020-03-17 14:04:12 +02:00
Ruslan Kuprieiev 02daaf391a dvc: bump to 0.89.0 0.89.0 2020-03-17 02:08:56 +02:00
Ruslan Kuprieiev 2ffebb0b5e logger: use standard lazy formatting (#3487)
Unfortunately overriding `format` is not enough in order to use our own
custom formatting in logger. `Record` uses percent formatting that is
hardcoded inside, so we would also have to override it, which is just
way too much effort that will inevitably break as we discover other
methods that are not ready for non-% formatting. So we are just giving
up and sticking to default %-formatting for lazy log messages.

Fixes #3397
2020-03-17 00:38:00 +02:00
Paweł Redzyński 87338c994f cmd: diff: don't load dir cache for diff (#3468) 2020-03-16 21:39:34 +02:00
Saugat PachhaiandCasper da Costa-Luis 109ded4068 dvc: support Python3.8 (#3463)
* py38: make wheels

- ci: set ARROW_LIBHDFS_DIR to load native library
- deps: pin pyarrow to 0.15.1 for py<=3.7
- pin deps for python<3.8
- pin deps for python==3.8 as well
- add windows Py38 on travis
- do not use speedcopy for Python3.8 and Windows

* use realpath to resolve symbolic links

* do not build pkgs for Windows and Linux on Python3.8

* reorder

* pyarrow: document regression fix

Co-authored-by: Casper da Costa-Luis <[email protected]>
2020-03-16 19:18:33 +00:00
Ruslan Kuprieiev 3f3384b7c4 pre-commit: use official dvc repo 2020-03-16 20:41:29 +02:00
Ruslan Kuprieiev b223696079 install: use hooks from master
Same as with shell completion, it is better to use the newest hooks.
2020-03-16 20:40:02 +02:00
c4164b3497 Use pre-commit for Git hooks (#3406)
* Initial design of using pre-commit for Git hooks

* Refactor to reuse existing code and provide merge capabilities

* Linter fixes

* More cleanup

* scm: make use_precommit_tool optional

* dvc: install our own hooks

* install: fix typo

* install: fix bugs in pre-commit integration

* install: don't forget 'always_run' for post-checkout

* install: no need to run pre-commit install if conf file doesn't exist

* install: fix bugs

* install: use python hooks

* add .pre-commit-hooks.yaml

Co-authored-by: Andrew Hare <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-03-16 20:34:01 +02:00
Guro Bokum dcf5af647e list: rename target param to path, update docs (#3462)
Close #3381
2020-03-16 20:22:48 +02:00
Saugat Pachhai 1b21789444 exceptions: use semantic exceptions (#3430) 2020-03-16 19:39:59 +02:00
Saugat PachhaiandJorge Orpinel 066ccea733 checkout: show summary by default and introduce --show-changes flag (#3401)
* checkout: implement --show-changes flag to show detailed checkout details
Show summary by default on checkout

checkout: show directory modified even when files are removed from inside it

checkout: refactor template logic

checkout: change output format

checkout: do not show summary if nothing's changed

checkout: do not show summary if show_changes is specified

checkout: show '/' at the end if it's a directory

rename variable name

test: cover stats for checkout

checkout: show relink as not modified

checkout: remove unwanted message during empty checkout

pull: show summary on checkout

import/get: change external_repo to catch CheckoutError

fix tests and check if path is url

checkout: support external deps/outputs

tests: checkout on external outputs

* fix tests

* checkout: make show changes default and introduce summary flag to show summary

* address suggestions

* exceptions: remove extra spacing on erorr

* Update dvc/command/checkout.py

Co-Authored-By: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2020-03-16 16:29:36 +02:00
Saugat Pachhai d2734dfe1e tests: remove use of mocker as context-manager (#3480) 2020-03-16 08:25:22 +05:45
Alexander Schepanovski e69b44d1ed dvc: optimize all target specific commands to not build graph (#3459) 2020-03-15 21:53:38 +02:00
Paweł Redzyński 767433d7a4 tests: add .gitignore files upon dvc_add (#3455) 2020-03-13 17:19:35 +02:00
Saugat Pachhai 682275dc0a add: do not verify hardlink if file is empty (#3428)
* add: do not verify hardlink if file is empty

Fixes #3390

* tests: spy

* fixup! tests: spy
2020-03-11 21:56:12 +02:00
Jorge OrpinelandRuslan Kuprieiev 9944efbd0f api: docstring updates (2) (#3426)
* api: update to latest definitions to match iterative/dvc.org/pull/908

* Update dvc/api.py

* metrics show: update -h output to match docs
per https://github.com/iterative/dvc.org/pull/1014/commits/2c3452119545987f77b764e4aa6fc94d051092a7

* api: update docstrings to match iterative/dvc.org/pull/908

* api: refactor DVC repo check in get_url, and document it

* dvc: cosmetic edits as I explored exceptions that api functions may raise

* api: copy default info to read() docstring from open()
per https://github.com/iterative/dvc/pull/3426#pullrequestreview-366949501

* api: improve open() docstring for clarity and add example
per https://github.com/iterative/dvc/pull/3426#pullrequestreview-366956856

* api: remove unnecessary info from get_url docstring
per https://github.com/iterative/dvc/pull/3426#discussion_r386203064

* api: produces->generated in open() docstring

* api: simplify open docstring
per https://github.com/iterative/dvc/pull/3426#discussion_r388218147

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-03-11 20:45:29 +02:00
Casper da Costa-Luis 7923446219 completion: bash: misc fixes and updates (#3467)
* completion: bash: fix, update and sort

* completion: bash: fix missing `compgen`s

* completion: bash: remove deprecated/hidden options

* completion: bash: fix extra parenthesis

- fixes metrics show -t

* completion: bash: re-add `gc --all-commits`

Due to #3429 superceding #2681
2020-03-11 15:56:20 +02:00
Max Risuhin 3ff804b3b2 gdrive: Google Service account support (#3269) 2020-03-11 15:49:33 +02:00
Ruslan Kuprieiev ef96e6b8ff dvc: bump to 0.88.0 0.88.0 2020-03-10 22:47:01 +02:00
Saugat Pachhai 7f94518008 deps: upgrade jsonpath_ng to latest version (#3464)
Fix #3456
2020-03-10 11:18:52 +02:00
Ruslan Kuprieiev 40519005f5 travis: hadoop: update tar location 2020-03-10 10:20:34 +02:00
Saugat Pachhai ea981ae955 gc: make --all-commits flag visible (#3429)
* gc: make --all-commits visible

* gc: fix help message and completion scripts

* bash: add longform --workspace flag
2020-03-10 08:29:07 +05:45
Ivan Shcheklein 4bceb9d9cd gdrive: share drive vs just folder id when we set corpora param (#3458) 2020-03-09 19:25:08 +02:00
Ruslan Kuprieiev 37e4a5dcd4 travis: use us-west-2 aws s3 region 2020-03-09 19:10:47 +02:00
Ruslan Kuprieiev 6479adb1a3 test: s3: move to dvc-temp bucket 2020-03-09 18:45:59 +02:00
Guro Bokum 02cd520acd api: colorize dvc list (#3351)
Close #3347
2020-03-09 14:07:23 +02:00
Alexander Schepanovski 357a6dd467 dvc: optimize building graph (#3443)
Use trie structure to efficiently check out, dep and stage paths being
prefixes of each other.
2020-03-09 12:13:07 +02:00
Ruslan Kuprieiev f05e6e0529 metrics show: update -h output to match docs (#3420)
per https://github.com/iterative/dvc.org/pull/1014/commits/2c3452119545987f77b764e4aa6fc94d051092a7
2020-03-05 16:11:43 +02:00
Ruslan Kuprieiev 06c634adee pipelines: fix edges gathering (#3421) 2020-03-05 14:16:57 +02:00
Alexander Schepanovski 05cc023164 test: check that no remote backend modules are loaded by default (#3400) 2020-02-28 14:45:51 +02:00
Ruslan Kuprieiev abc4e3c2d0 dvc: bump to 0.87.0 0.87.0 2020-02-28 02:21:39 +02:00
Casper da Costa-Luis 35a6e0fedd docs: update badges (#3380)
* docs: add and update badges

Add |Choco|
Add |Brew|
Fix style for |Conda|
Add |PyPI|
Add |Packages|
Fixes #3233
Related https://github.com/iterative/dvc.org/issues/904

* docs: add badge logos

* docs: minor badge tidy

* docs: add choco install, add OS, update order
2020-02-28 01:32:10 +02:00
ttekampeandGuro Bokum de453ab06d dont disable existing loggers when dvc is set up (#3345)
* dont disable existing loggers when dvc is set up

* corrected code style

* dont disable existing loggers, fixes

Co-authored-by: Guro Bokum <[email protected]>
2020-02-27 07:02:06 +02:00
Alexander Schepanovski d8778e7ca9 erepo: set up more logging (#3403) 2020-02-26 22:39:13 +02:00
Bruno Deferrari b136eb5151 erepo: fix relative url on default remote (#2756) (#3378)
* erepo: fix relative urls for remotes (#2756)

* erepo: add test case for relative path remotes (#2756)

* Remove unused argument

* erepo: move handling of relative remotes to a separate method (#2756)

* erepo: don't manually build paths on test (#2756)

* erepo: refactor handling of missing or relative upstream (#2756)

* erepo: move handling of source repo resource to `_fix_upstream` (#2756)

* erepo: improve relative remote tests (#2756)

- use `tmp_dir` fixture
- commit config file changes

* erepo: Remove comment (#2756)

* erepo: use scm_add to commit too, no need for separate scm.commit call (#2756)

* erepo: minor refactoring in `_fix_upstream` (#2756)

* erepo: add comment explaining that old relative paths are restored (#2756)
2020-02-26 22:37:29 +02:00
rxxg d7d5d8fcca Remove Linux pyfastcopy (#3412)
Fixes #3395
2020-02-26 22:35:48 +02:00
Casper da Costa-Luis 68a2f34a46 pull: fetch: external: respect jobs (#3413)
Fixes #3396
2020-02-26 22:35:19 +02:00