357 Commits
Author SHA1 Message Date
Ruslan Kuprieiev eae432818a git: tree: use item.path instead of item.name (#3505)
We are currently using this [0] hack to work around the fact that
item.name
is not always a basename. Even though `IndexObject` has it defined that
way
[1], but `Submodule` - doesn't [2]. So using `item.name` might cause
issues
such as [3], because GitPython doesn't pass name parameter when simply
going through objects [4]. In other places, when working specifically
with
submodules, it sets `_name` attribute explicitly [5].

Note that we still don't walk into submodules when walking through a git
repo. Might want to reconsider that later as for now we haven't heard
anything about it from actual users.

Fixes #3481

[0] https://github.com/iterative/dvc/blob/0.90.0/dvc/scm/git/tree.py#L15
[1]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/base.py#L170
[2]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/submodule/base.py#L1123
[3] https://github.com/gitpython-developers/GitPython/issues/597
[4]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/tree.py#L237
[5]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/submodule/base.py#L357
2020-03-18 19:18:58 +02:00
Casper da Costa-Luis f19d800f43 import-url: allow queries in URL (#3432)
* import-url: allow queries in URL

Fixes #3424

* Revert "import-url: allow queries in URL"

This reverts commit 48495d027963073fe260f596d85c3c179fee037f.

* path_info: add HTTPURLInfo base

* path_info: HTTPURLInfo complete

* remote: http(s): support params, queries, fragments

Fixes #3424
Closes #3432

* tests: path_info: HTTPURLInfo

* lint

* tests: path_info: HTTPURLInfo

Slightly more thorough testing
2020-03-17 19:23:56 +02:00
Ruslan Kuprieiev 6559b9d222 dvc: use protected mode by default (#3472)
* dvc: use protected mode by default

Fixes #3261
Fixes #2041

* unprotect: adjust help message

https://github.com/iterative/dvc.org/pull/1058/files/21aab371a487acf6f6e6201b29cd832e7c55ed23#r393324668
2020-03-17 19:16:21 +02:00
Saugat PachhaiandJorge Orpinel 066ccea733 checkout: show summary by default and introduce --show-changes flag (#3401)
* checkout: implement --show-changes flag to show detailed checkout details
Show summary by default on checkout

checkout: show directory modified even when files are removed from inside it

checkout: refactor template logic

checkout: change output format

checkout: do not show summary if nothing's changed

checkout: do not show summary if show_changes is specified

checkout: show '/' at the end if it's a directory

rename variable name

test: cover stats for checkout

checkout: show relink as not modified

checkout: remove unwanted message during empty checkout

pull: show summary on checkout

import/get: change external_repo to catch CheckoutError

fix tests and check if path is url

checkout: support external deps/outputs

tests: checkout on external outputs

* fix tests

* checkout: make show changes default and introduce summary flag to show summary

* address suggestions

* exceptions: remove extra spacing on erorr

* Update dvc/command/checkout.py

Co-Authored-By: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2020-03-16 16:29:36 +02:00
Saugat Pachhai d2734dfe1e tests: remove use of mocker as context-manager (#3480) 2020-03-16 08:25:22 +05:45
Alexander Schepanovski e69b44d1ed dvc: optimize all target specific commands to not build graph (#3459) 2020-03-15 21:53:38 +02:00
Max Risuhin 3ff804b3b2 gdrive: Google Service account support (#3269) 2020-03-11 15:49:33 +02:00
Guro Bokum 02cd520acd api: colorize dvc list (#3351)
Close #3347
2020-03-09 14:07:23 +02:00
Alexander Schepanovski 05cc023164 test: check that no remote backend modules are loaded by default (#3400) 2020-02-28 14:45:51 +02:00
Ivan Shcheklein eebbf3f812 change diff checksum arg name to hash (#3384) 2020-02-23 06:32:35 +02:00
Paweł Redzyński bd33b1ae30 Stage: get checksum from repo if no checksum in stage file (#3365) 2020-02-21 12:10:33 +02:00
68e543f7e1 remotes.http: Support dvc push for http remotes (#3343)
* remote.http: support _upload() via HTTP POST

- uploaded files are sent as chunked encoding POST data

Fixes #3247

* remote.http: add support for HTTP basic and digest auth

- username, password, ask_password options work the same way as for
  ssh remotes
- basic_auth (bool) and digest_auth (bool) options added for http(s)
  remotes
- digest_auth takes precedence over basic_auth if both are enabled

* remote.http: auth_method() unit tests

* tests: Add functional tests for HTTP remotes

- HTTP remotes now tested locally using a SimpleHTTPServer instance that
  allows reading/writing to a temp directory

* tests: fix http tests on py3.5, 3.6

* remote.http: add custom auth method

* s/custom_header/custom_auth_header/

* Apply suggestions from code review

Co-Authored-By: Saugat Pachhai <[email protected]>
Co-Authored-By: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Saugat Pachhai <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-02-21 12:05:55 +02:00
Paweł Redzyński 9b2ef002ea Stage: create: reset repo after new stage creation (#3349) 2020-02-19 20:25:04 +02:00
Mr. Outis 8492786ae4 logger: refactor, remove dead code (#3306)
* Remove dead code
* Rename _walk_exc to _causes, simplify it
* Split up stack trace and remove extra \n
* Import traceback only when needed
2020-02-17 03:19:37 +02:00
Saugat Pachhai 46e5053705 update: support specifying revision when updating (#3337)
* update: work with rev

* completion: update --rev completion scripts

* test: fix unit test assumption

* test: use fspath_py35
2020-02-15 17:55:41 +02:00
Ruslan Kuprieiev 8a2699dfbb schema: use a more flexible hash schema (#3333)
Fixes #3331
2020-02-15 01:13:45 +02:00
Casper da Costa-Luis 276a2b02a3 ui: make progress bars respect --quiet (#3316)
* ui: default no_progress_bar=None instead of False

Fixes #3314

* ui: catch remaining missing auto-quiet progress

* progress: treat disable=False as None

* test: progress.Tqdm(disable=False)

* revert no_progress_bar=None <= False

partially reverts e0342a8641f0101d
2020-02-15 00:53:52 +02:00
Saugat Pachhai 17c7b72cb0 logger: log datetime when on debug mode (#3325) 2020-02-14 13:58:15 +02:00
Alexander Schepanovski 6ccae711fe dvc: refactor config (#3298)
* dvc: refactor config

Advantages:
- read like a dict
- modify as a dict for testing or manipulation purposes
- wrap modifications into context manager to save them:

    with config.edit("local") as conf:
        conf["core"]["remote"] = "new-default"

Design decisions:
- dropped string constants
- remote sections are parsed into a simple subdict upon load
- remote config schema is now defined by url scheme
- relative paths are handled transparently upon load/save
- user config and remote manipulations are moved to commands
- same dev manipulations are done by simply modifying a config dict

* test: do not set cache.type for remotes that doesn't support it

* dvc: fix windows absolute paths validation in config

* test: fix windows separate drive test

* dvc: stylistic fixes in config
2020-02-13 19:35:37 +02:00
Guro Bokum 7e6e8ee169 api: add list command (#3246)
Close #2509
2020-02-12 22:10:48 +02:00
Mr. Outis 753aef4910 black: bump version 19.3b0 -> 19.10b0 (#3307) 2020-02-12 02:41:24 +02:00
Ruslan Kuprieiev 2361bceb70 repro: fix bug in _get_active_graph (#3301)
We are currently removing all of the deps stages from the DAG even if
those deps are referenced by some parallel branches of execution.
The new approach is to remove edges and then gradually remove nodes if
there are no more `in` edges left.

Discord context:
https://discordapp.com/channels/485586884165107732/485596304961962003/676464313942409265
2020-02-11 01:45:43 +02:00
Ruslan Kuprieiev 0c8897949e diff: use rev instead of ref (#3299)
Fixes #3245
2020-02-11 00:13:23 +02:00
Casper da Costa-Luis 3d5b8ecdd5 add: auto convert absolute_path => relative (#2975)
* add: auto convert absolute_path => relative

- fixes #2955

* use path_isin

* catch-all absolute paths in repo

* unneeded leftover

* revert realtive imports for later

* output: move abs2relpath from Base to Local

* output:local:fix parse_path

* remove path_info

* test:local versus base abs2relpath

* test:fix remaining abs2relpath test

* output: local path relative to wdir

* output: local: modify path before inheritance

* output: local: minor inheritance fix

* test: local: split output test

* test: local: positively explicit
2020-02-10 18:35:24 +02:00
Ruslan Kuprieiev d305803015 analytics: ignore request.post errors (#3292)
Users have been complaining about dvc analytics errors when they are not
connected to the internet.
2020-02-10 01:41:42 +02:00
Ruslan Kuprieiev 2786a4e59b logger: allow exc_info in all record types (#3290)
This better reflects the original logger formatter.

This also fixes annoying redundant newline between traceback and "need
any help" footer.
2020-02-09 22:22:20 +02:00
Alexander Schepanovski e28371f53a Fix blockers list in rwlock error message (#3285) 2020-02-06 16:25:17 +02:00
Mr. Outisand@dmpetrov e5145e6c55 diff: format summary to make it more human readable
Co-authored-by: @dmpetrov
2020-01-30 13:15:49 -06:00
Mr. Outis 793a069644 tests/diff: order dictionary (second try) 2020-01-30 10:00:56 -06:00
Mr. Outis e9bd9599a2 tests/diff: dont care about sorted jsons 2020-01-30 09:26:35 -06:00
Mr. Outis e501549bd7 diff: add a summary at the bottom 2020-01-29 18:54:45 -06:00
Mr. Outis 8e9d8145cd diff: remove deadcode, replace diffable with dicts 2020-01-29 17:42:35 -06:00
Mr. Outis 423a6163bc diff: make --checksums work with --json-show 2020-01-27 14:10:37 -06:00
Mr. Outis fced1718cc diff: --json -> --show-json 2020-01-27 12:53:24 -06:00
Mr. Outis 2370cb12b3 diff: use logger instead of print 2020-01-27 12:33:24 -06:00
Ruslan Kuprieiev 49039968f7 Merge pull request #3208 from fabiosantoscode/feature/3198-remove-spy-method
tests: replace spy() with pytest-mock spy
2020-01-23 10:52:43 +02:00
Fábio Santos fc92266114 address comments 2020-01-22 17:31:46 +00:00
Fábio Santos bdf665a34a don't use patch.object() 2020-01-21 20:52:48 +00:00
Fábio Santos 5dfcda54f9 tests: replace spy() with pytest-mock spy
Fixes #3198
2020-01-21 14:47:49 +00:00
Ruslan Kuprieiev e848bcb9d6 gdrive: allow configuring no_traverse 2020-01-20 23:49:49 +02:00
Paweł Redzyński cf08529362 utils: add tests for resolve out 2020-01-17 09:51:07 +01:00
Saugat PachhaiandPaweł Redzyński 98f6d56e8c test: refactor, apply suggestion from @pared
Co-Authored-By: Paweł Redzyński <[email protected]>
2020-01-16 22:59:58 +05:45
Saugat Pachhai 02a73a0d9b unit: test: check output of get --show-url command 2020-01-16 22:52:35 +05:45
Saugat Pachhai eb252e055b get: split command for getting url and getting files from url functions 2020-01-16 19:28:14 +05:45
Saugat Pachhai 4509f82bc3 get: implement --show-url to display only url/path to remote 2020-01-16 19:28:14 +05:45
Guro Bokum 4918c56ca3 remote: config checksum_jobs
Close #2920
2020-01-15 22:23:09 +08:00
Ruslan Kuprieiev b865a48b92 Merge pull request #3146 from pared/2896_fs_repo
fs: tests: remove repo_dir
2020-01-14 16:49:23 +02:00
Ruslan Kuprieiev 9c934c8abf Merge pull request #3147 from pared/2896_ssh_utils
ssh: test: unit, utils: test: unit: remove repo_dir
2020-01-14 16:38:08 +02:00
pawel cb5b5ed334 ssh: test: unit, utils: test: unit: remove repo_dir 2020-01-14 15:32:48 +01:00
pawel 144f10d8cb fs: tests: remove repo_dir 2020-01-14 15:25:55 +01:00