Commit Graph
5168 Commits
Author SHA1 Message Date
Saugat Pachhai 17c7b72cb0 logger: log datetime when on debug mode (#3325) 2020-02-14 13:58:15 +02:00
Paweł Redzyński 0c87771064 daemon config fix (#3324) 2020-02-14 13:13:04 +02:00
Alexander Schepanovski 7df06b278a erepo: fix known sha handling (#3323)
Closes #3280.
2020-02-14 12:59:31 +02:00
Gaurav Rai 51a5a4ec3b repo: Prevent repo from setting log level (#3321)
* Removed setting log level

* removed loglevel schema
2020-02-13 20:45:43 +02:00
Alexander Schepanovski 6ccae711fe dvc: refactor config (#3298)
* dvc: refactor config

Advantages:
- read like a dict
- modify as a dict for testing or manipulation purposes
- wrap modifications into context manager to save them:

    with config.edit("local") as conf:
        conf["core"]["remote"] = "new-default"

Design decisions:
- dropped string constants
- remote sections are parsed into a simple subdict upon load
- remote config schema is now defined by url scheme
- relative paths are handled transparently upon load/save
- user config and remote manipulations are moved to commands
- same dev manipulations are done by simply modifying a config dict

* test: do not set cache.type for remotes that doesn't support it

* dvc: fix windows absolute paths validation in config

* test: fix windows separate drive test

* dvc: stylistic fixes in config
2020-02-13 19:35:37 +02:00
Paweł Redzyński ccca126749 add: meaningful message upon adding overlapping paths (#3296) 2020-02-13 05:31:57 +02:00
Guro Bokum 7e6e8ee169 api: add list command (#3246)
Close #2509
2020-02-12 22:10:48 +02:00
Mr. Outis 753aef4910 black: bump version 19.3b0 -> 19.10b0 (#3307) 2020-02-12 02:41:24 +02:00
Ruslan Kuprieiev 9a78c0e2fd dvc: bump to 0.84.0 0.84.0 2020-02-11 22:09:59 +02:00
Ruslan Kuprieiev 7200bddd09 dvc: bump to 0.83.1 2020-02-11 22:08:30 +02:00
Alexander Schepanovski 47081cb118 erepo: cache all read only external repos by hexsha (#3286)
* erepo: cache all read only external repos by hexsha

So we have 3 things cached now separately:
- clean clones, to not ask for creds repatedly
- cache dirs, also shared between erepos with same origin
- checked out clones if they are read only, addressed by (url, hexsha)

Several additions to `Git` along the way:
- Git.is_sha() static method
- .pull() and .push() work with multiple returned records correctly
- .get_rev() and .resolve_rev() work faster
- .resolve_rev() looks for remote branches
- .has_rev()

Fixes #3280.

* git: improve .resolve_rev()

It follows `git checkout` logic now - if name can be unambiguously
resolved across known remotes then it's done.
2020-02-11 19:43:15 +02:00
Ivan Shcheklein 5e98a702a2 GDrive improvements and code cleanup (#3294)
* cleanup gdrive code, fix remote ids resolving

* fixes after testing GDrive cleanup

* address CodeClimate complexity, simplify a litte bit

* address CodeClimate complexity, further simplify complexity

* further refactoring need to fix GDrive tests

* tests: minor setup gdrive cloud simplification
2020-02-11 19:37:03 +02:00
Ruslan Kuprieiev 2361bceb70 repro: fix bug in _get_active_graph (#3301)
We are currently removing all of the deps stages from the DAG even if
those deps are referenced by some parallel branches of execution.
The new approach is to remove edges and then gradually remove nodes if
there are no more `in` edges left.

Discord context:
https://discordapp.com/channels/485586884165107732/485596304961962003/676464313942409265
2020-02-11 01:45:43 +02:00
Ruslan Kuprieiev 0c8897949e diff: use rev instead of ref (#3299)
Fixes #3245
2020-02-11 00:13:23 +02:00
Saugat Pachhai 8610c0c95e repo: raise exception if root_dir is not a directory path (#3295)
* repo: throw if root_dir in Repo.__init__() is not a directory path

When Repo is instantiated with invalid urls such as 'http://github.com/iterative/dvc.git',
it does os.path.realpath which is not intelligent enough and just concatenates with current
working directory, which is then traversed outward. It might happen that the outward folder
is a dvc directory which is not really what we want.

So, a check for root to be a directory is added. Otherwise, NotDvcRepoError is thrown with
a custom message.

* repo: set message from raise for NotDvcRepoError
2020-02-10 20:57:34 +02:00
Jorge Orpinel 7821fa6866 get/import: they don't work with any DVC project, only DVC repos (as well as Git repos) (#3277)
* get/import: update cmd and url arg descs with correct behavior

* get/import: update path arg desc as well (See previous commit)

* get/imp: udpate help output with simpler terms
per https://github.com/iterative/dvc/pull/3277#pullrequestreview-352650016
2020-02-10 19:43:00 +02:00
Casper da Costa-Luis 3d5b8ecdd5 add: auto convert absolute_path => relative (#2975)
* add: auto convert absolute_path => relative

- fixes #2955

* use path_isin

* catch-all absolute paths in repo

* unneeded leftover

* revert realtive imports for later

* output: move abs2relpath from Base to Local

* output:local:fix parse_path

* remove path_info

* test:local versus base abs2relpath

* test:fix remaining abs2relpath test

* output: local path relative to wdir

* output: local: modify path before inheritance

* output: local: minor inheritance fix

* test: local: split output test

* test: local: positively explicit
2020-02-10 18:35:24 +02:00
Ruslan Kuprieiev 65ee446642 dvc: bump to 0.83.0 0.83.0 2020-02-10 01:42:31 +02:00
Ruslan Kuprieiev d305803015 analytics: ignore request.post errors (#3292)
Users have been complaining about dvc analytics errors when they are not
connected to the internet.
2020-02-10 01:41:42 +02:00
Ruslan Kuprieiev 2786a4e59b logger: allow exc_info in all record types (#3290)
This better reflects the original logger formatter.

This also fixes annoying redundant newline between traceback and "need
any help" footer.
2020-02-09 22:22:20 +02:00
Ruslan Kuprieiev afcbc7e17a deps: update pydrive2 version 0.82.10 2020-02-09 18:48:38 +02:00
Ruslan Kuprieiev 0bb5a8210a dvc: bump to 0.82.10 2020-02-09 18:47:24 +02:00
Max Risuhin 90ee067972 gdrive: add gc command support (#3288) 2020-02-09 18:46:05 +02:00
Aljoscha SteffensandRuslan Kuprieiev e7b3297c2d check root .gitignore in '_ignored' (#3284)
* made '_ignored' check if the given file is already ignored by the root .gitignore. Added corresponding functional test

* code formatting

* Changed method input, adjusted tests, formatted code

* typo

* tests: use git logger fix in conftest

See https://github.com/iterative/dvc/issues/3167

* scm: git: remove redundant comments

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-02-06 19:52:27 +02:00
Alexander Schepanovski e28371f53a Fix blockers list in rwlock error message (#3285) 2020-02-06 16:25:17 +02:00
Mr. OutisandRuslan Kuprieiev 509ce3e16e rwlock: require info keys to be present in schema (#3283)
Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-02-05 19:24:41 +02:00
Ruslan Kuprieiev d9ed98b951 Merge pull request #3270 from mroutis/fix-3164
rwlock: include actionable in error message
2020-02-05 13:05:49 +02:00
Mr. Outis e6ed5cc6cf rwlock: check blockers on a single method 2020-02-04 20:41:56 -06:00
Ruslan Kuprieiev cb06a91368 dvc: bump to 0.82.9 0.82.9 2020-02-05 03:20:52 +02:00
Ruslan Kuprieiev 93d1b6dcb0 Merge pull request #3217 from fabiosantoscode/feature/1744-pipeline-eliminate-extra-connection
pipeline: show: outs: eliminate extra edges in DAG
2020-02-04 02:02:31 +02:00
Ruslan Kuprieiev ed04a9f5ff Merge pull request #3276 from fabiosantoscode/patch-1
commit: add missing space
2020-02-04 01:49:27 +02:00
Fábio Santos 758205a52a PR feedback 2020-02-03 18:45:44 +00:00
Fábio Santos 11ad0a9803 remove __ 2020-02-03 18:41:55 +00:00
Fábio Santos 2e091f67b5 commit: add missing space
The prompt shown when committing without `-f` had a missing space between sentences.
2020-02-03 17:30:18 +00:00
Ruslan Kuprieiev de1f67afbd Merge pull request #3244 from jorgeorpinel/master
diff/metics diff: update argument descriptions, et al.
2020-02-03 02:15:54 +02:00
Ruslan Kuprieiev 633e126ba9 dvc: bump to 0.82.8 0.82.8 2020-02-02 17:59:01 +02:00
Ruslan Kuprieiev 82eaa52fab Merge pull request #3274 from efiop/3248
remote: bring back manual close_pools()
2020-02-02 17:56:14 +02:00
Ruslan Kuprieiev 06ca3b7f82 remote: bring back manual close_pools()
We used to have this line previously, thinking that it was only a py2
issue, but it turned out that it happens on py3 as well. So bring this
line back for now.
2020-02-02 16:57:23 +02:00
Mr. Outis 1c17d02f57 rwlock: include actionable in error message
Fix: #3164
2020-02-01 20:22:20 -06:00
Jorge Orpinel c27f6efa65 diff: remove backquotes around "HEAD"
per https://github.com/iterative/dvc/pull/3244#pullrequestreview-351263562
2020-01-31 20:05:47 -06:00
Jorge Orpinel e70d5fdbc3 remove unnecessary line from PR template 2020-01-31 19:59:57 -06:00
Ruslan Kuprieiev 945a517f6c Merge pull request #3260 from mroutis/change-diff-format
diff: format summary to make it more human readable
2020-02-01 01:11:09 +02:00
Ruslan Kuprieiev 2674689442 dvc: bump to 0.82.7 0.82.7 2020-01-31 23:30:32 +02:00
Ruslan Kuprieiev fd6bd29303 setup: move pydot to install_requires
https://github.com/iterative/dvc/issues/3267
2020-01-31 22:51:41 +02:00
Ruslan Kuprieiev 92bbd437f3 Merge pull request #3268 from Suor/isdir
dvc: do not check for isdir on recursive out collect
2020-01-31 22:45:35 +02:00
Ruslan Kuprieiev 2613e122ff Merge pull request #3263 from casperdcl/shfmt
lint:beautysh pre-commit hook
2020-01-31 22:44:30 +02:00
Casper da Costa-Luis b0b69ce819 lint/pre-commit/ci: replace shfmt => beautysh 2020-01-31 19:14:51 +00:00
Casper da Costa-Luis 4e87bf9e0e try beautysh 2020-01-31 19:07:53 +00:00
Alexander Schepanovski 6545d2178b dvc: do not check for isdir on recursive out collect
The things is we create dirs as needed, so if outs are not checked out a
directory contraining nothing but outs may be absent.
2020-01-31 19:36:36 +07:00
Jorge Orpinel 6984546ae2 diff/metrics diff: simplify b_ref desc
per https://github.com/iterative/dvc/pull/3244#pullrequestreview-351111761
2020-01-30 18:25:11 -06:00