5594 Commits
Author SHA1 Message Date
Ruslan Kuprieiev a6ff46ce81 dvc: bump to 1.0.2 1.0.2 2020-06-24 20:10:26 +03:00
Saugat Pachhai b81ace54df Enable pylint checks (#4092)
* pylint: enable checks for imports

* ci: add github action for pylint

* enable pylint on pre-commit

* Install requirements

* use local pylint installation rather than of pre-commit's

* fix import errors

* more checks

* Silence unused imports

* Make a copy of dict
2020-06-24 18:02:39 +03:00
Ruslan Kuprieiev 8495b0521b rpm/deb/osxpkg: add url and license (#4098)
We are currently missing url and license metadata:
```
Name        : dvc
Arch        : x86_64
Version     : 0.94.0
Release     : 1
Size        : 172 M
Repo        : installed
From repo   : dvc
Summary     : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
URL         : http://example.com/no-uri-given
License     : unknown
Description : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
```

https://github.com/iterative/dvc/issues/4088#issuecomment-648270797
2020-06-23 21:46:39 +03:00
nik123 9fa5fb0baf ui: remove: .gitignore in cmd description (#4097) 2020-06-23 21:16:29 +03:00
Ruslan Kuprieiev efcf4e3312 git-hook: don't run when not in dvc repo (#4091)
Related to #4073
2020-06-23 16:32:28 +03:00
Ruslan Kuprieiev b3d72e11fe push/pull/fetch: unhide --run-cache (#4087)
Related to https://github.com/iterative/dvc.org/issues/1289
2020-06-23 14:41:34 +03:00
Saugat Pachhai 7d45a55072 tests: use pytest-lazyfixture (#4089) 2020-06-23 17:26:24 +05:45
Ruslan Kuprieiev 4181c0ec4a run: hide and forbid --file for multistage files (#4086)
Fixes #4084
2020-06-23 03:46:59 +03:00
Ruslan Kuprieiev 5cb775308d dvc: bump to 1.0.1 1.0.1 2020-06-22 23:10:33 +03:00
Ruslan Kuprieiev 2b7731e4f9 git: don't use multithreading for computhing dir hash (#4083)
GitPython is not threadsafe, which was causing issues when we were
computing hash for a directory.

Fixes #4079

Related to tree generalization #4050
2020-06-22 23:10:02 +03:00
Ruslan Kuprieiev 8f190a90ee git: make git_object_by_path private (#4082) 2020-06-22 22:27:39 +03:00
Ruslan Kuprieiev 57e476acc4 dvc: bump to 1.0.0 1.0.0 2020-06-22 02:10:24 +03:00
Casper da Costa-Luis 061513a2ae completion: misc fixes (#4076)
* command: add metavars

- makes help output look the same as before #3921

* deps: update shtab version

Part of #4070
2020-06-22 02:09:43 +03:00
Ruslan Kuprieiev 9f6f599101 import: fix no-exec help message 2020-06-20 13:31:09 +03:00
karajan1001andRuslan Kuprieiev 74930c01ec add --no-exec mode to import url (#4075)
* add --no-exec mode to import url

fix #3513

* Update tests/func/test_import_url.py

* Update tests/func/test_import_url.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-20 13:29:09 +03:00
Ruslan Kuprieiev a2f1367a9a dvc: bump to 1.0.0b6 1.0.0b6 2020-06-20 00:04:53 +03:00
nik123andRuslan Kuprieiev 5d97f92bdd remove: clean gitignore content (#4074)
* remove: clean gitignore content

* remove: refactor: using ignore_remove in remove

* Update dvc/output/base.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-19 19:38:18 +03:00
Saugat Pachhai 02b147b6f4 change codeclimate config (#4071) 2020-06-19 14:59:02 +03:00
Casper da Costa-Luis fa0f182028 completion: autogenerate (#3921)
* cli: split main_parser

* cli: basic parser tree printing

* cli: major completion tidy

* completion: more missing root commands

* completion: much automagic such beauty

- fixes #2985 automatically generate bash completion

* tests: bash completion

* tests: completion: tidy script a little

* minor tidy

* command: positional name consistency

* completion: bash: add logging

* double-check renames and consistency

* cli: add completion subcommand

* ci: completion: update test

* minor logging

* more completion logging verbosity

* fix completion of files

* add shtab

* purge unneeded completion => shtab

* completion: neaten UI

* completion: use `shtab>=0.0.2`

* completion: purge hardcoded completions!

* revert unneeded renames for now

* fix isort

* completion: tidy CLI API

* fix snap build

* fix snap build again

* snap: internally build completion

* allow repo-less completion

* command: tidy headless logic

* use CmdBaseNoRepo

* better DVC-file completion

* merge command.choices -> command.completion

* drop choices class

* remove completion optionals

* tidy docs
2020-06-19 04:54:20 +03:00
Ruslan Kuprieiev 15c787f2ac innosetup: commit images 2020-06-18 23:39:33 +03:00
Saugat Pachhai 4af3919062 Add flake8 extensions (#4067)
* remove unnecessary generators/comprehensions

* Use dict literals

* remove unused variables

* misc fixes

ignore .iteritems, is a false +ve
remove function call in default argument

* Flake8 config: complexity, show source on err, show err counts

* flake8: add extensions

* remove flake8-string-format

* Add flake8-debugger support

* Add flake8-string-format extension

* remove flake8-debugger

It's having some problems to install on Windows in Travis
2020-06-18 14:36:30 +03:00
Peter Rowlands (변기호) 3a469f2e93 dvcignore: ignore empty (None) patterns (#4066) 2020-06-18 14:31:30 +09:00
Ruslan Kuprieiev 51a8c782ae tests: create remote fixtures (#4063)
* tests: use remote fixtures

* tests: use remote fixtures for data cloud tests

* tests: treat remotes as tmp dirs
2020-06-18 01:36:13 +03:00
Ruslan Kuprieiev b579d06c8e dvc: bump to 1.0.0b5 1.0.0b5 2020-06-18 00:33:53 +03:00
Alexander Schepanovski 039cd49259 plots: clean up template anchors and collection (#4061) 2020-06-17 21:34:43 +03:00
Peter Rowlands (변기호) f516dd252a trees: handle tree.walk errors the same way as os.walk (#4059)
* trees: BaseTree.walk should handle errors like os.walk

* use onerror callback if exceptions should be logged or raised

* repo.ls: use onerror for tree.walk
2020-06-17 13:25:14 +03:00
Banti Gheneti 706dd05e8d remote/gs: fix url signing when using GoogleStorage on GoogleComputeEngine (#4057)
Fixes #4056
2020-06-17 13:24:02 +03:00
Saugat Pachhai 5d957b1830 cache relpath and path_in_repo (#4053) 2020-06-17 04:28:10 +03:00
Saugat Pachhai 387867fbbe optimize: remove fsync when dumping rwlock data (#4052)
Fixes #3653
2020-06-16 17:55:36 +03:00
Ruslan Kuprieiev 7c6351d039 dvc: bump to 1.0.0b4 1.0.0b4 2020-06-16 15:17:19 +03:00
Ruslan Kuprieiev 613ca0bdb5 get-url/import-url: save dependency before downloading (#4049)
This allows us to properly process the errors like missing target.

Fixes #3897
2020-06-16 13:05:21 +03:00
Peter Rowlands (변기호) 0fb9870de6 remote: rename "checksum" -> "hash" (#4051)
* remote: s/get_checksum/get_hash/

* remote/tree: s/get_file_checksum/get_file_hash/

* remote: s/get_dir_checksum/get_dir_hash/

* remote: s/checksum_to_path/hash_to_path/

* remote: s/path_to_checksum/path_to_hash/

* remote: cleanup remaining checksum references

* remote.index: s/checksum/hash/

* note: does not rename "checksum" field in the index DB schema
2020-06-16 12:16:51 +03:00
Ruslan Kuprieiev 73e60a8c13 Delete pre-commit.yaml 2020-06-15 23:35:10 +03:00
Ruslan Kuprieiev 9d1dde4051 Create pre-commit.yaml 2020-06-15 23:32:39 +03:00
Ruslan Kuprieiev 38dc0f85d4 dvc: bump to 1.0.0b3 1.0.0b3 2020-06-15 22:15:57 +03:00
Paweł Redzyński d2dbe9649c plots: add smooth template (#4046) 2020-06-15 22:15:27 +03:00
Ruslan Kuprieiev ece6f2d3ee dvc: bump to 1.0.0b2 1.0.0b2 2020-06-15 21:36:05 +03:00
Ruslan Kuprieiev b672622324 plots: make sure template is using data anchor (#4045)
Fixes #3969
2020-06-15 21:14:07 +03:00
Ruslan Kuprieiev c9391df313 plots: modify: handle --unset for nonexistent options (#4044)
* tests: plots: reorginize the structure

* plots: modify: handle --unset for nonexistent options

Fixes #3933
2020-06-15 20:31:08 +03:00
Peter Rowlands (변기호) a0c229cf9b remote: continue remote/cache/tree refactoring (#4041)
* remote: move constants into trees

* remote: replace Remote() Cache() helper functions

* output: update for remote changes

* cache/data_cloud: update for remote changes

* dependency: update for remote changes

* stage: update for remote changes

* remote: move remaining non-index methods into tree

* tests: update remote unit tests

* remote: LocalCache needs to lock remote index, not cache index

* fix bugs from moved PARAM_CHECKSUM/PARAM_RELPATH

* tests: repo tree open needs state context

* remote: make Remote.gc a class method

* operates on obj.tree, so that GC can be done with both remote and
  cache instances

* bug fixes

* tests: update test_data_cloud

* bug fixes

* tests: update func tests
2020-06-15 16:22:55 +03:00
karajan1001 a59f90fac9 Performance improvement of dvcignore (#3967)
fix#3869
1.Use big regex.

* Solve windows

* add rule order test

* Solve ignore order

* remove list comprehensions
2020-06-15 11:37:01 +02:00
Ruslan Kuprieiev bba502314d dvc: bump to 1.0.0b1 1.0.0b1 2020-06-14 20:44:44 +03:00
Saugat Pachhai bc4bf66d17 (bugfix) get/import: fix broken progress bar (#4039)
PR #3672 (6d8499e) extended `LocalRemote::_get_plans` to return one
`checksums` too. As all of the args from `_get_plans` was passed
down to `download()`, it recognized extra arg of checksum as
`no_progress_bar` due to which it became True and stopped showing
progress bar at all.

Fix #3874
2020-06-14 20:42:53 +03:00
f7348af1e0 metrics & plots: help output improvements (#3963)
* metrics & plots: help output improvements

* metrics & plots: ui: updated help messages

* refactor: removed redundant parentheses

* metrics: ui: updated 'targets' description in 'metrics show'

* metrics: ui: reordered params to match the order in cmd ref

* plots: ui: params modified to match cmd ref

* plots: ui: updated diff help

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Jorge Orpinel <[email protected]>
2020-06-14 20:39:33 +03:00
Casper da Costa-Luis 4d140a9944 gdrive: add open (#3916)
* gdrive: add open

Fixes #3408
Related #2865
Fixes #3897

* dependency: add gdrive

* test: api: open: gdrive

* rollback pydrive dependency

* Revert "dependency: add gdrive"

This reverts commit fd333261d2e9fe1364cc278138c79890a1f5fe7a.

* tests: remotes: GDrive: fix get_url

* tests: api: fully test GDrive

* minor typo fix

* attempt gdrive tests fix

* fix gdrive credentials config

* replace GDrivePathNotFound => FileMissingError

* fix test_open_external[GDrive]

* add ensure_dir_scm

* minor exception fix
2020-06-13 23:53:25 +03:00
Casper da Costa-Luis 2ba73daedc snap: make future major version upgrades easier (#4038)
Related: #3999
Related: #4037
2020-06-13 23:27:03 +03:00
Ruslan Kuprieiev 7a31a35cbf dvc: bump to 1.0.0b0 1.0.0b0 2020-06-13 00:56:14 +03:00
Saugat Pachhai 24a14563cc run: preserve order when -f/--force overwrite stage (#4035)
This commit preserves comments, meta and order inside a dvc.yaml
when `-f/--force` is used. But, it does not maintain comments/meta
for .dvc files.

Related: #4014
2020-06-13 00:55:15 +03:00
Ruslan Kuprieiev a1d3440f85 metrics: diff: introduce --precision (#4026)
Rounding to 5 digits by default.

Fixes #3904
2020-06-13 00:54:42 +03:00
Saugat Pachhai 45dcb7c27d Update dag.py (#4036) 2020-06-12 20:33:19 +03:00