Commit Graph
5622 Commits
Author SHA1 Message Date
Ruslan Kuprieiev ec88fa8dec [RFC] tests: make read_text() work with directories (#4155)
This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.

```
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
```

This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we `trees_equal`
doesn't work.
2020-07-03 16:16:43 +03:00
Ruslan Kuprieiev fb99f2e332 dvc: bump to 1.1.6 1.1.6 2020-07-03 03:05:16 +03:00
Ruslan Kuprieiev bb2cf74328 gdrive: pop prefix kwarg (#4157)
https://travis-ci.com/github/iterative/dvc/jobs/357157580#L8485
2020-07-03 03:04:26 +03:00
Ruslan Kuprieiev bd9eb8faba dvc: bump to 1.1.5 1.1.5 2020-07-03 01:06:51 +03:00
Peter Rowlands (변기호) fa16b56cba s3,gs: don't append "/" to prefixed path when listing objects (#4149)
* s3,gs: don't append "/" to prefixed path when listing objects

* remote: use optional prefix kwarg for walk_files

* if prefix is True, path will be walked as a prefix
* if prefix is False, path will be treated as a directory (trailing
  slash will be appended for S3-like remotes)

* tests: add unit test for BaseRemoteTree.list_paths
2020-07-02 20:04:04 +03:00
Ruslan Kuprieiev 07da9f1e5c tests: remotes: use TmpDir-like fixtures (#4140) 2020-07-02 04:46:40 +03:00
Casper da Costa-Luis 5d2ac9fca2 completion: update dependency (#4148)
* completion: use new simpler `shtab` API

* completion: use FILE|DIR
2020-07-02 00:15:39 +03:00
Ruslan Kuprieiev 27af048d3e azure: use knack directly instead of azure-cli-core (#4147) 1.1.4 2020-07-01 22:54:25 +03:00
Ruslan Kuprieiev 5d85692719 dvc: bump to 1.1.4 2020-07-01 22:53:56 +03:00
Ruslan Kuprieiev 6e66ddfcb9 dvc: bump to 1.1.3 1.1.3 2020-07-01 20:59:51 +03:00
Ruslan Kuprieiev 00c135f7d5 setup: limit flufl version
flufl.lock released 4.0 in which there is no finalize method anymore, hence
```
dvc/lock.py:143:16: E1101: Instance of 'HardlinkLock' has no 'finalize' member (no-member)
```
2020-07-01 01:55:30 +03:00
Ruslan Kuprieiev 8cbefaaa88 dvc: bump to 1.1.2 1.1.2 2020-07-01 01:32:51 +03:00
Ruslan Kuprieiev 0275a0d8d8 azure: use CLI config (#4137)
Fixes #4129
2020-06-30 14:13:25 +03:00
Dmitry Petrov b51ee18fc0 Reorder package widgets (#4138) 2020-06-30 13:50:49 +03:00
Peter Rowlands (변기호) c6a024f83b RepoTree: preserve relative paths in copytree() (#4133) 2020-06-30 15:09:22 +09:00
Saugat Pachhai 1c0f5cdf4b pylint: enable additional checks (#4122)
Created a separate pylint plugin to disable some checks for tests.
2020-06-29 19:15:05 +03:00
Alexander Schepanovski a7e825a038 metrics: skip missing metrics silently (#4130)
Reverts an issue caused by #4005
2020-06-29 19:00:46 +03:00
Ruslan Kuprieiev 7b97d7249d hdfs: implement makedirs separately (#4128)
Clouds that have a non-noop `makedirs` should implement it in their own
method. Hdfs somehow slipped under the radar and had `mkdir` used
explicitly in `copy` and `upload` methods.
2020-06-29 10:27:02 +03:00
Ruslan Kuprieiev 83a2afc7b0 setup: update shtab 1.1.1 2020-06-27 02:36:56 +03:00
Ruslan Kuprieiev 0a83027536 dvc: bump to 1.1.1 2020-06-27 02:36:20 +03:00
Ruslan Kuprieiev d809723bc9 dvc: add trailing newline in --show-md (#4124)
To finalize md table.

Fixes #4123
2020-06-27 00:02:18 +03:00
Ruslan Kuprieiev 35e0dd9206 dvc: bump to 1.1.0 1.1.0 2020-06-26 17:49:00 +03:00
Saugat Pachhai 047352027f pylint: enable attribute-outside-init check (#4119)
* pylint: enable attribute-outside-init check

* fix tests

* Fix mock for time

This was an interesting one. I was mocking `time` on function definition
and then, using those later on test run. When all the tests were run, there
would be some delay that'd make tests fail. Hmmm ...
2020-06-26 17:22:38 +03:00
Jorge OrpinelandRuslan Kuprieiev 81a4cea71a plots: improve non-out error message (#4115)
* plot: link to cmd ref in output

* plots: change non-plot output error msg
per https://github.com/iterative/dvc/pull/3994#discussion_r438917424

* Update dvc/repo/plots/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-25 23:53:21 +03:00
Saugat Pachhai ad306d7bf3 Enable most of the pylint checks (#4111) 2020-06-25 18:10:14 +03:00
Ruslan Kuprieiev d3457fb8f9 tests: data_cloud: migrate the leftovers to pytest (#4109)
Related to #1819
2020-06-25 16:13:43 +03:00
Ruslan Kuprieiev 59a143eb08 list: add support for tracked directories (#4108)
* tree: walk: dynamically load directories as we go

This allows us to save time by not loading directories that we won't be
using anyway. This also allows us to list directories for which we don't
need the access to remote.

* list: add support for tracked directories
2020-06-25 16:11:39 +03:00
Paweł RedzyńskiandJorge Orpinel 64c1321c9d un/freeze: adjust stage naming to 1.0 (#4102)
* un/freeze: adjust stage naming to 1.0

* Update dvc/command/freeze.py

* Update dvc/command/freeze.py

Co-authored-by: Jorge Orpinel <[email protected]>
2020-06-24 22:51:29 +03:00
Ruslan Kuprieiev a6ff46ce81 dvc: bump to 1.0.2 1.0.2 2020-06-24 20:10:26 +03:00
Saugat Pachhai b81ace54df Enable pylint checks (#4092)
* pylint: enable checks for imports

* ci: add github action for pylint

* enable pylint on pre-commit

* Install requirements

* use local pylint installation rather than of pre-commit's

* fix import errors

* more checks

* Silence unused imports

* Make a copy of dict
2020-06-24 18:02:39 +03:00
Ruslan Kuprieiev 8495b0521b rpm/deb/osxpkg: add url and license (#4098)
We are currently missing url and license metadata:
```
Name        : dvc
Arch        : x86_64
Version     : 0.94.0
Release     : 1
Size        : 172 M
Repo        : installed
From repo   : dvc
Summary     : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
URL         : http://example.com/no-uri-given
License     : unknown
Description : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
```

https://github.com/iterative/dvc/issues/4088#issuecomment-648270797
2020-06-23 21:46:39 +03:00
nik123 9fa5fb0baf ui: remove: .gitignore in cmd description (#4097) 2020-06-23 21:16:29 +03:00
Ruslan Kuprieiev efcf4e3312 git-hook: don't run when not in dvc repo (#4091)
Related to #4073
2020-06-23 16:32:28 +03:00
Ruslan Kuprieiev b3d72e11fe push/pull/fetch: unhide --run-cache (#4087)
Related to https://github.com/iterative/dvc.org/issues/1289
2020-06-23 14:41:34 +03:00
Saugat Pachhai 7d45a55072 tests: use pytest-lazyfixture (#4089) 2020-06-23 17:26:24 +05:45
Ruslan Kuprieiev 4181c0ec4a run: hide and forbid --file for multistage files (#4086)
Fixes #4084
2020-06-23 03:46:59 +03:00
Ruslan Kuprieiev 5cb775308d dvc: bump to 1.0.1 1.0.1 2020-06-22 23:10:33 +03:00
Ruslan Kuprieiev 2b7731e4f9 git: don't use multithreading for computhing dir hash (#4083)
GitPython is not threadsafe, which was causing issues when we were
computing hash for a directory.

Fixes #4079

Related to tree generalization #4050
2020-06-22 23:10:02 +03:00
Ruslan Kuprieiev 8f190a90ee git: make git_object_by_path private (#4082) 2020-06-22 22:27:39 +03:00
Ruslan Kuprieiev 57e476acc4 dvc: bump to 1.0.0 1.0.0 2020-06-22 02:10:24 +03:00
Casper da Costa-Luis 061513a2ae completion: misc fixes (#4076)
* command: add metavars

- makes help output look the same as before #3921

* deps: update shtab version

Part of #4070
2020-06-22 02:09:43 +03:00
Ruslan Kuprieiev 9f6f599101 import: fix no-exec help message 2020-06-20 13:31:09 +03:00
karajan1001andRuslan Kuprieiev 74930c01ec add --no-exec mode to import url (#4075)
* add --no-exec mode to import url

fix #3513

* Update tests/func/test_import_url.py

* Update tests/func/test_import_url.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-20 13:29:09 +03:00
Ruslan Kuprieiev a2f1367a9a dvc: bump to 1.0.0b6 1.0.0b6 2020-06-20 00:04:53 +03:00
nik123andRuslan Kuprieiev 5d97f92bdd remove: clean gitignore content (#4074)
* remove: clean gitignore content

* remove: refactor: using ignore_remove in remove

* Update dvc/output/base.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-19 19:38:18 +03:00
Saugat Pachhai 02b147b6f4 change codeclimate config (#4071) 2020-06-19 14:59:02 +03:00
Casper da Costa-Luis fa0f182028 completion: autogenerate (#3921)
* cli: split main_parser

* cli: basic parser tree printing

* cli: major completion tidy

* completion: more missing root commands

* completion: much automagic such beauty

- fixes #2985 automatically generate bash completion

* tests: bash completion

* tests: completion: tidy script a little

* minor tidy

* command: positional name consistency

* completion: bash: add logging

* double-check renames and consistency

* cli: add completion subcommand

* ci: completion: update test

* minor logging

* more completion logging verbosity

* fix completion of files

* add shtab

* purge unneeded completion => shtab

* completion: neaten UI

* completion: use `shtab>=0.0.2`

* completion: purge hardcoded completions!

* revert unneeded renames for now

* fix isort

* completion: tidy CLI API

* fix snap build

* fix snap build again

* snap: internally build completion

* allow repo-less completion

* command: tidy headless logic

* use CmdBaseNoRepo

* better DVC-file completion

* merge command.choices -> command.completion

* drop choices class

* remove completion optionals

* tidy docs
2020-06-19 04:54:20 +03:00
Ruslan Kuprieiev 15c787f2ac innosetup: commit images 2020-06-18 23:39:33 +03:00
Saugat Pachhai 4af3919062 Add flake8 extensions (#4067)
* remove unnecessary generators/comprehensions

* Use dict literals

* remove unused variables

* misc fixes

ignore .iteritems, is a false +ve
remove function call in default argument

* Flake8 config: complexity, show source on err, show err counts

* flake8: add extensions

* remove flake8-string-format

* Add flake8-debugger support

* Add flake8-string-format extension

* remove flake8-debugger

It's having some problems to install on Windows in Travis
2020-06-18 14:36:30 +03:00
Peter Rowlands (변기호) 3a469f2e93 dvcignore: ignore empty (None) patterns (#4066) 2020-06-18 14:31:30 +09:00