Commit Graph
5632 Commits
Author SHA1 Message Date
Ruslan Kuprieiev b8a42939ad config: use repo.tree (#4190)
Fixes #4188
2020-07-12 06:45:04 +03:00
Ruslan Kuprieiev 375998c664 tests: hdfs: use hadoop/docker fixtures (#4183)
Fixed #4054
2020-07-10 21:38:48 +03:00
Ruslan Kuprieiev 39989ad233 dvc: bump to 1.1.8 1.1.8 2020-07-10 21:27:45 +03:00
Saugat Pachhai fe1d7e9f1d params: fix skipping of params dvc.lock when it's a falsy value (#4185)
The falsy values were being skipped when merging params values
from dvc.yaml and dvc.lock, showing the values as always changed.

These values were never added to `info`, but were found in
`self.params` making them appear as changed.

Fixes #4184
2020-07-09 21:29:26 +03:00
Ruslan Kuprieiev 0899b277c0 tests: launch oss/azure containers from fixtures (#4178)
* tests: launch oss/azure containers from fixtures

Related to #4054

* ci: hdfs: remove unneeded fix

We are using pyarrow 0.17.0 already.
2020-07-08 19:54:17 +03:00
karajan1001andkarajan1001 253a0eae13 Add more tests according to gitignore (#4166)
* Add more tests according to .gitignore

* Test update

* Update test

* Solve question of path

* update one test

* Update 2 tests

* Add one tests

* comment those failed tests caused by other package

* parametrize some tests

Co-authored-by: karajan1001 <[email protected]>
2020-07-07 23:04:52 +03:00
Ruslan Kuprieiev 8964967af0 cli: show subcommand-specific help (#4173)
Fixes #4043
2020-07-07 13:54:41 +03:00
Ruslan Kuprieiev 520e01f113 dvc: bump to 1.1.7 1.1.7 2020-07-04 01:47:07 +03:00
Ruslan Kuprieiev 30212f886b tests: create remote fixture (#4163)
Usually we need it to parametrize tests with, so there is no
reason to duplicate the code. I've left `local_remote` fixture
just because it is so convenient and often used in a lot of tests.
2020-07-03 22:30:56 +03:00
Ruslan Kuprieiev 8dd264c025 status: pass tree as a kwarg (#4156)
* [RFC] tests: make read_text() work with directories

This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.

```
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
```

This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we `trees_equal`
doesn't work.

* status: pass tree as a kwarg

Fixes #4150

Depends on https://github.com/iterative/dvc/pull/4155
2020-07-03 16:17:01 +03:00
Ruslan Kuprieiev ec88fa8dec [RFC] tests: make read_text() work with directories (#4155)
This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.

```
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
```

This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we `trees_equal`
doesn't work.
2020-07-03 16:16:43 +03:00
Ruslan Kuprieiev fb99f2e332 dvc: bump to 1.1.6 1.1.6 2020-07-03 03:05:16 +03:00
Ruslan Kuprieiev bb2cf74328 gdrive: pop prefix kwarg (#4157)
https://travis-ci.com/github/iterative/dvc/jobs/357157580#L8485
2020-07-03 03:04:26 +03:00
Ruslan Kuprieiev bd9eb8faba dvc: bump to 1.1.5 1.1.5 2020-07-03 01:06:51 +03:00
Peter Rowlands (변기호) fa16b56cba s3,gs: don't append "/" to prefixed path when listing objects (#4149)
* s3,gs: don't append "/" to prefixed path when listing objects

* remote: use optional prefix kwarg for walk_files

* if prefix is True, path will be walked as a prefix
* if prefix is False, path will be treated as a directory (trailing
  slash will be appended for S3-like remotes)

* tests: add unit test for BaseRemoteTree.list_paths
2020-07-02 20:04:04 +03:00
Ruslan Kuprieiev 07da9f1e5c tests: remotes: use TmpDir-like fixtures (#4140) 2020-07-02 04:46:40 +03:00
Casper da Costa-Luis 5d2ac9fca2 completion: update dependency (#4148)
* completion: use new simpler `shtab` API

* completion: use FILE|DIR
2020-07-02 00:15:39 +03:00
Ruslan Kuprieiev 27af048d3e azure: use knack directly instead of azure-cli-core (#4147) 1.1.4 2020-07-01 22:54:25 +03:00
Ruslan Kuprieiev 5d85692719 dvc: bump to 1.1.4 2020-07-01 22:53:56 +03:00
Ruslan Kuprieiev 6e66ddfcb9 dvc: bump to 1.1.3 1.1.3 2020-07-01 20:59:51 +03:00
Ruslan Kuprieiev 00c135f7d5 setup: limit flufl version
flufl.lock released 4.0 in which there is no finalize method anymore, hence
```
dvc/lock.py:143:16: E1101: Instance of 'HardlinkLock' has no 'finalize' member (no-member)
```
2020-07-01 01:55:30 +03:00
Ruslan Kuprieiev 8cbefaaa88 dvc: bump to 1.1.2 1.1.2 2020-07-01 01:32:51 +03:00
Ruslan Kuprieiev 0275a0d8d8 azure: use CLI config (#4137)
Fixes #4129
2020-06-30 14:13:25 +03:00
Dmitry Petrov b51ee18fc0 Reorder package widgets (#4138) 2020-06-30 13:50:49 +03:00
Peter Rowlands (변기호) c6a024f83b RepoTree: preserve relative paths in copytree() (#4133) 2020-06-30 15:09:22 +09:00
Saugat Pachhai 1c0f5cdf4b pylint: enable additional checks (#4122)
Created a separate pylint plugin to disable some checks for tests.
2020-06-29 19:15:05 +03:00
Alexander Schepanovski a7e825a038 metrics: skip missing metrics silently (#4130)
Reverts an issue caused by #4005
2020-06-29 19:00:46 +03:00
Ruslan Kuprieiev 7b97d7249d hdfs: implement makedirs separately (#4128)
Clouds that have a non-noop `makedirs` should implement it in their own
method. Hdfs somehow slipped under the radar and had `mkdir` used
explicitly in `copy` and `upload` methods.
2020-06-29 10:27:02 +03:00
Ruslan Kuprieiev 83a2afc7b0 setup: update shtab 1.1.1 2020-06-27 02:36:56 +03:00
Ruslan Kuprieiev 0a83027536 dvc: bump to 1.1.1 2020-06-27 02:36:20 +03:00
Ruslan Kuprieiev d809723bc9 dvc: add trailing newline in --show-md (#4124)
To finalize md table.

Fixes #4123
2020-06-27 00:02:18 +03:00
Ruslan Kuprieiev 35e0dd9206 dvc: bump to 1.1.0 1.1.0 2020-06-26 17:49:00 +03:00
Saugat Pachhai 047352027f pylint: enable attribute-outside-init check (#4119)
* pylint: enable attribute-outside-init check

* fix tests

* Fix mock for time

This was an interesting one. I was mocking `time` on function definition
and then, using those later on test run. When all the tests were run, there
would be some delay that'd make tests fail. Hmmm ...
2020-06-26 17:22:38 +03:00
Jorge OrpinelandRuslan Kuprieiev 81a4cea71a plots: improve non-out error message (#4115)
* plot: link to cmd ref in output

* plots: change non-plot output error msg
per https://github.com/iterative/dvc/pull/3994#discussion_r438917424

* Update dvc/repo/plots/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-25 23:53:21 +03:00
Saugat Pachhai ad306d7bf3 Enable most of the pylint checks (#4111) 2020-06-25 18:10:14 +03:00
Ruslan Kuprieiev d3457fb8f9 tests: data_cloud: migrate the leftovers to pytest (#4109)
Related to #1819
2020-06-25 16:13:43 +03:00
Ruslan Kuprieiev 59a143eb08 list: add support for tracked directories (#4108)
* tree: walk: dynamically load directories as we go

This allows us to save time by not loading directories that we won't be
using anyway. This also allows us to list directories for which we don't
need the access to remote.

* list: add support for tracked directories
2020-06-25 16:11:39 +03:00
Paweł RedzyńskiandJorge Orpinel 64c1321c9d un/freeze: adjust stage naming to 1.0 (#4102)
* un/freeze: adjust stage naming to 1.0

* Update dvc/command/freeze.py

* Update dvc/command/freeze.py

Co-authored-by: Jorge Orpinel <[email protected]>
2020-06-24 22:51:29 +03:00
Ruslan Kuprieiev a6ff46ce81 dvc: bump to 1.0.2 1.0.2 2020-06-24 20:10:26 +03:00
Saugat Pachhai b81ace54df Enable pylint checks (#4092)
* pylint: enable checks for imports

* ci: add github action for pylint

* enable pylint on pre-commit

* Install requirements

* use local pylint installation rather than of pre-commit's

* fix import errors

* more checks

* Silence unused imports

* Make a copy of dict
2020-06-24 18:02:39 +03:00
Ruslan Kuprieiev 8495b0521b rpm/deb/osxpkg: add url and license (#4098)
We are currently missing url and license metadata:
```
Name        : dvc
Arch        : x86_64
Version     : 0.94.0
Release     : 1
Size        : 172 M
Repo        : installed
From repo   : dvc
Summary     : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
URL         : http://example.com/no-uri-given
License     : unknown
Description : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
```

https://github.com/iterative/dvc/issues/4088#issuecomment-648270797
2020-06-23 21:46:39 +03:00
nik123 9fa5fb0baf ui: remove: .gitignore in cmd description (#4097) 2020-06-23 21:16:29 +03:00
Ruslan Kuprieiev efcf4e3312 git-hook: don't run when not in dvc repo (#4091)
Related to #4073
2020-06-23 16:32:28 +03:00
Ruslan Kuprieiev b3d72e11fe push/pull/fetch: unhide --run-cache (#4087)
Related to https://github.com/iterative/dvc.org/issues/1289
2020-06-23 14:41:34 +03:00
Saugat Pachhai 7d45a55072 tests: use pytest-lazyfixture (#4089) 2020-06-23 17:26:24 +05:45
Ruslan Kuprieiev 4181c0ec4a run: hide and forbid --file for multistage files (#4086)
Fixes #4084
2020-06-23 03:46:59 +03:00
Ruslan Kuprieiev 5cb775308d dvc: bump to 1.0.1 1.0.1 2020-06-22 23:10:33 +03:00
Ruslan Kuprieiev 2b7731e4f9 git: don't use multithreading for computhing dir hash (#4083)
GitPython is not threadsafe, which was causing issues when we were
computing hash for a directory.

Fixes #4079

Related to tree generalization #4050
2020-06-22 23:10:02 +03:00
Ruslan Kuprieiev 8f190a90ee git: make git_object_by_path private (#4082) 2020-06-22 22:27:39 +03:00
Ruslan Kuprieiev 57e476acc4 dvc: bump to 1.0.0 1.0.0 2020-06-22 02:10:24 +03:00