The falsy values were being skipped when merging params values
from dvc.yaml and dvc.lock, showing the values as always changed.
These values were never added to `info`, but were found in
`self.params` making them appear as changed.
Fixes#4184
* Add more tests according to .gitignore
* Test update
* Update test
* Solve question of path
* update one test
* Update 2 tests
* Add one tests
* comment those failed tests caused by other package
* parametrize some tests
Co-authored-by: karajan1001 <[email protected]>
Usually we need it to parametrize tests with, so there is no
reason to duplicate the code. I've left `local_remote` fixture
just because it is so convenient and often used in a lot of tests.
* [RFC] tests: make read_text() work with directories
This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.
```
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
```
This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we `trees_equal`
doesn't work.
* status: pass tree as a kwarg
Fixes#4150
Depends on https://github.com/iterative/dvc/pull/4155
This PR is just a showcase to see how we like it. I've converted a few
tests to it, but there will be much more to come.
```
assert (tmp_dir / "file") == "contents"
assert (tmp_dir / "dir").read_text() == {"subdir": {"file": "hello"}}
```
This approach is nicer because it allows us to use a declaratory
asserts instead of relying on fs files. The problem becomes much more
apparent once you get into remote workspaces, where we `trees_equal`
doesn't work.
* s3,gs: don't append "/" to prefixed path when listing objects
* remote: use optional prefix kwarg for walk_files
* if prefix is True, path will be walked as a prefix
* if prefix is False, path will be treated as a directory (trailing
slash will be appended for S3-like remotes)
* tests: add unit test for BaseRemoteTree.list_paths
* pylint: enable attribute-outside-init check
* fix tests
* Fix mock for time
This was an interesting one. I was mocking `time` on function definition
and then, using those later on test run. When all the tests were run, there
would be some delay that'd make tests fail. Hmmm ...
* tree: walk: dynamically load directories as we go
This allows us to save time by not loading directories that we won't be
using anyway. This also allows us to list directories for which we don't
need the access to remote.
* list: add support for tracked directories
* pylint: enable checks for imports
* ci: add github action for pylint
* enable pylint on pre-commit
* Install requirements
* use local pylint installation rather than of pre-commit's
* fix import errors
* more checks
* Silence unused imports
* Make a copy of dict
* remove unnecessary generators/comprehensions
* Use dict literals
* remove unused variables
* misc fixes
ignore .iteritems, is a false +ve
remove function call in default argument
* Flake8 config: complexity, show source on err, show err counts
* flake8: add extensions
* remove flake8-string-format
* Add flake8-debugger support
* Add flake8-string-format extension
* remove flake8-debugger
It's having some problems to install on Windows in Travis
* trees: BaseTree.walk should handle errors like os.walk
* use onerror callback if exceptions should be logged or raised
* repo.ls: use onerror for tree.walk
* remote: move constants into trees
* remote: replace Remote() Cache() helper functions
* output: update for remote changes
* cache/data_cloud: update for remote changes
* dependency: update for remote changes
* stage: update for remote changes
* remote: move remaining non-index methods into tree
* tests: update remote unit tests
* remote: LocalCache needs to lock remote index, not cache index
* fix bugs from moved PARAM_CHECKSUM/PARAM_RELPATH
* tests: repo tree open needs state context
* remote: make Remote.gc a class method
* operates on obj.tree, so that GC can be done with both remote and
cache instances
* bug fixes
* tests: update test_data_cloud
* bug fixes
* tests: update func tests
This commit preserves comments, meta and order inside a dvc.yaml
when `-f/--force` is used. But, it does not maintain comments/meta
for .dvc files.
Related: #4014
The relative path for external git directory was computed relative to
the root of the repo itself, which created a relative path which is
current working directory. So, all of the contents of the erepo to download
was dumped in the pwd rather than inside of it's own directory.
Fix#4033
* remote: move get_file_checksum() into tree
* makes RemoteTree and RepoTree consistent with regard to checksum
calculation
* remote: save() now takes explicit tree parameter
* when tree is remote.tree, save will be a move + link operation
(same as default existing behavior)
* when saving path from a different tree, save will be a copy operation
* tests: update for moved remote/tree functions
* remote: move get_checksum into tree
* remote: separate cloud remote and cache classes
* tests: update unit tests for remote/cache separation
* remote: cloud cache should extend cloud remote
* remote: move LocalRemote.get
* tests: update func tests
* dependency: update for moved get_checksum
* remote: fix state lookup bug
* dvcignore: ignore paths outside of CleanTree root
* remote: local remote/cache should always use its own working tree
* repo tree (CleanTree) instance should not be relevant to local cache
operations. When the dvcignore CleanTree is relevant, repo.tree should
be used. For local remote/cache operations, the local remote work_tree
should be used.
* state: tie state to local cache working tree
* tests: update tests
* state: use os.path.exists instead of WorkingTree.exists in some cases
Currently `RepoTree.open` will throw FileNotFoundError if you have a
file that is tracked by dvc, but which has no md5 or a cache for which
is missing, even though you have the file itself in your workspace.
This corresponds very nicely to the `--no-commit` or `--no-exec`
scenario, where you might not want to commit your artifacts to cache
just yet.
Fixes#3974
Also, makes fixes for `--overwrite` rename to `--force`
and also remove `remove_with_prompt` which was only being
used with `force\=True` anyway.
Fix#4003
Ruslan Kuprieiev
Saugat Pachhai
karajan1001andkarajan1001
Peter Rowlands (변기호)
Alexander Schepanovski
nik123andRuslan Kuprieiev
Casper da Costa-Luis
Paweł Redzyński
