The falsy values were being skipped when merging params values
from dvc.yaml and dvc.lock, showing the values as always changed.
These values were never added to `info`, but were found in
`self.params` making them appear as changed.
Fixes#4184
* BaseRemote: start moving tree functions into BaseRemoteTree
* remote: add LocalRemoteTree
* remote: add cloud remote trees
* remote: move remove(), makedirs() into tree
* remote: move move(), copy(), symlink(), hardlink(), reflink() into tree
* fix attributes for moved remote methods
* tests: update unit tests for moved remote tree methods
* tests: mv unit/remote/test_remote_dir.py unit/remote/test_remote_tree.py
* fix attributes for moved remote tree functions
* tests: update func tests for moved remote tree methods
* test fixes
* fix DS warnings
* azure: support external dependencies and outputs
- Implement get_file_checksum with get_etag
- Implement a DependencyAzure and OutputAzure classes
- Add AzureOutput to OUTS, OUTS_MAP and CHECKSUMS_SCHEMA
- Add AzureDependency to DEPS and DEPS_MAP
- Add unit tests for AzureOutput and AzureDependency
Fixes#3540.
* Use content_md5 instead of etag
The etag is different for the same file in different locations.
* Use etag and remove external outputs for Azure
Due to inconsistencies with Azure's Content-MD5 property, it is not
stable enough to be used at the moment. Using the etag offers enough
reliability to support external dependencies. See also:
https://discordapp.com/channels/485586884165107732/565699007037571084/713462619469643796
Currently `dvc run -p angles` is broken when you use it with `params.yaml`
that looks like this:
```
angles:
- 2
- 3
- 100
- 3.2
- 36.2
- 3.6
```
because we are incorrectly using `flatten` instead of simply using some
xpath-like library to access params that are specified.
* Work in progress
* added file parsing and name validation + adjust schema
* Exceptions on bad input
* Support multiple parameters
* Support multi 's in
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
* fix formatting
* dep: param: fix schema
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* run: check if --params is specified
* dvc: rework param handling
* params: fix exc formatting
* params: support basic nested configs
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Work in progress
* added file parsing and name validation + adjust schema
* Exceptions on bad input
* Support multiple parameters
* Support multi 's in
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
* Restyled by black
Co-authored-by: elgehelge <[email protected]>
Co-authored-by: Restyled.io <[email protected]>
Related to #1629
Speeds up HDFS tests significantly. E.g. ExternalHDFS test goes from 600
sec to 190sec. test_open_external[HDFS] from 160sec to 3sec.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Saugat Pachhai
Peter Rowlands (변기호)
Steffan Sluis
Casper da Costa-Luis
nik123
Ruslan Kuprieiev
Helge Munk JacobsenandRuslan Kuprieiev
Mr. Outis
Ivan Shcheklein