Commit Graph
20 Commits
Author SHA1 Message Date
skshetry 7ab4c59d70 test: run pytest-style checks (#8901) 2023-01-28 14:10:53 +05:45
Saugat Pachhai (सौगात) 570e3a5359 dvc.fs: move fs to dvc.objects.fs
Some of the APIs/FileSystems are reimported inside dvc.fs module.
2022-05-16 13:15:19 +03:00
Saugat Pachhai (सौगात) 5eb16ab05e parsing: use relative paths; force absolute paths to relative 2022-04-18 14:19:56 +03:00
Nikita Kodenko b6fa1178e7 tests: migrate unittest to pytest 2022-02-08 18:51:17 +09:00
Saugat Pachhai fbcc3f389c TmpDir: add serialize utilities (#6620) 2021-09-15 16:13:06 +05:45
Ruslan Kuprieiev 89b40afee7 fs: make constructor closer to fsspec (#5911)
* fs: don't use repo for jobs/hash_jobs

* fs: local: exists: use fsspec's LocalFileSystem

* fs: make constructor comply with fsspec
2021-05-12 22:03:09 +03:00
Ruslan Kuprieiev b0fd40f12c dvc: rename tree -> fs (#5461) 2021-02-13 15:40:44 +02:00
Saugat Pachhai 01a4473c20 parametrization: fix error when the file in vars is a directory (#5171)
* Fix error when the file in `vars` is a directory

* Fix on windows

* mock tree only
2020-12-28 23:03:09 +02:00
Saugat Pachhai 620d082b0b refactor resolver and optimize (#5077)
minor refactors, code comments

Rename interim to definitions

provide location of entry during init for definition

small fixups to set_temporarily

Refactor loop to use dict comprehension

Provide same interface for definitions

Make it work for composite lists

Check for sequence rather than a list

refactor tests

Fixes #4955
Fixes #4957
Fixes #4963
2020-12-14 19:22:15 +05:45
Saugat Pachhai 3a3fa7cdde parametrization: convert bool to string as "true"/"false" (#5004)
Previously, Python's `str()` was being used that
resulted in boolean transformed into "True"/"False".

Fixes #4996
2020-12-01 16:05:18 +05:45
Saugat Pachhai c65fdcabae tests: add error message tests for resolving stages (#4927) 2020-11-20 19:42:59 +05:45
Saugat Pachhai 7f66db7039 parametrization: improve 'vars' error message (#4924)
* Catches MergeError
* Catches params file not existing error
2020-11-20 17:46:34 +05:45
Saugat Pachhai 470f411cd6 parametrization: improve set error message (#4923) 2020-11-20 17:08:05 +05:45
Saugat Pachhai d721e919e4 parametrization: improve key error message (#4922) 2020-11-20 16:38:32 +05:45
Saugat Pachhai c03fbb0056 parametrization: raise appropriate error message on merge error (#4921) 2020-11-20 16:04:33 +05:45
Saugat Pachhai 40c3bbd1ba Change use and vars to just vars (#4880)
The schema is changed of `vars` to accomodate the both use of
`imports` and setting variables locally.

Now:
```
vars:
  - params.yaml
  - foo: foo
    bar: bar
```

Also, same distinction occurs for the stages.
2020-11-20 09:19:13 +05:45
Saugat Pachhai 4220baa602 parametrization: Support keys of the dictionary to be interpolated (#4869) 2020-11-10 17:32:58 +05:45
Saugat Pachhai 9f1330a387 Fix merge_update not converting data to Node object (#4846)
We wrap every data to one of the `Node` object. But, since we were
passing `self.data` instead of `self`, it was skipping the conversion
and instead was just trying to set data as-is, which would have make
our latter assumptions incorrect, as other codebases assume it to be
a `Node`.

I have added asserts in a few places. Also, I noticed that, on a few
test assumptions, when `resolving`, we were not converting values
back to it's original form (`CtxDict` -> `dict`, `CtxList` -> `list`
and `Value` -> _). So, I have added `node.value` property, as it is
a complement of the above fix.

This caused a ripple effect for the changes in the API as `foreach`
now resolved and changed the data to the original form,
which prevents us from tracking the data later in the "field".
So, `resolve`/`select` got a real `unwrap` option that should
reliably work for all kinds of `Node` objects.
2020-11-07 03:02:24 +02:00
Saugat Pachhai aa7b4c6340 Add tests for resolving stage data (#4753)
* Add test, and fix resolving of paths for params

Improve merging of vars and imported params, improve tracking
Resolve wdir correctly

* Fix windows relpath

* isort imports
2020-10-21 05:47:48 +05:45
Saugat Pachhai b0ae5e2251 Support parametrization from params or vars section in dvc.yaml (#4751)
* Implement importing from params

* Change varname

* Add tests

* fix pylint issues
2020-10-20 04:16:13 -07:00