22 Commits
Author SHA1 Message Date
Ivan Shcheklein 2431ec6189 Rename dvcx to DataChain in DVC <> DataChain integration (#10611) 2024-11-05 12:27:45 +05:45
skshetry c62a0380e9 templating: support on top-level params/metrics/plots/artifacts/datasets (#10359)
* templating: support on top-level params/metrics/plots/artifacts/datasets

* make resolving top-level ordered

* prevent updating parametrized dataset

* fix dataset tests

* add tests

* fix comment

* support parametrizing artifacts name
2024-03-22 12:22:16 +00:00
pre-commit-ci[bot]andSaugat Pachhai 49709a4810 [pre-commit.ci] pre-commit autoupdate (#10350)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.2.2 → v0.3.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.2.2...v0.3.2)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix ruff errors

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2024-03-13 16:43:10 +05:45
skshetry 9a1bcd7f1d nit: fit statement/expr in the same line instead of hanging (#10267)
The code are wrapped due to us using line-length=79 before, but now some
of the code can easily fit in 88 chars.

The formatter did not change this automatically when we changed line-length,
because of magic trailing commas that gets added when they turn into multiple lines.

Black/ruff wrapping them in parens and splitting them into multiple lines
was making it difficult to read.
2024-01-30 18:23:04 +05:45
Piotr Bielak 23d2ed3d59 Unroll dict-comprehension in matrix test 2023-10-31 08:05:38 -05:00
Piotr Bielak fcc69ab6eb parsing: Add key to matrix stages
This commit adds the `key` property to the templaing context of `matrix`
stages. Similarly to `foreach` stages, the `key` is available using
`${key}` substitution.

Fixes #10043
2023-10-31 08:05:38 -05:00
skshetry d1160a28e0 dvc.yaml: add matrix-do (#9725)
* dvc.yaml: add matrix-do

* remove docs

* no do in matrix

* add types
2023-08-04 13:29:15 +05:45
skshetry fd4696cc50 Revert "drop stage-level vars" (#9647)
Revert "drop stage-level vars (#9556)"

This reverts commit 358c5e0c39.
2023-06-22 18:44:51 +05:45
skshetry 358c5e0c39 drop stage-level vars (#9556)
drop stage-level vars support
2023-06-07 11:10:08 +05:45
daniele 6c1b786564 parsing: fix foreach resolution with int keys (#8904) 2023-03-20 18:40:30 +05:45
skshetry a86470ccbc update black and change line-length to 88 (#8953) 2023-02-04 05:28:07 +02:00
skshetry 7ab4c59d70 test: run pytest-style checks (#8901) 2023-01-28 14:10:53 +05:45
David de la Iglesia Castro a33bbc97d5 parsing: Escape str interpolation in dict unpacking.
Use subprocess.list2cmdline (windows) / shlex.quote

Closes #8203
Closes #8370
2022-11-18 20:00:16 +01:00
David de la Iglesia Castro 909941308b parsing: Support dict unpacking in cmd.
Allow to use dictionaries as values for template interpolation but only inside the `cmd` key.

See tests/func/parsing/test_interpolated_entry.py::test_cmd_dict for detailed syntax.

Add `config.parsing` section for configuring behavior of ambiguous data types
like booleans and lists.
2022-07-05 09:02:15 +02:00
Ruslan Kuprieiev a80a85e6bd dvc: detach gitfs from local paths
Our gitfs paths are currently tied to the local path in which the cloned
repository resides in, which results in a lot of hacky assumptions throughout
the code.

This PR detaches gitfs from local paths completely, making dvc work with a
proper fsspec gitfs implementation.

Part of #7543
2022-05-11 18:17:39 +03:00
Saugat Pachhai (सौगात) 5eb16ab05e parsing: use relative paths; force absolute paths to relative 2022-04-18 14:19:56 +03:00
Batuhan Taskaya 9aec8f1926 dvc: get rid of path_info
Intermediate PR in removing path_info from our code base.

It is completely removed from the main codebase and is still
being used in tests, which will be migrated in the followups.

For fs-specific simple path manipulation, we've introduced
`fs.path` helper (similar to `os.path`).

Fixes #6774
2021-11-10 00:52:48 +02:00
Saugat Pachhai fbcc3f389c TmpDir: add serialize utilities (#6620) 2021-09-15 16:13:06 +05:45
Saugat Pachhai b9000ed926 pre-commit: update black to 21.5b1 (#6032)
* pre-commit: update black to 21.5b1

also updated others, but black is more noticeable

* remove magic trailing comma
2021-05-18 15:51:15 +00:00
Saugat Pachhai 3b3b5c8997 parsing: resolve relative paths when loading parameters (#5734)
* parsing: resolve relative paths when loading parameters

* test with git commits as well
2021-03-30 23:50:04 +03:00
Saugat Pachhai 11c94b10f2 disallow overriding item and key in the foreach loop (#5104)
* Allow reserving keys in the context to prevent overriding

* Use nullcontext from funcy (3.6 Python)

* Display path in error message
2020-12-15 19:06:39 +05:45
Saugat Pachhai 620d082b0b refactor resolver and optimize (#5077)
minor refactors, code comments

Rename interim to definitions

provide location of entry during init for definition

small fixups to set_temporarily

Refactor loop to use dict comprehension

Provide same interface for definitions

Make it work for composite lists

Check for sequence rather than a list

refactor tests

Fixes #4955
Fixes #4957
Fixes #4963
2020-12-14 19:22:15 +05:45