Commit Graph
24 Commits
Author SHA1 Message Date
skshetry c223cdaecd params/metrics: do not fail during dir expansion (#10375)
params/metrics: assume file when dir expansion raises error

Closes https://github.com/iterative/dvc/issues/10018.

We have to be careful when expanding paths. This PR fall backs to assume it
as a file when either of `isdir` and `find` raises error due to some mishaps.
The consequence of this assumption happens when we try to read the file.
In that case, if the directory exists, and we happen to try to open it
as a file, dvc will raise `IsADirectoryError`, but that is handled correctly
and won't fail catastrophically like this did.
2024-03-28 07:30:12 +05:45
skshetry c62a0380e9 templating: support on top-level params/metrics/plots/artifacts/datasets (#10359)
* templating: support on top-level params/metrics/plots/artifacts/datasets

* make resolving top-level ordered

* prevent updating parametrized dataset

* fix dataset tests

* add tests

* fix comment

* support parametrizing artifacts name
2024-03-22 12:22:16 +00:00
skshetry 9a1bcd7f1d nit: fit statement/expr in the same line instead of hanging (#10267)
The code are wrapped due to us using line-length=79 before, but now some
of the code can easily fit in 88 chars.

The formatter did not change this automatically when we changed line-length,
because of magic trailing commas that gets added when they turn into multiple lines.

Black/ruff wrapping them in parens and splitting them into multiple lines
was making it difficult to read.
2024-01-30 18:23:04 +05:45
skshetry f2673acb53 metrics/params: support reading from cached files and directories, use repo-relative path consistently (#9909)
* metrics

* fix path handling

* fix dvc metrics diff

* params show/diff

* share common utils

* remove onerror_collect from exp_show

* extract read_params

* move around code

* fix path handling on Windows

* rename _show to _gather_metrics

* remove _collect_vars

* add tests

* Revert "remove _collect_vars"

This reverts commit 6d2cb500c32a882497406d66fb13e613790c3436.

* bring back _collect_vars

* fix tests

* fix paths
2023-09-11 17:41:37 +05:45
skshetry 26152292aa remove support for reading and generating Dvcfile (#9546) 2023-06-06 15:29:10 +05:45
skshetry a86470ccbc update black and change line-length to 88 (#8953) 2023-02-04 05:28:07 +02:00
skshetry da27174e02 dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups, etc. (#8812)
dvcfile: rename PipelineFile to ProjectFile, get rid of Dvcfile class, cleanups etc
2023-01-15 12:45:10 +05:45
Ruslan Kuprieiev 0d71194741 dvc: normalize targets before entering brancher
Fixes #7888
2022-07-04 16:34:49 +03:00
David de la Iglesia Castro f106a68424 params.show: Fix deps for stages using same params file. 2022-06-21 20:52:19 +03:00
Saugat Pachhai (सौगात) a3bab48f41 params: allow tracking file 2022-03-21 13:47:44 +05:45
b652670cbf Dependabot/pip/pylint 2.11.1 (#6843)
* build(deps): Bump pylint from 2.9.6 to 2.11.1

Bumps [pylint](https://github.com/PyCQA/pylint) from 2.9.6 to 2.11.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/main/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.9.6...v2.11.1)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix pylint problems

* Updation after rebase.

* system: use binary mode

* Set remote execution script encoding type locale

* Update dvc/machine/__init__.py

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-10-20 14:34:10 +03:00
Gao 6de3e734ed Revert "build(deps): Bump pylint from 2.9.6 to 2.11.1 (#6636)" (#6842)
This reverts commit 464620806c.
2021-10-20 12:41:00 +03:00
464620806c build(deps): Bump pylint from 2.9.6 to 2.11.1 (#6636)
* build(deps): Bump pylint from 2.9.6 to 2.11.1

Bumps [pylint](https://github.com/PyCQA/pylint) from 2.9.6 to 2.11.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/main/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/v2.9.6...v2.11.1)

---
updated-dependencies:
- dependency-name: pylint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix pylint problems
* system: use binary mode

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: karajan1001 <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-10-20 10:32:44 +08:00
Paweł Redzyński 21af7438db plots/metrics/params/experiments show: stop throwing exceptions (#5984)
* params/metrics/plots/exp show: introduce onerror

* fixup

* plots/metrics/params: removing errors

* removing more errors

* removing errors

* some more removing

* new plots ux

* new ux for metrics

* new ux for params

* fixing tests

* fixups

* minor test fixes

* minor test fixes

* commands: plots/params: fix ui imports

* working on error handlng decorator, stopped at show checkpoints

* error_handler, further dvelopment

* plots just before moving loading to collect

* appending errors to results: initial passing

* fixup

* onerror: remove onerror fixture

* fixups

* plots/metrics/params/experiments: temporarily remove error warnings

* plots: extract parsing out of class method

* metrics/experiments: provide exception serializer for show-json

* params/plots/metrics: signal errors

* rebase fixups

* fix TODOs

* fixup

* skshetry's review refactor
2021-07-16 15:36:04 +02:00
Paweł RedzyńskiandJorge Orpinel c39f91ecf4 params: read from ParamsDependency if possible (#5550)
* params: read from ParamsDependency if possible

* params: introduce include_untracked flag

* params: include untracked upon targets

* experiments: add option to show tracked params

* fixing test

* params: rename tracked to deps

* params: roll back to default params.yaml behavior

* experiments: add option to show only used params on diff

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/params.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2021-04-12 17:08:02 +02:00
Paweł Redzyński 4d36bc4de8 params: introduce --targets (#5122)
* params: add targets option

* fixup

* remove unnecessary changes

* fix isort

* params: diff: add test for --targets

* tests: roll back metric/plots unification, introduce collect test

* metrics/plots: refactoring

* missing plots test added
2020-12-22 04:19:13 +02:00
Alexanderandaandrusenko e928034fac add py support for ParamsDependency (#4456)
* add py support for ParamsDependency

* add py support by ast

* add support modify py params

Co-authored-by: aandrusenko <[email protected]>
2020-09-24 17:25:44 +03:00
dtrifiroandDaniele Trifirò e2b1fb1f64 add toml support for ParamsDependency (#4258)
* add toml support for ParamsDependency

* add toml support to dvc show

Co-authored-by: Daniele Trifirò <[email protected]>
2020-07-24 19:09:04 +03:00
Ruslan Kuprieiev 9ee62ef8f0 dvc: rename "working tree" to "workspace" (#3914)
Fixes #3892
2020-05-31 17:39:04 +03:00
Saugat Pachhai e8f6cd7f5b run: raise error when command is not specified (#3850)
* run: raise error when command is not specified

* tests: adjust tests/func/plots/test_plots.py

* tests: adjust tests/func/plots/test_diff.py

* tests: add command to metrics/plots/params

* tests: fix test_remote.py

* tests: fix test_ls.py

* tests: fix test_repro* and test_run*

* tests: fix script in test_ls, was failing on Win
2020-05-27 12:13:26 +03:00
Ruslan Kuprieiev 93d7b9173b tests: use --single-stage 2020-05-04 20:23:49 +03:00
Saugat Pachhai 7edee84306 dvc: implement params support for pipeline file (#3694)
* dvc: implement params support for pipeline file

* reorganize dump/load of params

* add tests for params

* fix order of dictionary in Python 3.5
2020-04-29 14:35:26 +03:00
Ruslan Kuprieiev cdef709e8a params: fix bug when collecting multiple params (#3615) 2020-04-09 15:41:35 +03:00
Ruslan Kuprieiev c1ac6bb06b dvc: introduce dvc params diff (#3573)
* params: use repo.tree

* remote: local: use `repo.tree` for `exists()`

* param: handle missing params

* stage: move status warning to debug

* dvc: introduce `dvc params diff`

Fixes #3528
2020-04-09 05:18:49 +03:00