Commit Graph
28 Commits
Author SHA1 Message Date
Paweł RedzyńskiandRuslan Kuprieiev 862ab8de62 live logs initial (#4902)
* plots: allow dir plots

* first scenario

* separate dvclive tests

* rename index to step

* another test

* refactor

* its ok to have step in data

* sort plots

* introduce --logs

* add logs to run commands

* variable name refactoring

* fix logs option

* fixup

* cmd: run: fix unit tests

* plots: modify: fix tests

* improve dir handling

* fixup

* dvclive: logs summary

* api: introduce logs summarization method

* handle no repo case

* fix granular dir plots

* api: dvclive: simplify

* dvclive: roll back logs flag

* cmd: introduce logs

* api: adjust to logs

* test: dvclive fix

* rename logs to dvclive

* live: command: add uninitialized

* outs: dvclive type

* dvclive: support providing config via evn variables

* dvclive: treat as metrics

* dvclive integration: add support for summary on/off

* dvc integration: add temporary test for configuration export

* remove requirements

* cmd: introduce logs

* rename logs to dvclive

* cmd: introduce logs

* remove logs leftowvers

* dvclive: env: adjust to chekpoints env variables

* command: live: remove unnecessary help

* fix linter suggestions

* type hinting

* review refactor

* replace dvclive with live

* stage: loading: move live loading to separate method

* stage: serialize: allow only single live output

* live: add show/diff

* stage: serialize: roll back single live enforcing

* fixup

* rebase fixups

* output: live: force single instance

* live: command: add unit tests

* Update dvc/output/base.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/serialize.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* review v2

* live: fix summary tracking by experiments

* naming fixups

* api: live: add tests

* tests: api: separate live tests

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-01-04 11:29:41 +02:00
Paweł Redzyński c28e671f47 tests: plots: fix plots call (#5183) 2020-12-30 22:59:32 +02:00
Paweł Redzyński 4d36bc4de8 params: introduce --targets (#5122)
* params: add targets option

* fixup

* remove unnecessary changes

* fix isort

* params: diff: add test for --targets

* tests: roll back metric/plots unification, introduce collect test

* metrics/plots: refactoring

* missing plots test added
2020-12-22 04:19:13 +02:00
Paweł Redzyński 9940540917 metrics/plots: test metrics collection on no target on revision (#4789)
* metrics: include revision information when collecting metrics files

* plots: test warning message on no plot on revision

* plots/metrics: move to one package, introduce common tests

* tests: fix utils placement
2020-10-30 02:00:09 +02:00
Paweł Redzyński 5038fd225b plots/metrics: support show for no repo (#4761) 2020-10-26 20:04:49 +02:00
Paweł Redzyński 9f90c806d5 params, plots, metrics: unify collection (#4603)
* params, plots, metrics: unify collection

* collect: split into smaller parts
2020-10-09 06:26:11 +03:00
Ruslan Kuprieiev 775e1e8374 repo: don't create .dvc in read-only repos (#4674)
* lock: use ABC

* lock: add LockNoop

* plots: remove unused code

* plots: create templates only on init

* dvc: don't create .dvc in read-only repos

* repo: explicitly init LockNoop
2020-10-07 23:51:14 +03:00
Ruslan Kuprieiev de24794cd5 repo: move templates to plots (#4629) 2020-09-29 00:38:11 +03:00
Ruslan Kuprieiev 1ea8cc9e54 metrics/plots: support non-dvc repos (#4627) 2020-09-28 23:06:22 +03:00
Paweł Redzyński e1b34442aa plot: accept any viable target in dvc repo (#4602) 2020-09-23 17:02:12 +03:00
Alexander Schepanovski e1f43965a2 Skip no data plots while rendering (#4541) 2020-09-07 23:25:43 +03:00
Saugat Pachhai 14745039a4 Move to yaml1.2 (#4415)
* use yaml 1.2 everywhere

This commit:
* gets rid of pyyaml
* uses ruamel everywhere
* tries to unite toml and yaml

However, this will require a few more changes that will come soonish.
Eg: Current `parse_yaml` and `parse_toml` behavior could be error-prone,
so, it's better to introduce `tree` inside these serialization methods
(which I will try to tackle next).

Another is, yaml load can be slow for json. We could speed it up just by
using `json.load(s)` here (also in upcoming patch).

* Fix tests
2020-08-19 17:56:38 +03:00
Saugat Pachhai ad306d7bf3 Enable most of the pylint checks (#4111) 2020-06-25 18:10:14 +03:00
Ruslan Kuprieiev b672622324 plots: make sure template is using data anchor (#4045)
Fixes #3969
2020-06-15 21:14:07 +03:00
Ruslan Kuprieiev c9391df313 plots: modify: handle --unset for nonexistent options (#4044)
* tests: plots: reorginize the structure

* plots: modify: handle --unset for nonexistent options

Fixes #3933
2020-06-15 20:31:08 +03:00
Paweł Redzyński 584c97a58d plots: fix handling of multiple plots from single revision (#4031) 2020-06-12 16:08:08 +03:00
Ruslan Kuprieiev 87276faef2 RepoTree: work with dirty repos (#4005)
Currently `RepoTree.open` will throw FileNotFoundError if you have a
file that is tracked by dvc, but which has no md5 or a cache for which
is missing, even though you have the file itself in your workspace.

This corresponds very nicely to the `--no-commit` or `--no-exec`
scenario, where you might not want to commit your artifacts to cache
just yet.

Fixes #3974
2020-06-11 09:50:27 +03:00
Saugat Pachhai 1d6c80001d plots modify: complain if template does not exist (#3995) 2020-06-10 16:12:56 +05:45
Ruslan Kuprieiev afe539aaba plots: rename --no-csv-header to --no-header (#4001)
* plots: rename --no-csv-header to --no-header

Fixes #3996

* fix typo
2020-06-10 03:34:25 +03:00
Alexander Schepanovski 7230d87137 Refactor plots (#3994)
- separate data collection and rendering
- provide a way to use custom template loader
- drop hard-coding datafiles in template feature
2020-06-09 22:47:17 +03:00
Paweł Redzyński 56833387ce plots: find metric data in yaml dictionary (#3992) 2020-06-09 16:42:28 +03:00
Alexander Schepanovski 9a55b909f8 Rename --xlab to --x-label, X_TITLE to X_LABEL, same for y (#3948)
Renamed:
- cli options
- dvcfile plot props
- template anchors
- variable names throughout the code
2020-06-03 20:30:17 +03:00
Ruslan Kuprieiev ff54ac1bfb plots: fix error when there are no plots in repo (#3940)
Consistent with `dvc metrics show/diff` behaviour.

Fixes #3925
2020-06-03 08:54:48 +03:00
Ruslan Kuprieiev 9ee62ef8f0 dvc: rename "working tree" to "workspace" (#3914)
Fixes #3892
2020-05-31 17:39:04 +03:00
Alexander SchepanovskiandRuslan Kuprieiev 516d82e411 Add plots props and plots modify params (#3911)
* Add plots props and `plots modify` params

* Update dvc/command/plots.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-05-30 14:37:11 +03:00
Saugat Pachhai e8f6cd7f5b run: raise error when command is not specified (#3850)
* run: raise error when command is not specified

* tests: adjust tests/func/plots/test_plots.py

* tests: adjust tests/func/plots/test_diff.py

* tests: add command to metrics/plots/params

* tests: fix test_remote.py

* tests: fix test_ls.py

* tests: fix test_repro* and test_run*

* tests: fix script in test_ls, was failing on Win
2020-05-27 12:13:26 +03:00
Ruslan Kuprieiev 09136f2378 plots: add plot markers to DVC files (#3807)
* run: add --plot/--plot-no-cache

Part of #3409

* plot: move show/diff to API

* plots: show marked plots by default

Involves some refactoring to make plots comply with the rest of the code
base.

* dvc.yaml: use top-level metrics and plots

* dvc.yaml: add support for plot template

* plots: add support for templates in dvc.yaml

* dvc.yaml: allow metrics/plots to have persist/cache

* plots: render single page by-default and introduce --show-json

* plot: use "working tree" instead of "workspace"

Just to sync this with the rest of the codebase. We should consider
renaming it to "workspace" everywhere, but for now I opt to not touch
it to not break something that depends on it.

* plots: rename -f|--file to -o|--out

Better corresponds to other commands like `import`.

* plots: use RepoTree and brancher instead of api

* stage: get rid of OutputParams

* tests: plots: add simple diff test

* plots: use `--targets` instead of `--datafile`

Makes it comply with `dvc metrics/params` CLI options.

* tests: unit: plots: add --show-json test

* stage: cleanup exception for format error

Based on feedback from the users.

* lockfile: use relpath in corruption error
2020-05-20 04:48:53 +03:00
Ruslan Kuprieiev cec4b4e003 dvc: rename plot to plots (#3802)
* dvc: rename plot to plots

Follows `dvc params/metrics` convention and is needed in preparation for new
commands and some refactoring.

Part of #3409

* completion: rename plot -> plots
2020-05-14 22:28:22 +03:00