Commit Graph
38 Commits
Author SHA1 Message Date
Paweł Redzyński ab08db5065 plots: introduce support for images (#6431)
* plots: support images initial

* plots: data: unify PlotsData

* plots: data conversion refactor

* plots: renderers: abstract functionality

* plots: uncommenting tests

* plots: self implement grouping

* plots: rename group

* plots: extract rendering tests

* plots: move rendering tests to unit

* plots: add image rendering tests

* plots: dont use dpath to find vega plot

* plots: test find_vega

* plots: test render function

* plots: move rendering out of repo package

* plots: refactoring

* plots: self review refactor
2021-08-26 20:00:31 +03:00
Paweł Redzyński 67b42ebe4f plots: cleanup data extraction (#6355)
* plots: cleanup data extraction

* fixup
2021-07-23 10:28:36 +09:00
Paweł Redzyński 21af7438db plots/metrics/params/experiments show: stop throwing exceptions (#5984)
* params/metrics/plots/exp show: introduce onerror

* fixup

* plots/metrics/params: removing errors

* removing more errors

* removing errors

* some more removing

* new plots ux

* new ux for metrics

* new ux for params

* fixing tests

* fixups

* minor test fixes

* minor test fixes

* commands: plots/params: fix ui imports

* working on error handlng decorator, stopped at show checkpoints

* error_handler, further dvelopment

* plots just before moving loading to collect

* appending errors to results: initial passing

* fixup

* onerror: remove onerror fixture

* fixups

* plots/metrics/params/experiments: temporarily remove error warnings

* plots: extract parsing out of class method

* metrics/experiments: provide exception serializer for show-json

* params/plots/metrics: signal errors

* rebase fixups

* fix TODOs

* fixup

* skshetry's review refactor
2021-07-16 15:36:04 +02:00
Christian ClaussandSaugat Pachhai 40f24e5668 pre-commit: introduce codespell check
* codespell --ignore-words-list="ba,datas,fo"

* up-to-date -> uptodate

* Add pre-commit hook...

* Pre-commit needs quotes?

* Single quotes?

* Arrrgggsss

* Update .pre-commit-config.yaml

Co-authored-by: Saugat Pachhai <[email protected]>

Co-authored-by: Saugat Pachhai <[email protected]>
2021-07-07 17:14:15 +02:00
Saugat Pachhai 476af100de ui: migrating rest to the ui.write (#6176)
* complete migration to ui.write

* fix small bugs
2021-06-21 22:14:59 +03:00
Saugat Pachhai b9000ed926 pre-commit: update black to 21.5b1 (#6032)
* pre-commit: update black to 21.5b1

also updated others, but black is more noticeable

* remove magic trailing comma
2021-05-18 15:51:15 +00:00
Ruslan Kuprieiev 0ad1408da7 plots: handle NoRemoteError for dirs (#5976) 2021-05-07 14:35:35 +03:00
Saugat Pachhai 76ece8c64a plots: quote and resolve paths properly (#5670)
This commit fixes 3 things:
1. The path printed on the terminal is properly is quoted.  This was
problematic when the output is piped to `xdg-open` if the path contained
spaces or quotes or special characters.

2. The path is resolved. This is not an issue at all, but makes the
output nicer.

3. The path with which the browser is opened with uses relative path.
This is done to solve issue in WSL, that the absolute path is not
accessible outside the WSL for host programs as the linux filesystem
is available on `/wsl$` path.
2021-03-24 20:27:24 +05:45
Paweł Redzyński 92b95bae9b plots: ignore binary plot files (#5523) 2021-03-01 15:00:42 +02:00
Ruslan Kuprieiev e0842ce01e dvc: rename cache -> odb (#5481) 2021-02-16 22:49:25 +02:00
Paweł RedzyńskiandRuslan Kuprieiev 862ab8de62 live logs initial (#4902)
* plots: allow dir plots

* first scenario

* separate dvclive tests

* rename index to step

* another test

* refactor

* its ok to have step in data

* sort plots

* introduce --logs

* add logs to run commands

* variable name refactoring

* fix logs option

* fixup

* cmd: run: fix unit tests

* plots: modify: fix tests

* improve dir handling

* fixup

* dvclive: logs summary

* api: introduce logs summarization method

* handle no repo case

* fix granular dir plots

* api: dvclive: simplify

* dvclive: roll back logs flag

* cmd: introduce logs

* api: adjust to logs

* test: dvclive fix

* rename logs to dvclive

* live: command: add uninitialized

* outs: dvclive type

* dvclive: support providing config via evn variables

* dvclive: treat as metrics

* dvclive integration: add support for summary on/off

* dvc integration: add temporary test for configuration export

* remove requirements

* cmd: introduce logs

* rename logs to dvclive

* cmd: introduce logs

* remove logs leftowvers

* dvclive: env: adjust to chekpoints env variables

* command: live: remove unnecessary help

* fix linter suggestions

* type hinting

* review refactor

* replace dvclive with live

* stage: loading: move live loading to separate method

* stage: serialize: allow only single live output

* live: add show/diff

* stage: serialize: roll back single live enforcing

* fixup

* rebase fixups

* output: live: force single instance

* live: command: add unit tests

* Update dvc/output/base.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/serialize.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* review v2

* live: fix summary tracking by experiments

* naming fixups

* api: live: add tests

* tests: api: separate live tests

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-01-04 11:29:41 +02:00
Paweł Redzyński c28e671f47 tests: plots: fix plots call (#5183) 2020-12-30 22:59:32 +02:00
Paweł Redzyński 4d36bc4de8 params: introduce --targets (#5122)
* params: add targets option

* fixup

* remove unnecessary changes

* fix isort

* params: diff: add test for --targets

* tests: roll back metric/plots unification, introduce collect test

* metrics/plots: refactoring

* missing plots test added
2020-12-22 04:19:13 +02:00
Paweł Redzyński 9940540917 metrics/plots: test metrics collection on no target on revision (#4789)
* metrics: include revision information when collecting metrics files

* plots: test warning message on no plot on revision

* plots/metrics: move to one package, introduce common tests

* tests: fix utils placement
2020-10-30 02:00:09 +02:00
Paweł Redzyński 5038fd225b plots/metrics: support show for no repo (#4761) 2020-10-26 20:04:49 +02:00
Paweł Redzyński 9f90c806d5 params, plots, metrics: unify collection (#4603)
* params, plots, metrics: unify collection

* collect: split into smaller parts
2020-10-09 06:26:11 +03:00
Ruslan Kuprieiev 775e1e8374 repo: don't create .dvc in read-only repos (#4674)
* lock: use ABC

* lock: add LockNoop

* plots: remove unused code

* plots: create templates only on init

* dvc: don't create .dvc in read-only repos

* repo: explicitly init LockNoop
2020-10-07 23:51:14 +03:00
Ruslan Kuprieiev de24794cd5 repo: move templates to plots (#4629) 2020-09-29 00:38:11 +03:00
Ruslan Kuprieiev 1ea8cc9e54 metrics/plots: support non-dvc repos (#4627) 2020-09-28 23:06:22 +03:00
Paweł Redzyński e1b34442aa plot: accept any viable target in dvc repo (#4602) 2020-09-23 17:02:12 +03:00
Alexander Schepanovski e1f43965a2 Skip no data plots while rendering (#4541) 2020-09-07 23:25:43 +03:00
Saugat Pachhai 14745039a4 Move to yaml1.2 (#4415)
* use yaml 1.2 everywhere

This commit:
* gets rid of pyyaml
* uses ruamel everywhere
* tries to unite toml and yaml

However, this will require a few more changes that will come soonish.
Eg: Current `parse_yaml` and `parse_toml` behavior could be error-prone,
so, it's better to introduce `tree` inside these serialization methods
(which I will try to tackle next).

Another is, yaml load can be slow for json. We could speed it up just by
using `json.load(s)` here (also in upcoming patch).

* Fix tests
2020-08-19 17:56:38 +03:00
Saugat Pachhai ad306d7bf3 Enable most of the pylint checks (#4111) 2020-06-25 18:10:14 +03:00
Ruslan Kuprieiev b672622324 plots: make sure template is using data anchor (#4045)
Fixes #3969
2020-06-15 21:14:07 +03:00
Ruslan Kuprieiev c9391df313 plots: modify: handle --unset for nonexistent options (#4044)
* tests: plots: reorginize the structure

* plots: modify: handle --unset for nonexistent options

Fixes #3933
2020-06-15 20:31:08 +03:00
Paweł Redzyński 584c97a58d plots: fix handling of multiple plots from single revision (#4031) 2020-06-12 16:08:08 +03:00
Ruslan Kuprieiev 87276faef2 RepoTree: work with dirty repos (#4005)
Currently `RepoTree.open` will throw FileNotFoundError if you have a
file that is tracked by dvc, but which has no md5 or a cache for which
is missing, even though you have the file itself in your workspace.

This corresponds very nicely to the `--no-commit` or `--no-exec`
scenario, where you might not want to commit your artifacts to cache
just yet.

Fixes #3974
2020-06-11 09:50:27 +03:00
Saugat Pachhai 1d6c80001d plots modify: complain if template does not exist (#3995) 2020-06-10 16:12:56 +05:45
Ruslan Kuprieiev afe539aaba plots: rename --no-csv-header to --no-header (#4001)
* plots: rename --no-csv-header to --no-header

Fixes #3996

* fix typo
2020-06-10 03:34:25 +03:00
Alexander Schepanovski 7230d87137 Refactor plots (#3994)
- separate data collection and rendering
- provide a way to use custom template loader
- drop hard-coding datafiles in template feature
2020-06-09 22:47:17 +03:00
Paweł Redzyński 56833387ce plots: find metric data in yaml dictionary (#3992) 2020-06-09 16:42:28 +03:00
Alexander Schepanovski 9a55b909f8 Rename --xlab to --x-label, X_TITLE to X_LABEL, same for y (#3948)
Renamed:
- cli options
- dvcfile plot props
- template anchors
- variable names throughout the code
2020-06-03 20:30:17 +03:00
Ruslan Kuprieiev ff54ac1bfb plots: fix error when there are no plots in repo (#3940)
Consistent with `dvc metrics show/diff` behaviour.

Fixes #3925
2020-06-03 08:54:48 +03:00
Ruslan Kuprieiev 9ee62ef8f0 dvc: rename "working tree" to "workspace" (#3914)
Fixes #3892
2020-05-31 17:39:04 +03:00
Alexander SchepanovskiandRuslan Kuprieiev 516d82e411 Add plots props and plots modify params (#3911)
* Add plots props and `plots modify` params

* Update dvc/command/plots.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plots/show.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-05-30 14:37:11 +03:00
Saugat Pachhai e8f6cd7f5b run: raise error when command is not specified (#3850)
* run: raise error when command is not specified

* tests: adjust tests/func/plots/test_plots.py

* tests: adjust tests/func/plots/test_diff.py

* tests: add command to metrics/plots/params

* tests: fix test_remote.py

* tests: fix test_ls.py

* tests: fix test_repro* and test_run*

* tests: fix script in test_ls, was failing on Win
2020-05-27 12:13:26 +03:00
Ruslan Kuprieiev 09136f2378 plots: add plot markers to DVC files (#3807)
* run: add --plot/--plot-no-cache

Part of #3409

* plot: move show/diff to API

* plots: show marked plots by default

Involves some refactoring to make plots comply with the rest of the code
base.

* dvc.yaml: use top-level metrics and plots

* dvc.yaml: add support for plot template

* plots: add support for templates in dvc.yaml

* dvc.yaml: allow metrics/plots to have persist/cache

* plots: render single page by-default and introduce --show-json

* plot: use "working tree" instead of "workspace"

Just to sync this with the rest of the codebase. We should consider
renaming it to "workspace" everywhere, but for now I opt to not touch
it to not break something that depends on it.

* plots: rename -f|--file to -o|--out

Better corresponds to other commands like `import`.

* plots: use RepoTree and brancher instead of api

* stage: get rid of OutputParams

* tests: plots: add simple diff test

* plots: use `--targets` instead of `--datafile`

Makes it comply with `dvc metrics/params` CLI options.

* tests: unit: plots: add --show-json test

* stage: cleanup exception for format error

Based on feedback from the users.

* lockfile: use relpath in corruption error
2020-05-20 04:48:53 +03:00
Ruslan Kuprieiev cec4b4e003 dvc: rename plot to plots (#3802)
* dvc: rename plot to plots

Follows `dvc params/metrics` convention and is needed in preparation for new
commands and some refactoring.

Part of #3409

* completion: rename plot -> plots
2020-05-14 22:28:22 +03:00