Files
dvc/tests/func/api/test_live.py
T
Paweł RedzyńskiandRuslan Kuprieiev 862ab8de62 live logs initial (#4902)
* plots: allow dir plots

* first scenario

* separate dvclive tests

* rename index to step

* another test

* refactor

* its ok to have step in data

* sort plots

* introduce --logs

* add logs to run commands

* variable name refactoring

* fix logs option

* fixup

* cmd: run: fix unit tests

* plots: modify: fix tests

* improve dir handling

* fixup

* dvclive: logs summary

* api: introduce logs summarization method

* handle no repo case

* fix granular dir plots

* api: dvclive: simplify

* dvclive: roll back logs flag

* cmd: introduce logs

* api: adjust to logs

* test: dvclive fix

* rename logs to dvclive

* live: command: add uninitialized

* outs: dvclive type

* dvclive: support providing config via evn variables

* dvclive: treat as metrics

* dvclive integration: add support for summary on/off

* dvc integration: add temporary test for configuration export

* remove requirements

* cmd: introduce logs

* rename logs to dvclive

* cmd: introduce logs

* remove logs leftowvers

* dvclive: env: adjust to chekpoints env variables

* command: live: remove unnecessary help

* fix linter suggestions

* type hinting

* review refactor

* replace dvclive with live

* stage: loading: move live loading to separate method

* stage: serialize: allow only single live output

* live: add show/diff

* stage: serialize: roll back single live enforcing

* fixup

* rebase fixups

* output: live: force single instance

* live: command: add unit tests

* Update dvc/output/base.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/serialize.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* review v2

* live: fix summary tracking by experiments

* naming fixups

* api: live: add tests

* tests: api: separate live tests

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-01-04 11:29:41 +02:00

56 lines
1.3 KiB
Python

import io
import json
import logging
from typing import Dict, List
import pytest
from funcy import last
from dvc.api.live import summary
from tests.utils import dump_sv
def _dumps_tsv(metrics: List[Dict]):
stream = io.StringIO()
dump_sv(stream, metrics)
stream.seek(0)
return stream.read()
@pytest.fixture
def live_results(tmp_dir):
def make(path="logs"):
datapoints = [{"metric": 0.0, "step": 0}, {"metric": 0.5, "step": 1}]
tmp_dir.gen(
{
(tmp_dir / path).with_suffix(".json"): json.dumps(
last(datapoints)
),
(tmp_dir / path / "metric.tsv"): _dumps_tsv(datapoints),
}
)
yield make
def test_live_summary_no_repo(tmp_dir, live_results, caplog):
live_results("logs")
with caplog.at_level(logging.INFO, logger="dvc"):
summary("logs")
summary_path = tmp_dir / "logs.html"
assert summary_path.exists()
assert f"file://{str(summary_path)}" in caplog.text
def test_live_summary(tmp_dir, dvc, live_results, caplog):
live_results("logs")
with caplog.at_level(logging.INFO, logger="dvc"):
summary("logs")
summary_path = tmp_dir / "logs.html"
assert summary_path.exists()
assert f"file://{str(summary_path)}" in caplog.text