Files
dvc/dvc/testing
pre-commit-ci[bot]andSaugat Pachhai 3783d65dde [pre-commit.ci] pre-commit autoupdate (#9841)
* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/astral-sh/ruff-pre-commit: v0.0.282 → v0.0.285](https://github.com/astral-sh/ruff-pre-commit/compare/v0.0.282...v0.0.285)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix ruff issues

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
2023-08-22 15:22:57 +05:45
..
2021-12-02 09:08:40 +02:00
2023-05-03 14:58:54 +09:00
2023-02-08 23:05:45 +02:00

DVC pytest plugin

dvc.testing.benchmarks
======================

Benchmark test definitions are now part of ``dvc.testing``.
For CLI usage and `bench.dvc.org <https://bench.dvc.org>`_ details see `dvc-bench <https://github.com/iterative/dvc-bench>`_.

``dvc.testing.benchmarks`` structure:

* cli: should be able to run these with any dvc (rpm, deb, pypi, snap, etc) (could be used in dvc-test repo too)

  * commands: granular tests for individual commands. These should have a cached setup, so that we could use them during rapid development instead of our hand-written scripts. Every test could be run in a separate machine.
  * stories: multistage start-to-end benchmarks, useful for testing workflows (e.g. in documentation, see test_sharing inspired by `Storing and sharing <https://dvc.org/doc/start/data-management/data-versioning#storing-and-sharing>`_. Every full story could be run in a separate machine.

* api: for python api only.

  * methods: granular tests for individual methods (e.g. ``api.open/read``). Same reasoning as in ``cli.commands``
  * stories: same as ``cli.stories`` but for our api. E.g. imagine using our api with pandas or smth like that.