Commit Graph
100 Commits
Author SHA1 Message Date
Paweł Redzyński 8ca3a7b684 fs: repo: raise proper FileNotFoundError in metadata
Related: #7120
2021-12-17 15:48:29 +02:00
Paweł Redzyński e24de3a0f3 plots: introduce --json option
Fixes: #7073
2021-12-15 11:05:57 +01:00
Paweł Redzyński 7250e1b336 dvc: fix installation from archives
Fixes: #7124
2021-12-11 05:49:28 +02:00
Paweł Redzyński eb41b8b3e0 setup: bump gcsfs
Related: #7053
2021-12-09 15:20:22 +02:00
Paweł Redzyński 954ac6ab03 plots: roll back templates in packages change
Fixes: #6911
Reverts: a1b146669c
2021-12-06 09:55:13 +02:00
Paweł Redzyński 6723a34bda render: image: make html less ugly 2021-12-05 15:42:07 +02:00
Paweł Redzyński 57c6b1036f gha: allow builds for new 3.9 versions
Closes: #6975
2021-11-25 14:40:40 +02:00
Paweł Redzyński a9512e8585 plots: collect: return callable instead of data
Related to: #6923, iterative/viewer#2657
2021-11-22 21:31:20 +02:00
Paweł Redzyński e0fd0790fc plots: special chars handling for vega plots paths
Fixes: #6894
2021-11-17 01:57:35 +02:00
Paweł Redzyński ab56e4bb9b gha: limit python version to 3.9.7
Related: https://bugs.python.org/issue45235
Closes: #6956
2021-11-11 01:14:03 +02:00
Paweł Redzyński 36ed182f26 argparse: dont override cmd
Related: #6956, https://bugs.python.org/issue45235
2021-11-11 01:14:03 +02:00
Paweł Redzyński 22cc2a5cbc plots: fix resolve template path bug
Fix: #6854
2021-11-02 16:49:56 +02:00
Paweł Redzyński db75878bef plots: make generator out of collect (#6793) 2021-10-15 17:24:02 +03:00
Paweł Redzyński a1b146669c plots: load templates from package (#6550)
* plots: vega: make interactive linear default plot

* plots: stop dumping default templates

* plots: move templates to package initial

* plots: update default plots schema

* plots: write defaults on init

* templates: migrate to importlib.resources

* fixup

* fixup
2021-10-14 08:40:31 +02:00
Paweł RedzyńskiandJorge Orpinel 6da15629c4 plots: modify: add note on modyfying image type plots (#6742)
* plots: modify: add note on modyfying image type plots

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2021-10-12 16:08:49 +03:00
Paweł Redzyński d6ecef7169 plots: fix wrong link (#6693) 2021-10-11 12:22:20 +02:00
Paweł Redzyński 8580e66d60 gha: move cancelaltion into other workflows (#6771) 2021-10-08 18:03:07 +03:00
Paweł Redzyński dd86076ba6 gha: cancel previous workflows (#6756) 2021-10-07 17:59:44 +03:00
Paweł Redzyński 3e68cd7105 plots: use dpath to explore data (#6639) 2021-09-17 15:09:28 +02:00
Paweł Redzyński fa31fd1ef4 tests: requirements: remove beautifoulsoup4 (#6571) 2021-09-09 08:59:20 +00:00
Paweł Redzyński ab08db5065 plots: introduce support for images (#6431)
* plots: support images initial

* plots: data: unify PlotsData

* plots: data conversion refactor

* plots: renderers: abstract functionality

* plots: uncommenting tests

* plots: self implement grouping

* plots: rename group

* plots: extract rendering tests

* plots: move rendering tests to unit

* plots: add image rendering tests

* plots: dont use dpath to find vega plot

* plots: test find_vega

* plots: test render function

* plots: move rendering out of repo package

* plots: refactoring

* plots: self review refactor
2021-08-26 20:00:31 +03:00
Paweł Redzyński a7175501aa ruamel.yaml: adjust to new DuplicatedKeyError (#6464) 2021-08-19 20:02:16 +03:00
Paweł Redzyński 67b42ebe4f plots: cleanup data extraction (#6355)
* plots: cleanup data extraction

* fixup
2021-07-23 10:28:36 +09:00
Paweł Redzyński 21af7438db plots/metrics/params/experiments show: stop throwing exceptions (#5984)
* params/metrics/plots/exp show: introduce onerror

* fixup

* plots/metrics/params: removing errors

* removing more errors

* removing errors

* some more removing

* new plots ux

* new ux for metrics

* new ux for params

* fixing tests

* fixups

* minor test fixes

* minor test fixes

* commands: plots/params: fix ui imports

* working on error handlng decorator, stopped at show checkpoints

* error_handler, further dvelopment

* plots just before moving loading to collect

* appending errors to results: initial passing

* fixup

* onerror: remove onerror fixture

* fixups

* plots/metrics/params/experiments: temporarily remove error warnings

* plots: extract parsing out of class method

* metrics/experiments: provide exception serializer for show-json

* params/plots/metrics: signal errors

* rebase fixups

* fix TODOs

* fixup

* skshetry's review refactor
2021-07-16 15:36:04 +02:00
Paweł Redzyński d509d77193 list: add --show-json option (#5888)
* list: add --show-json option

* list: add test for --show-json
2021-04-30 11:40:44 +02:00
Paweł Redzyński d87ace0968 html: enable providing custom html page templates (#5694)
* html: enable providing custom html page templates

* plots: add html template configuration option
2021-04-20 11:02:14 +02:00
Paweł RedzyńskiandJorge Orpinel c39f91ecf4 params: read from ParamsDependency if possible (#5550)
* params: read from ParamsDependency if possible

* params: introduce include_untracked flag

* params: include untracked upon targets

* experiments: add option to show tracked params

* fixing test

* params: rename tracked to deps

* params: roll back to default params.yaml behavior

* experiments: add option to show only used params on diff

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/experiments.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/params.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2021-04-12 17:08:02 +02:00
Paweł Redzyński eccc320606 remote: use configured jobs number (#5747) 2021-04-08 17:52:42 +03:00
Paweł Redzyński 4a8cb806d2 live integration: get rid of dvc from live (#5466)
* live integration: get rid of dvc from live

* live: remove from api, add test for html generation during run

* stage: run: convert monitors to class context managers

* run: make monitors run in single thread

* run: move monitor logic out of run
2021-03-03 00:45:35 +09:00
Paweł Redzyński 92b95bae9b plots: ignore binary plot files (#5523) 2021-03-01 15:00:42 +02:00
Paweł Redzyński 63f32936b2 live integration: turn on tests (#5487) 2021-02-19 09:25:28 +01:00
Paweł Redzyński 14958ff85f plots: fix bug preventing usage from being displayed (#5492) 2021-02-18 14:24:47 +01:00
Paweł Redzyński a7da6c2a80 live: support cache and no_cache use case (#5394)
* live: add cache functionality

* live: add tests for cache use case
2021-02-07 03:51:18 +02:00
Paweł Redzyński 31e4a3325d live integration: rename report to html (#5366) 2021-02-02 13:44:38 +01:00
Paweł Redzyński 13db267a5d collect: unify message for no file in workspace (#5339) 2021-02-01 14:25:22 +01:00
Paweł Redzyński 6a9c0c5ce5 live/checkpoints: test continue (#5300)
* live: test resuming

* initial test, not working

* experiments: introduce DVCLIVE_RESUME env variable on resume
2021-01-24 22:06:37 +02:00
Paweł Redzyński 2277a1a655 metrics/plots/params: return ordered (#5297)
* metrics/plots/params: return ordered

* metrics/params/plots: add tests for return order
2021-01-22 20:57:43 +02:00
Paweł Redzyński 28f6beaf78 checkpoints/live: move execution env to stage (#5257)
* checkpoints/live: move execution env to stage

* stage: env: provide DVC_ROOT for all repo runs
2021-01-18 10:35:07 +02:00
Paweł RedzyńskiandSaugat Pachhai 2ee68b01b0 system: prevent mypy from checking windows-related method (#5260)
* system: prevent mypy from checking windows-related method

* Update dvc/system.py

Co-authored-by: Saugat Pachhai <[email protected]>

Co-authored-by: Saugat Pachhai <[email protected]>
2021-01-13 13:48:57 +02:00
Paweł Redzyński 386d0c463a live: add report switch (#5223) 2021-01-07 13:52:21 +02:00
Paweł RedzyńskiandRuslan Kuprieiev 862ab8de62 live logs initial (#4902)
* plots: allow dir plots

* first scenario

* separate dvclive tests

* rename index to step

* another test

* refactor

* its ok to have step in data

* sort plots

* introduce --logs

* add logs to run commands

* variable name refactoring

* fix logs option

* fixup

* cmd: run: fix unit tests

* plots: modify: fix tests

* improve dir handling

* fixup

* dvclive: logs summary

* api: introduce logs summarization method

* handle no repo case

* fix granular dir plots

* api: dvclive: simplify

* dvclive: roll back logs flag

* cmd: introduce logs

* api: adjust to logs

* test: dvclive fix

* rename logs to dvclive

* live: command: add uninitialized

* outs: dvclive type

* dvclive: support providing config via evn variables

* dvclive: treat as metrics

* dvclive integration: add support for summary on/off

* dvc integration: add temporary test for configuration export

* remove requirements

* cmd: introduce logs

* rename logs to dvclive

* cmd: introduce logs

* remove logs leftowvers

* dvclive: env: adjust to chekpoints env variables

* command: live: remove unnecessary help

* fix linter suggestions

* type hinting

* review refactor

* replace dvclive with live

* stage: loading: move live loading to separate method

* stage: serialize: allow only single live output

* live: add show/diff

* stage: serialize: roll back single live enforcing

* fixup

* rebase fixups

* output: live: force single instance

* live: command: add unit tests

* Update dvc/output/base.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/__init__.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/stage/serialize.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* review v2

* live: fix summary tracking by experiments

* naming fixups

* api: live: add tests

* tests: api: separate live tests

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-01-04 11:29:41 +02:00
Paweł Redzyński c28e671f47 tests: plots: fix plots call (#5183) 2020-12-30 22:59:32 +02:00
Paweł Redzyński 4d36bc4de8 params: introduce --targets (#5122)
* params: add targets option

* fixup

* remove unnecessary changes

* fix isort

* params: diff: add test for --targets

* tests: roll back metric/plots unification, introduce collect test

* metrics/plots: refactoring

* missing plots test added
2020-12-22 04:19:13 +02:00
Paweł Redzyński f6f18bc201 plots: make linear and scatter interactive (#4975) 2020-12-01 14:55:30 +01:00
Paweł Redzyński 7501302baa metrics/plots: common tests: check for particular exceptions (#4865) 2020-11-18 14:14:19 +02:00
Paweł Redzyński d3931def53 tests: real_s3 fixture: use specific path (#4884) 2020-11-12 16:36:08 +01:00
Paweł Redzyński f4ee0e16b4 plots/metrics: fix some communication disrepancies (#4837)
* plots/metrics: unified parsing errors communication

* plots/metrics: unify no metrics/plots messaging

* fixup

* fixup
2020-11-07 02:56:14 +02:00
Paweł Redzyński edd8344659 plots: fix plots path when calling from different dir (#4843) 2020-11-05 08:11:24 +01:00
Paweł Redzyński 9940540917 metrics/plots: test metrics collection on no target on revision (#4789)
* metrics: include revision information when collecting metrics files

* plots: test warning message on no plot on revision

* plots/metrics: move to one package, introduce common tests

* tests: fix utils placement
2020-10-30 02:00:09 +02:00
Paweł Redzyński 6cf194f85f plot: fix no metrics data detection (#4764) 2020-10-26 20:05:55 +02:00
Paweł Redzyński 5038fd225b plots/metrics: support show for no repo (#4761) 2020-10-26 20:04:49 +02:00
Paweł Redzyński 4d75e75631 test_update_import: roll back skip (#4781) 2020-10-26 15:28:30 +02:00
Paweł Redzyński 33ece011fb TestCmdCacheDir: roll back darwin skip (#4673) 2020-10-21 23:36:16 +03:00
Paweł Redzyński 2b90d7dfaf tests, packages: roll back environmental packages setup (#4728) 2020-10-15 14:19:25 +03:00
Paweł Redzyński 8f8c561d1b test_api: roll back xfail (#4660) 2020-10-15 10:40:53 +02:00
Paweł Redzyński 3e011d70d5 github actions: windows: fix packaging (#4711) 2020-10-13 21:31:48 +02:00
Paweł Redzyński 549f342db9 gha: fix github token providing for assets upload (#4703) 2020-10-12 18:18:20 +03:00
Paweł Redzyński 0da1664f3f gha: get rid of unnecessary if (#4702) 2020-10-12 13:33:15 +03:00
Paweł Redzyński fba94a4569 tests build: buil on push/pr to master (#4680) 2020-10-10 00:22:08 +03:00
Paweł Redzyński 09f9506ca0 github actions: migrate to envrionmental files (#4684) 2020-10-09 11:33:06 +03:00
Paweł Redzyński 9f90c806d5 params, plots, metrics: unify collection (#4603)
* params, plots, metrics: unify collection

* collect: split into smaller parts
2020-10-09 06:26:11 +03:00
Paweł Redzyński 548c2dcc1f test_str_workdir_outside_repo: roll back skip on windows (#4672) 2020-10-07 19:15:14 +03:00
Paweł Redzyński ab22118cd4 metrics, plots: update cmd targets help (#4620) 2020-09-30 04:55:00 +03:00
Paweł Redzyński e1b34442aa plot: accept any viable target in dvc repo (#4602) 2020-09-23 17:02:12 +03:00
Paweł Redzyński b31bd1cb30 github actions tests: initial (#4469)
* github actions tests: initial

* build: tests: add codecov action
2020-09-22 23:05:06 +03:00
Paweł Redzyński 2b3c30cb0c metrics: accept any viable target in DvcRepo (#4590) 2020-09-22 22:53:55 +03:00
Paweł Redzyński a82d6e4173 metrics: don't raise on dir during collection (#4443) 2020-08-24 02:55:20 +03:00
Paweł Redzyński ef944e6509 http: replace HEAD with streamed GET (#4390) 2020-08-18 12:45:06 +03:00
Paweł Redzyński 6ae0be2179 metrics: show: debug info on parsing problems (#4370) 2020-08-17 17:37:15 +03:00
Paweł Redzyński 35f125f3df outputs: validate that output is not ignored (#4386) 2020-08-14 14:30:12 +03:00
Paweł Redzyński 9c772f5f6c get-url: allow weak etags (#4372) 2020-08-13 12:03:47 +03:00
Paweł Redzyński 91f3036cf1 output: local: use repo tree if possible (#4125) 2020-07-16 23:03:47 +03:00
Paweł Redzyński e4dafb8552 RepoDependency: never changed (#4209) 2020-07-16 14:42:33 +03:00
Paweł RedzyńskiandJorge Orpinel 64c1321c9d un/freeze: adjust stage naming to 1.0 (#4102)
* un/freeze: adjust stage naming to 1.0

* Update dvc/command/freeze.py

* Update dvc/command/freeze.py

Co-authored-by: Jorge Orpinel <[email protected]>
2020-06-24 22:51:29 +03:00
Paweł Redzyński d2dbe9649c plots: add smooth template (#4046) 2020-06-15 22:15:27 +03:00
Paweł Redzyński 584c97a58d plots: fix handling of multiple plots from single revision (#4031) 2020-06-12 16:08:08 +03:00
Paweł Redzyński 56833387ce plots: find metric data in yaml dictionary (#3992) 2020-06-09 16:42:28 +03:00
Paweł Redzyński d74358eecd remote: local: protect uploaded files (#3858) 2020-05-22 22:59:48 +03:00
Paweł Redzyński f0e991790e plot: fixup for template help message (#3854) 2020-05-22 20:30:02 +03:00
61f30ed5ba plots: point template help info to manual (#3830)
* plots: point template help info to manual

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Jorge Orpinel <[email protected]>
2020-05-22 00:53:04 +03:00
Paweł Redzyński b52c925d5f lock: make froze message clickable (#3833) 2020-05-20 16:05:09 +03:00
Paweł Redzyński f44ddcf64e plot: dont zero y axis (#3813) 2020-05-19 16:20:21 +03:00
Paweł Redzyński d81b56978f plot: add doctype at the beggining of html embedding (#3787) 2020-05-13 00:17:05 +03:00
Paweł Redzyński 451c7aac04 tests: erepo_dir: extract remote setup to separate method (#3748) 2020-05-12 02:05:22 +03:00
Paweł Redzyński 9714b14456 plot: fix help messages (#3734) 2020-05-07 13:22:55 +03:00
Paweł Redzyński aa99cc9934 plot: check field exists (#3742) 2020-05-06 18:33:10 +03:00
Paweł Redzyński af8ac006ad plot: completion for zsh (#3733) 2020-05-04 15:01:29 +03:00
Paweł RedzyńskiandRuslan Kuprieiev e553511085 Metrics - plotting for multiple revisions initial (#3577)
* init

* rename to plot data insertion basig on dicts update

* revision support

* roll back revision

* plot makedirs for backward compatibility

* log path

* pretty plot link to visualization page

* make target default title

* efiop review

* efiop review

* plot multiple initial

* add some missing metric file tests

* proper id generation

* proper id generation

* add confusion matrix template

* refactor tests

* plot from dvct file

* plot from dvct

* brush up commands

* fix confusion matrix multiple plot

* plot: change confusion matrix data schema

* should be working as intended

* support for src file in dvct files

* minor fixes

* plot: support json templates

* plot: rename confusion template

* plot: polish command behaviour

* fix test for json

* plot: test command

* some minor fixes for tests

* plot: unit test loading

* plot: unit test loading

* plot: handle TODOS

* cleanup

* use mocker

* plot: support tsv

* plot: command refactoring

* plot: fix windows issues with tests

* plot: test: some more windows fixes

* plot: _load_from_revisions complexity fix

* plot: reduce complexity

* plot: complexity reduction

* plot: deepsource suggestions

* plot: move template path evaluation

* fixup

* fixup

* exception on no datafile and no template

* json metric load with OrderedDict

* plot: improve handling non-existing files on revisions

* plot: improve handling non-existing files on revisions

* change default plot path

* some exceptions and fixes

* add yaml metrics support

* fixup

* some more suggestions

* default filename fix

* efiop review requests

* log exception on failur

* move revisions deduction to commands

* json templates

* extract template filling to separate method

* some parsing improvements

* add columns functionality

* extract default data transformation to separate method

* plot: initial support for jsonpath

* plot: rename columns to filters, tests are dict based

* plot: fixups

* plot: refactoring

* repo: plot: convert to package

* plot: data loading refactor, support searching for data

* plot: raise if wrong fields provided

* plot: command description

* plot: default: pass y axis info for default plot

* plot: get rid of fieldnames, expect ordered data

* plot: handle default plot in separate method

* plot: fix default

* plot: command option names fixes

* refactoring

* fixes

* plot: provide option for stdout redirection

* plot: rename show-json to no-html

* plot: add no-csv-header option

* plot: improve error message for wrongly structured metric

* plot: match template name exactly, whit suffix appended only

* plot: dmpetrov and ivan review

* plot: refactor --stdout help message

* plot: move template to repo/plot

* plot: add -x and -y options

* plot: add -x and -y options

* plot: command: order change

* plot: scatter

* plot: rename confusion matrix template, new name generation format

* plot: add title anchor

* plot: review from jorgeorpinel

* plot: rename filter and result options to select and file

* plot: add --title, --x-title, --y-title

* plot: xlab ylab

* Update dvc/repo/plot/template.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plot/template.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* efiop review

* plot: bash completion

* plot: static code analysis fixes

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-05-01 22:45:03 +03:00
Paweł Redzyński eee3d7674f diff: analyse working tree when dirty (#3534) 2020-03-27 15:17:55 +02:00
Paweł Redzyński 87338c994f cmd: diff: don't load dir cache for diff (#3468) 2020-03-16 21:39:34 +02:00
Paweł Redzyński 767433d7a4 tests: add .gitignore files upon dvc_add (#3455) 2020-03-13 17:19:35 +02:00
Paweł Redzyński a9bc65ee1f analytics: fix resolving used scm (#3405) 2020-02-26 02:29:46 +02:00
Paweł Redzyński 8d04d6d3ce SCM: allow multiple DVC repos inside single SCM repo (#3257) 2020-02-25 14:40:20 +02:00
Paweł Redzyński bd33b1ae30 Stage: get checksum from repo if no checksum in stage file (#3365) 2020-02-21 12:10:33 +02:00
Paweł Redzyński 9b2ef002ea Stage: create: reset repo after new stage creation (#3349) 2020-02-19 20:25:04 +02:00
Paweł Redzyński 5b737a0a0b imp_url: make default stage fname accompany import target (#3312) 2020-02-14 17:05:08 +02:00
Paweł Redzyński 0c87771064 daemon config fix (#3324) 2020-02-14 13:13:04 +02:00
Paweł Redzyński ccca126749 add: meaningful message upon adding overlapping paths (#3296) 2020-02-13 05:31:57 +02:00
Paweł Redzyński 6557fed55d remote: verify cache exists on dir checksum 2020-01-24 13:37:33 +01:00
Paweł Redzyński 44b7216861 some doc link fixes 2020-01-24 11:57:17 +01:00