Commit Graph
66 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 68a0c39ad1 run: remove deprecated -y|--yes (#3737) 2020-05-04 17:51:07 +03:00
Ruslan Kuprieiev 505b88afae diff: make output consistent with other diff commands (#3727)
* diff: make output consistent with other diff commands

Fixes #3435

* tests: plot: fix flaky test
2020-05-04 00:25:55 +03:00
nik123 415a85c6be status: --recursive flag (#3726)
* status: --recursive flag

* tests: fixed test_status_recursive for Windows platform
2020-05-03 21:12:17 +03:00
Paweł RedzyńskiandRuslan Kuprieiev e553511085 Metrics - plotting for multiple revisions initial (#3577)
* init

* rename to plot data insertion basig on dicts update

* revision support

* roll back revision

* plot makedirs for backward compatibility

* log path

* pretty plot link to visualization page

* make target default title

* efiop review

* efiop review

* plot multiple initial

* add some missing metric file tests

* proper id generation

* proper id generation

* add confusion matrix template

* refactor tests

* plot from dvct file

* plot from dvct

* brush up commands

* fix confusion matrix multiple plot

* plot: change confusion matrix data schema

* should be working as intended

* support for src file in dvct files

* minor fixes

* plot: support json templates

* plot: rename confusion template

* plot: polish command behaviour

* fix test for json

* plot: test command

* some minor fixes for tests

* plot: unit test loading

* plot: unit test loading

* plot: handle TODOS

* cleanup

* use mocker

* plot: support tsv

* plot: command refactoring

* plot: fix windows issues with tests

* plot: test: some more windows fixes

* plot: _load_from_revisions complexity fix

* plot: reduce complexity

* plot: complexity reduction

* plot: deepsource suggestions

* plot: move template path evaluation

* fixup

* fixup

* exception on no datafile and no template

* json metric load with OrderedDict

* plot: improve handling non-existing files on revisions

* plot: improve handling non-existing files on revisions

* change default plot path

* some exceptions and fixes

* add yaml metrics support

* fixup

* some more suggestions

* default filename fix

* efiop review requests

* log exception on failur

* move revisions deduction to commands

* json templates

* extract template filling to separate method

* some parsing improvements

* add columns functionality

* extract default data transformation to separate method

* plot: initial support for jsonpath

* plot: rename columns to filters, tests are dict based

* plot: fixups

* plot: refactoring

* repo: plot: convert to package

* plot: data loading refactor, support searching for data

* plot: raise if wrong fields provided

* plot: command description

* plot: default: pass y axis info for default plot

* plot: get rid of fieldnames, expect ordered data

* plot: handle default plot in separate method

* plot: fix default

* plot: command option names fixes

* refactoring

* fixes

* plot: provide option for stdout redirection

* plot: rename show-json to no-html

* plot: add no-csv-header option

* plot: improve error message for wrongly structured metric

* plot: match template name exactly, whit suffix appended only

* plot: dmpetrov and ivan review

* plot: refactor --stdout help message

* plot: move template to repo/plot

* plot: add -x and -y options

* plot: add -x and -y options

* plot: command: order change

* plot: scatter

* plot: rename confusion matrix template, new name generation format

* plot: add title anchor

* plot: review from jorgeorpinel

* plot: rename filter and result options to select and file

* plot: add --title, --x-title, --y-title

* plot: xlab ylab

* Update dvc/repo/plot/template.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plot/template.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* efiop review

* plot: bash completion

* plot: static code analysis fixes

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-05-01 22:45:03 +03:00
Ruslan Kuprieiev 45944ddde7 metrics: rework metrics implementation (#3658)
Dropping support for all types except json and yaml. No more xpath
support.

Fixes #3572
2020-04-22 13:39:59 +03:00
Saugat Pachhai 128fa7f887 dvc: add a support for multistage dvcfile. (#3584)
Now, pipeline stage files can house multiple stages, and separate lockfiles
are created which has the checksums, whereas Dvcfile will be clean and human
readable and editable. The *.dvc files will be generated for output files.

It is available via a hidden flag: -n.

Fixes #1871
PR: #3584
2020-04-17 22:58:36 +03:00
Ruslan Kuprieiev 8455b7bece params: diff: introduce --all (#3631)
* params: diff: introduce --all

Makes `dvc params diff` also show unchanged params.

* params: diff: show all present parameters
2020-04-15 13:21:24 +03:00
Saugat PachhaiandRuslan Kuprieiev 65854c4946 refactor: extract stage's relationship with Dvcfile out from Stage (#3619)
* refactor stage into dvcfile

* Split create_stage out of dvcfile to Stage::create() and overwrite

* Remove logging from Repo::run

* tests: stop using --remove-outs

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-04-12 23:09:29 +03:00
Ruslan Kuprieiev 63fe2893a8 run: remove deprecated --cwd (#3623)
It has been deprecated for a long time now and it significantly
complicates the code, hiding multiple bugs.
2020-04-11 19:57:05 +03:00
Ruslan Kuprieiev 0e854f9534 params: diff: sort table (#3617) 2020-04-09 21:23:18 +03:00
Ruslan Kuprieiev c1ac6bb06b dvc: introduce dvc params diff (#3573)
* params: use repo.tree

* remote: local: use `repo.tree` for `exists()`

* param: handle missing params

* stage: move status warning to debug

* dvc: introduce `dvc params diff`

Fixes #3528
2020-04-09 05:18:49 +03:00
Ruslan Kuprieiev 26980ae45b metrics: diff: fix float precision formatting (#3609)
Texttable will try to detect the column type automatically, which
results in a low precision of the float that it is showing. We need to
disable that auto-formatting and just use `str(val)` instead.
2020-04-07 19:42:25 +03:00
Ruslan Kuprieiev 7bd473371c push/pull/status/metrics: add support for --all-commits (#3587)
First steps for #1691
2020-04-04 19:30:11 +03:00
Ruslan Kuprieiev 8e675a9d42 metrics: diff: don't print "No changes" (#3576)
Makes it easier to use in shell scripts. E.g. `-z $(dvc metrics diff)`, which
has the same semantics as `git diff`. Also syncs up with the behaviour
of `dvc diff`.

Fixes #3469
2020-04-02 04:54:40 +03:00
Ruslan Kuprieiev 0bf6fdae14 metrics: diff: handle deleted/new metric (#3529)
Partial fix for https://github.com/iterative/dvc/issues/3469
2020-03-25 05:23:21 +02:00
Helge Munk JacobsenandRuslan Kuprieiev 28748ff345 introduce hyper parameters and config (#3515)
* Work in progress

* added file parsing and name validation + adjust schema

* Exceptions on bad input

* Support multiple parameters

* Support multi 's in
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

* fix formatting

* dep: param: fix schema

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* run: check if --params is specified

* dvc: rework param handling

* params: fix exc formatting

* params: support basic nested configs

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-03-24 17:56:39 +02:00
Saugat Pachhai 1455e9551e pull/checkout: fix empty output if no state gets changed (#3520) 2020-03-24 13:48:06 +02:00
Saugat PachhaiandJorge Orpinel 066ccea733 checkout: show summary by default and introduce --show-changes flag (#3401)
* checkout: implement --show-changes flag to show detailed checkout details
Show summary by default on checkout

checkout: show directory modified even when files are removed from inside it

checkout: refactor template logic

checkout: change output format

checkout: do not show summary if nothing's changed

checkout: do not show summary if show_changes is specified

checkout: show '/' at the end if it's a directory

rename variable name

test: cover stats for checkout

checkout: show relink as not modified

checkout: remove unwanted message during empty checkout

pull: show summary on checkout

import/get: change external_repo to catch CheckoutError

fix tests and check if path is url

checkout: support external deps/outputs

tests: checkout on external outputs

* fix tests

* checkout: make show changes default and introduce summary flag to show summary

* address suggestions

* exceptions: remove extra spacing on erorr

* Update dvc/command/checkout.py

Co-Authored-By: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2020-03-16 16:29:36 +02:00
Saugat Pachhai d2734dfe1e tests: remove use of mocker as context-manager (#3480) 2020-03-16 08:25:22 +05:45
Guro Bokum 02cd520acd api: colorize dvc list (#3351)
Close #3347
2020-03-09 14:07:23 +02:00
Ivan Shcheklein eebbf3f812 change diff checksum arg name to hash (#3384) 2020-02-23 06:32:35 +02:00
Saugat Pachhai 46e5053705 update: support specifying revision when updating (#3337)
* update: work with rev

* completion: update --rev completion scripts

* test: fix unit test assumption

* test: use fspath_py35
2020-02-15 17:55:41 +02:00
Guro Bokum 7e6e8ee169 api: add list command (#3246)
Close #2509
2020-02-12 22:10:48 +02:00
Ruslan Kuprieiev 0c8897949e diff: use rev instead of ref (#3299)
Fixes #3245
2020-02-11 00:13:23 +02:00
Mr. Outisand@dmpetrov e5145e6c55 diff: format summary to make it more human readable
Co-authored-by: @dmpetrov
2020-01-30 13:15:49 -06:00
Mr. Outis 793a069644 tests/diff: order dictionary (second try) 2020-01-30 10:00:56 -06:00
Mr. Outis e9bd9599a2 tests/diff: dont care about sorted jsons 2020-01-30 09:26:35 -06:00
Mr. Outis e501549bd7 diff: add a summary at the bottom 2020-01-29 18:54:45 -06:00
Mr. Outis 8e9d8145cd diff: remove deadcode, replace diffable with dicts 2020-01-29 17:42:35 -06:00
Mr. Outis 423a6163bc diff: make --checksums work with --json-show 2020-01-27 14:10:37 -06:00
Mr. Outis fced1718cc diff: --json -> --show-json 2020-01-27 12:53:24 -06:00
Mr. Outis 2370cb12b3 diff: use logger instead of print 2020-01-27 12:33:24 -06:00
Saugat PachhaiandPaweł Redzyński 98f6d56e8c test: refactor, apply suggestion from @pared
Co-Authored-By: Paweł Redzyński <[email protected]>
2020-01-16 22:59:58 +05:45
Saugat Pachhai 02a73a0d9b unit: test: check output of get --show-url command 2020-01-16 22:52:35 +05:45
Saugat Pachhai eb252e055b get: split command for getting url and getting files from url functions 2020-01-16 19:28:14 +05:45
Saugat Pachhai 4509f82bc3 get: implement --show-url to display only url/path to remote 2020-01-16 19:28:14 +05:45
Ruslan Kuprieiev 88fb097275 Merge pull request #3138 from pared/2896_repro_cmd
tests: repro: unit: migrate to dir helpers
2020-01-14 15:29:03 +02:00
Ruslan Kuprieiev 6869ee65e0 Merge pull request #3139 from pared/2896_run_cmd
tests: run: unit: migrate to dir helpers
2020-01-14 15:28:56 +02:00
pawel 26c4845157 tests: run: unit: migrate to dir helpers 2020-01-14 14:02:26 +01:00
pawel ba63900b29 tests: repro: unit: migrate to dir helpers 2020-01-14 13:56:56 +01:00
pawel cabe619d97 update: tests: unit: migrate to dir helpers 2020-01-14 13:55:02 +01:00
Ruslan Kuprieiev e393e29ef5 metrics: introduce diff
This first implementation is based on existing `dvc metrics show`
functionality, but is also able to auto-detect json to properly
show diff for it.
2020-01-13 06:45:48 +02:00
pawel e1176bf42d imp/imp_url cmd test: remove unnecessary dvc fixture 2020-01-06 16:40:15 +01:00
pawel a620658884 http/ imp tests - migrate do dir helpers 2020-01-06 16:18:01 +01:00
Alexander Schepanovski 1143a8c57f test: add checkout parse-cli-api test 2019-12-12 20:16:17 +07:00
Ruslan Kuprieiev 404620e267 test: update: use dvc_repo fixture
To fix test failure.
2019-11-19 19:12:49 +02:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Jorge Orpinel a48481b01f dvc: change ` for ' in dynamic parts of output messages (#2644)
* dvc: change ` for ' in dinamic parts of DVC outputs

per https://github.com/iterative/dvc/pull/2644#issuecomment-545151398

* dvc: use ' for dynamic output and ` for DVC names

per https://github.com/iterative/dvc/pull/2644#discussion_r338417698
2019-10-28 22:00:53 +02:00
Ruslan Kuprieiev 357ce82f28 run: introduce --always-changed
This is an extension of our callback stages (no deps, so always
considered as changed) for stages with dependencies, so that these
stages could be used properly in the middle of the DAG. See attached
issue for more info.

Related to #2378
2019-09-09 18:03:28 +03:00
Alexander Schepanovski db19957b8a Download refactor (#2275)
* remote: drop not working http resume

Closes #2136

* remote: drop @transfer_context()

It is not needed for ssh anymore and GS/S3 are switched to client
instances in cached properties.

* remote: drop upload/download chunking
2019-07-16 16:27:44 +03:00