Commit Graph
62 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 45944ddde7 metrics: rework metrics implementation (#3658)
Dropping support for all types except json and yaml. No more xpath
support.

Fixes #3572
2020-04-22 13:39:59 +03:00
Saugat Pachhai 128fa7f887 dvc: add a support for multistage dvcfile. (#3584)
Now, pipeline stage files can house multiple stages, and separate lockfiles
are created which has the checksums, whereas Dvcfile will be clean and human
readable and editable. The *.dvc files will be generated for output files.

It is available via a hidden flag: -n.

Fixes #1871
PR: #3584
2020-04-17 22:58:36 +03:00
Ruslan Kuprieiev 8455b7bece params: diff: introduce --all (#3631)
* params: diff: introduce --all

Makes `dvc params diff` also show unchanged params.

* params: diff: show all present parameters
2020-04-15 13:21:24 +03:00
Saugat PachhaiandRuslan Kuprieiev 65854c4946 refactor: extract stage's relationship with Dvcfile out from Stage (#3619)
* refactor stage into dvcfile

* Split create_stage out of dvcfile to Stage::create() and overwrite

* Remove logging from Repo::run

* tests: stop using --remove-outs

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-04-12 23:09:29 +03:00
Ruslan Kuprieiev 63fe2893a8 run: remove deprecated --cwd (#3623)
It has been deprecated for a long time now and it significantly
complicates the code, hiding multiple bugs.
2020-04-11 19:57:05 +03:00
Ruslan Kuprieiev 0e854f9534 params: diff: sort table (#3617) 2020-04-09 21:23:18 +03:00
Ruslan Kuprieiev c1ac6bb06b dvc: introduce dvc params diff (#3573)
* params: use repo.tree

* remote: local: use `repo.tree` for `exists()`

* param: handle missing params

* stage: move status warning to debug

* dvc: introduce `dvc params diff`

Fixes #3528
2020-04-09 05:18:49 +03:00
Ruslan Kuprieiev 26980ae45b metrics: diff: fix float precision formatting (#3609)
Texttable will try to detect the column type automatically, which
results in a low precision of the float that it is showing. We need to
disable that auto-formatting and just use `str(val)` instead.
2020-04-07 19:42:25 +03:00
Ruslan Kuprieiev 7bd473371c push/pull/status/metrics: add support for --all-commits (#3587)
First steps for #1691
2020-04-04 19:30:11 +03:00
Ruslan Kuprieiev 8e675a9d42 metrics: diff: don't print "No changes" (#3576)
Makes it easier to use in shell scripts. E.g. `-z $(dvc metrics diff)`, which
has the same semantics as `git diff`. Also syncs up with the behaviour
of `dvc diff`.

Fixes #3469
2020-04-02 04:54:40 +03:00
Ruslan Kuprieiev 0bf6fdae14 metrics: diff: handle deleted/new metric (#3529)
Partial fix for https://github.com/iterative/dvc/issues/3469
2020-03-25 05:23:21 +02:00
Helge Munk JacobsenandRuslan Kuprieiev 28748ff345 introduce hyper parameters and config (#3515)
* Work in progress

* added file parsing and name validation + adjust schema

* Exceptions on bad input

* Support multiple parameters

* Support multi 's in
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!

* fix formatting

* dep: param: fix schema

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* run: check if --params is specified

* dvc: rework param handling

* params: fix exc formatting

* params: support basic nested configs

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-03-24 17:56:39 +02:00
Saugat Pachhai 1455e9551e pull/checkout: fix empty output if no state gets changed (#3520) 2020-03-24 13:48:06 +02:00
Saugat PachhaiandJorge Orpinel 066ccea733 checkout: show summary by default and introduce --show-changes flag (#3401)
* checkout: implement --show-changes flag to show detailed checkout details
Show summary by default on checkout

checkout: show directory modified even when files are removed from inside it

checkout: refactor template logic

checkout: change output format

checkout: do not show summary if nothing's changed

checkout: do not show summary if show_changes is specified

checkout: show '/' at the end if it's a directory

rename variable name

test: cover stats for checkout

checkout: show relink as not modified

checkout: remove unwanted message during empty checkout

pull: show summary on checkout

import/get: change external_repo to catch CheckoutError

fix tests and check if path is url

checkout: support external deps/outputs

tests: checkout on external outputs

* fix tests

* checkout: make show changes default and introduce summary flag to show summary

* address suggestions

* exceptions: remove extra spacing on erorr

* Update dvc/command/checkout.py

Co-Authored-By: Jorge Orpinel <[email protected]>

Co-authored-by: Jorge Orpinel <[email protected]>
2020-03-16 16:29:36 +02:00
Saugat Pachhai d2734dfe1e tests: remove use of mocker as context-manager (#3480) 2020-03-16 08:25:22 +05:45
Guro Bokum 02cd520acd api: colorize dvc list (#3351)
Close #3347
2020-03-09 14:07:23 +02:00
Ivan Shcheklein eebbf3f812 change diff checksum arg name to hash (#3384) 2020-02-23 06:32:35 +02:00
Saugat Pachhai 46e5053705 update: support specifying revision when updating (#3337)
* update: work with rev

* completion: update --rev completion scripts

* test: fix unit test assumption

* test: use fspath_py35
2020-02-15 17:55:41 +02:00
Guro Bokum 7e6e8ee169 api: add list command (#3246)
Close #2509
2020-02-12 22:10:48 +02:00
Ruslan Kuprieiev 0c8897949e diff: use rev instead of ref (#3299)
Fixes #3245
2020-02-11 00:13:23 +02:00
Mr. Outisand@dmpetrov e5145e6c55 diff: format summary to make it more human readable
Co-authored-by: @dmpetrov
2020-01-30 13:15:49 -06:00
Mr. Outis 793a069644 tests/diff: order dictionary (second try) 2020-01-30 10:00:56 -06:00
Mr. Outis e9bd9599a2 tests/diff: dont care about sorted jsons 2020-01-30 09:26:35 -06:00
Mr. Outis e501549bd7 diff: add a summary at the bottom 2020-01-29 18:54:45 -06:00
Mr. Outis 8e9d8145cd diff: remove deadcode, replace diffable with dicts 2020-01-29 17:42:35 -06:00
Mr. Outis 423a6163bc diff: make --checksums work with --json-show 2020-01-27 14:10:37 -06:00
Mr. Outis fced1718cc diff: --json -> --show-json 2020-01-27 12:53:24 -06:00
Mr. Outis 2370cb12b3 diff: use logger instead of print 2020-01-27 12:33:24 -06:00
Saugat PachhaiandPaweł Redzyński 98f6d56e8c test: refactor, apply suggestion from @pared
Co-Authored-By: Paweł Redzyński <[email protected]>
2020-01-16 22:59:58 +05:45
Saugat Pachhai 02a73a0d9b unit: test: check output of get --show-url command 2020-01-16 22:52:35 +05:45
Saugat Pachhai eb252e055b get: split command for getting url and getting files from url functions 2020-01-16 19:28:14 +05:45
Saugat Pachhai 4509f82bc3 get: implement --show-url to display only url/path to remote 2020-01-16 19:28:14 +05:45
Ruslan Kuprieiev 88fb097275 Merge pull request #3138 from pared/2896_repro_cmd
tests: repro: unit: migrate to dir helpers
2020-01-14 15:29:03 +02:00
Ruslan Kuprieiev 6869ee65e0 Merge pull request #3139 from pared/2896_run_cmd
tests: run: unit: migrate to dir helpers
2020-01-14 15:28:56 +02:00
pawel 26c4845157 tests: run: unit: migrate to dir helpers 2020-01-14 14:02:26 +01:00
pawel ba63900b29 tests: repro: unit: migrate to dir helpers 2020-01-14 13:56:56 +01:00
pawel cabe619d97 update: tests: unit: migrate to dir helpers 2020-01-14 13:55:02 +01:00
Ruslan Kuprieiev e393e29ef5 metrics: introduce diff
This first implementation is based on existing `dvc metrics show`
functionality, but is also able to auto-detect json to properly
show diff for it.
2020-01-13 06:45:48 +02:00
pawel e1176bf42d imp/imp_url cmd test: remove unnecessary dvc fixture 2020-01-06 16:40:15 +01:00
pawel a620658884 http/ imp tests - migrate do dir helpers 2020-01-06 16:18:01 +01:00
Alexander Schepanovski 1143a8c57f test: add checkout parse-cli-api test 2019-12-12 20:16:17 +07:00
Ruslan Kuprieiev 404620e267 test: update: use dvc_repo fixture
To fix test failure.
2019-11-19 19:12:49 +02:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Jorge Orpinel a48481b01f dvc: change ` for ' in dynamic parts of output messages (#2644)
* dvc: change ` for ' in dinamic parts of DVC outputs

per https://github.com/iterative/dvc/pull/2644#issuecomment-545151398

* dvc: use ' for dynamic output and ` for DVC names

per https://github.com/iterative/dvc/pull/2644#discussion_r338417698
2019-10-28 22:00:53 +02:00
Ruslan Kuprieiev 357ce82f28 run: introduce --always-changed
This is an extension of our callback stages (no deps, so always
considered as changed) for stages with dependencies, so that these
stages could be used properly in the middle of the DAG. See attached
issue for more info.

Related to #2378
2019-09-09 18:03:28 +03:00
Alexander Schepanovski db19957b8a Download refactor (#2275)
* remote: drop not working http resume

Closes #2136

* remote: drop @transfer_context()

It is not needed for ssh anymore and GS/S3 are switched to client
instances in cached properties.

* remote: drop upload/download chunking
2019-07-16 16:27:44 +03:00
Ruslan Kuprieiev 94fe86afe8 tests: import-url: rewrite and move error msg test (#2262)
This one was failing randomly.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-14 03:09:08 +03:00
Ruslan Kuprieiev 932b0d7af9 introduce dvc update for ext repos (#2218)
* erepo: dvc: implement `update` for imports from ext repos

Fixes #1774
Fixes #2139
Fixes #2201

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* deps: repo: make  private

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* update: only support repo imports

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-07-06 02:28:35 +03:00
Ruslan Kuprieiev 918ef646dc appveyor: test py2 and py3
Related to #2166

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-23 13:33:47 +03:00
Ruslan Kuprieiev 1ba8e0171d dvc: replace pkg concept with external repo
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-21 01:16:00 +03:00