Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
5,606 Commits 7 Branches 579 Tags
a7e825a0386b6aa08a33eac5262246c86b868588
Commit Graph
295 Commits
Author SHA1 Message Date
Saugat Pachhai b81ace54df Enable pylint checks (#4092) 
* pylint: enable checks for imports

* ci: add github action for pylint

* enable pylint on pre-commit

* Install requirements

* use local pylint installation rather than of pre-commit's

* fix import errors

* more checks

* Silence unused imports

* Make a copy of dict
 2020-06-24 18:02:39 +03:00
Ruslan Kuprieiev 8495b0521b rpm/deb/osxpkg: add url and license (#4098) 
We are currently missing url and license metadata:
```
Name        : dvc
Arch        : x86_64
Version     : 0.94.0
Release     : 1
Size        : 172 M
Repo        : installed
From repo   : dvc
Summary     : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
URL         : http://example.com/no-uri-given
License     : unknown
Description : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
```

https://github.com/iterative/dvc/issues/4088#issuecomment-648270797
 2020-06-23 21:46:39 +03:00
Casper da Costa-Luis fa0f182028 completion: autogenerate (#3921) 
* cli: split main_parser

* cli: basic parser tree printing

* cli: major completion tidy

* completion: more missing root commands

* completion: much automagic such beauty

- fixes #2985 automatically generate bash completion

* tests: bash completion

* tests: completion: tidy script a little

* minor tidy

* command: positional name consistency

* completion: bash: add logging

* double-check renames and consistency

* cli: add completion subcommand

* ci: completion: update test

* minor logging

* more completion logging verbosity

* fix completion of files

* add shtab

* purge unneeded completion => shtab

* completion: neaten UI

* completion: use `shtab>=0.0.2`

* completion: purge hardcoded completions!

* revert unneeded renames for now

* fix isort

* completion: tidy CLI API

* fix snap build

* fix snap build again

* snap: internally build completion

* allow repo-less completion

* command: tidy headless logic

* use CmdBaseNoRepo

* better DVC-file completion

* merge command.choices -> command.completion

* drop choices class

* remove completion optionals

* tidy docs
 2020-06-19 04:54:20 +03:00
Ruslan Kuprieiev 15c787f2ac innosetup: commit images 2020-06-18 23:39:33 +03:00
Casper da Costa-Luis 2ba73daedc snap: make future major version upgrades easier (#4038) 
Related: #3999
Related: #4037
 2020-06-13 23:27:03 +03:00
Saugat PachhaiandRuslan Kuprieiev 9e7dc2640e schema: provide a jsonschema for dvc.yaml (#4028) 
* schema: provide a jsonschema for dvc.yaml

* Update scripts/schema/dvc-yaml.json

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update scripts/schema/dvc-yaml.json

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-06-12 20:09:59 +05:45
Casper da Costa-Luis 5efc3bbee2 snap: build fixes (#3990) 
* snap: drop sudo

Fixes #3989

* snap: ci: fix builds

* snap: ci: developer notifications

* snap: minimise size

* debug commit

* Revert "snap: minimise size"

This reverts commit 77b41a96f68b2c8df833803d0a644172b7db4779.

* fix image info

* Revert "debug commit"

This reverts commit 232baf0ad78335b0f3806456a3f20b82cdf0f24c.

* snap: v1 channels

Related: #3872
 2020-06-10 00:00:15 +03:00
Saugat Pachhai b47b2c5673 tests: update mock-ssh-server (#3986) 2020-06-09 12:39:10 +03:00
Ruslan Kuprieiev 791ce22759 pyinstaller: add hook for google-api-python-client (#3965) 
Fixes #3913
 2020-06-08 11:53:12 +03:00
Ruslan Kuprieiev 036ce9398e tests: move mock-ssh-server git dependency (#3953) 
This one doesn't allow us to release new pre-releases, because pypi
forbids that:

https://travis-ci.com/github/iterative/dvc/jobs/343343967#L2447

Let's install it separately for now, until there is a new official
mock-ssh-server release.
 2020-06-04 05:54:29 +03:00
Ruslan Kuprieiev 13012f9c0c plots: completion/spelling fixups (#3952) 
* completion: don't forget --show-vega

Fixes #3890

* plots: VEGA -> Vega
 2020-06-04 05:49:20 +03:00
Alexander Schepanovski 9a55b909f8 Rename --xlab to --x-label, X_TITLE to X_LABEL, same for y (#3948) 
Renamed:
- cli options
- dvcfile plot props
- template anchors
- variable names throughout the code
 2020-06-03 20:30:17 +03:00
Ruslan Kuprieiev ac4f351daf dvc: require --external for external outputs (#3929) 
Fixes #1545
 2020-06-03 06:13:09 +03:00
Ruslan Kuprieiev 86f3457dec dvc: remove dvc metrics add/remove (#3928) 
We will be re-introducing `metrics modify` in the future and
will likely revisit `add/remove` too, so it is better to get
rid of these for now and introduce later with a clear scneario
in mind.

Fixes #3849
Fixes #3907
 2020-06-02 00:17:03 +03:00
Ruslan Kuprieiev d1e08b6157 dvc: replace pipeline with dag (#3905) 
`pipeline list` is no more, it had some very questionable applications
(none to be precise) and was duplicating old `pipeline show` functionality.
We might introduce a different command to assist in listing (e.g. for
shell completion) in the future.

New `dvc dag` replaces `dvc pipeline show` and by-default will show
all pipelines in the project in ASCII (so there is no more --ascii)
and also `--dot` support, just like before. When given a target,
`dvc dag` (in any mode) will show DAG that leads to the target,
skipping the descendants, unless `--full` is specified.
 2020-06-01 02:35:36 +03:00
Ruslan Kuprieiev 9ee62ef8f0 dvc: rename "working tree" to "workspace" (#3914) 
Fixes #3892
 2020-05-31 17:39:04 +03:00
Ruslan Kuprieiev 0bb49d2549 dvc: rename dvc lock -> dvc freeze (#3915) 
Fixes #3901
 2020-05-31 03:38:21 +03:00
Ruslan Kuprieiev f504245f35 pre-commit: don't run dvc in parallel (#3912) 
* pre-commit: use local hooks

* pre-commit: don't run dvc in parallel

By default pre-commit will try to run hooks in parallel, which results
in some dvc instances failing to acquire repo locks.

Fixes #3751
 2020-05-29 21:20:46 +03:00
karajan1001 c5920b00ab Auto-completion of the dvc remote rename command (#3868) 
fix #3847
 2020-05-24 20:41:35 +03:00
Casper da Costa-Luis ce8036e535 snap: release non-stable-semver tags to beta (#3865) 
* snap: release non-stable-semver tags to beta

- fixes #3860

* snap: document --beta
 2020-05-24 20:40:53 +03:00
Ruslan Kuprieiev b7ad2dc841 test: fix hdfs/azurite/oss tests (#3864) 
* travis: fix typo

* tests: azure: fix expected etag format

* test: ssh: increase the number of tries

* travis: don't install docker

* dvc: update pyarrow

* tests: ssh: don't leak connections

* remote: don't fail when cache/remote doesn't have prefix subdir
 2020-05-23 23:46:40 +03:00
Ruslan Kuprieiev 865d355554 params/metrics: introduce --no-path and --old (#3845) 
* metrics: diff: add --no-path

Part of #3690

* params: diff: introduce --no-path

* metrics: diff: introduce --old

Part of #3690
 2020-05-22 04:35:42 +03:00
Ruslan Kuprieiev 130b76c7a5 plots: remove --select CLI option (#3840) 
* plots: remove --select from CLI

It needs reconsideration, we can't release it as is right now. E.g. it
expects xpath to start with `$`, but that will get interpreted by your
shell, which is bad.

* completion: update plots
 2020-05-21 17:57:27 +03:00
Shankho Boron GhoshandRestyled.io 19eaced304 dvc : renamed --outs-only to --dvc-only (#3797) 
* Renamed --outs-only to --dvc-only

* Changed outs_only to dvc_only

* replace output_only with dvc_only

* Restyled by black

Co-authored-by: Restyled.io <[email protected]>
 2020-05-20 18:24:16 +03:00
Ruslan Kuprieiev cec4b4e003 dvc: rename plot to plots (#3802) 
* dvc: rename plot to plots

Follows `dvc params/metrics` convention and is needed in preparation for new
commands and some refactoring.

Part of #3409

* completion: rename plot -> plots
 2020-05-14 22:28:22 +03:00
Ruslan Kuprieiev 974cf097dd run/repro: rename "build cache" -> "run cache" (#3788) 
Part of #3777
 2020-05-14 02:27:18 +03:00
Saugat Pachhai cc16b28a26 diff: autocompletion script for params, and show-md (#3786) 2020-05-12 16:41:09 +03:00
Paweł Redzyński af8ac006ad plot: completion for zsh (#3733) 2020-05-04 15:01:29 +03:00
Paweł RedzyńskiandRuslan Kuprieiev e553511085 Metrics - plotting for multiple revisions initial (#3577) 
* init

* rename to plot data insertion basig on dicts update

* revision support

* roll back revision

* plot makedirs for backward compatibility

* log path

* pretty plot link to visualization page

* make target default title

* efiop review

* efiop review

* plot multiple initial

* add some missing metric file tests

* proper id generation

* proper id generation

* add confusion matrix template

* refactor tests

* plot from dvct file

* plot from dvct

* brush up commands

* fix confusion matrix multiple plot

* plot: change confusion matrix data schema

* should be working as intended

* support for src file in dvct files

* minor fixes

* plot: support json templates

* plot: rename confusion template

* plot: polish command behaviour

* fix test for json

* plot: test command

* some minor fixes for tests

* plot: unit test loading

* plot: unit test loading

* plot: handle TODOS

* cleanup

* use mocker

* plot: support tsv

* plot: command refactoring

* plot: fix windows issues with tests

* plot: test: some more windows fixes

* plot: _load_from_revisions complexity fix

* plot: reduce complexity

* plot: complexity reduction

* plot: deepsource suggestions

* plot: move template path evaluation

* fixup

* fixup

* exception on no datafile and no template

* json metric load with OrderedDict

* plot: improve handling non-existing files on revisions

* plot: improve handling non-existing files on revisions

* change default plot path

* some exceptions and fixes

* add yaml metrics support

* fixup

* some more suggestions

* default filename fix

* efiop review requests

* log exception on failur

* move revisions deduction to commands

* json templates

* extract template filling to separate method

* some parsing improvements

* add columns functionality

* extract default data transformation to separate method

* plot: initial support for jsonpath

* plot: rename columns to filters, tests are dict based

* plot: fixups

* plot: refactoring

* repo: plot: convert to package

* plot: data loading refactor, support searching for data

* plot: raise if wrong fields provided

* plot: command description

* plot: default: pass y axis info for default plot

* plot: get rid of fieldnames, expect ordered data

* plot: handle default plot in separate method

* plot: fix default

* plot: command option names fixes

* refactoring

* fixes

* plot: provide option for stdout redirection

* plot: rename show-json to no-html

* plot: add no-csv-header option

* plot: improve error message for wrongly structured metric

* plot: match template name exactly, whit suffix appended only

* plot: dmpetrov and ivan review

* plot: refactor --stdout help message

* plot: move template to repo/plot

* plot: add -x and -y options

* plot: add -x and -y options

* plot: command: order change

* plot: scatter

* plot: rename confusion matrix template, new name generation format

* plot: add title anchor

* plot: review from jorgeorpinel

* plot: rename filter and result options to select and file

* plot: add --title, --x-title, --y-title

* plot: xlab ylab

* Update dvc/repo/plot/template.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/repo/plot/template.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* efiop review

* plot: bash completion

* plot: static code analysis fixes

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-05-01 22:45:03 +03:00
Ruslan Kuprieiev a7b77672e6 innosetup: rename exe app name 
Fixes https://github.com/iterative/dvc/issues/3687
 2020-04-28 16:08:17 +03:00
Casper da Costa-Luis e05157f8e0 completion: bash: fix files (#3613) 
Fixes lock, metrics and move autocompletion of files
 2020-04-10 10:02:40 +03:00
Jorge Orpinel f325921315 diff: update help output to match docs, et al. (#3601) 
* diff: update help output to match docs
per https://github.com/iterative/dvc/pull/3566#pullrequestreview-388346350

* terms: git->Git, scm->SCM, spacing
 2020-04-07 12:50:11 +03:00
Jorge Orpinel d2ea180a9a pull: update help output (-R option) (#3555) 2020-03-31 19:53:34 +03:00
Saugat Pachhai e1334448ea completion: add --summary flag on checkout (#3491) 2020-03-17 14:04:12 +02:00
Saugat PachhaiandCasper da Costa-Luis 109ded4068 dvc: support Python3.8 (#3463) 
* py38: make wheels

- ci: set ARROW_LIBHDFS_DIR to load native library
- deps: pin pyarrow to 0.15.1 for py<=3.7
- pin deps for python<3.8
- pin deps for python==3.8 as well
- add windows Py38 on travis
- do not use speedcopy for Python3.8 and Windows

* use realpath to resolve symbolic links

* do not build pkgs for Windows and Linux on Python3.8

* reorder

* pyarrow: document regression fix

Co-authored-by: Casper da Costa-Luis <[email protected]>
 2020-03-16 19:18:33 +00:00
Guro Bokum dcf5af647e list: rename target param to path, update docs (#3462) 
Close #3381
 2020-03-16 20:22:48 +02:00
Casper da Costa-Luis 7923446219 completion: bash: misc fixes and updates (#3467) 
* completion: bash: fix, update and sort

* completion: bash: fix missing `compgen`s

* completion: bash: remove deprecated/hidden options

* completion: bash: fix extra parenthesis

- fixes metrics show -t

* completion: bash: re-add `gc --all-commits`

Due to #3429 superceding #2681
 2020-03-11 15:56:20 +02:00
Ruslan Kuprieiev 40519005f5 travis: hadoop: update tar location 2020-03-10 10:20:34 +02:00
Saugat Pachhai ea981ae955 gc: make --all-commits flag visible (#3429) 
* gc: make --all-commits visible

* gc: fix help message and completion scripts

* bash: add longform --workspace flag
 2020-03-10 08:29:07 +05:45
Ruslan Kuprieiev 37e4a5dcd4 travis: use us-west-2 aws s3 region 2020-03-09 19:10:47 +02:00
Ivan Shcheklein eebbf3f812 change diff checksum arg name to hash (#3384) 2020-02-23 06:32:35 +02:00
Casper da Costa-Luis 411057d1b4 completion: bash: properly recurse targets (#3374) 
* completion: bash: properly recurse commands

TODO: subcommands

* completion: bash: add subcommand path completion

* completion: bash: comments
 2020-02-21 04:03:23 +02:00
Casper da Costa-Luis 2820aedb87 completion: bash: allow multi-arg completion, minor tidy (#3367) 2020-02-19 23:41:48 +02:00
Casper da Costa-Luis a9161af593 Fix bash completion (#3360) 
* completion: safer function names

* completion: fix missing expansions

Fixes #1923

* complete: bash: all filenames

* complete: bash: DVC-files

* complete: bash: safer and quicker opts
 2020-02-19 04:16:09 +02:00
Jorge Orpinel b23f43a7e7 term: avoid "under X control" and other help output updates (#3322) 
* get/import: update cmd and url arg descs with correct behavior

* get/import: update path arg desc as well (See previous commit)

* get/imp: udpate help output with simpler terms
per https://github.com/iterative/dvc/pull/3277#pullrequestreview-352650016

* api: url->URL in a docstring

* metrics show: update help output per
per https://github.com/iterative/dvc.org/pull/933#pullrequestreview-351862579

* diff: update `diff` and `metrics diff` help desc to match docs
per https://github.com/iterative/dvc.org/pull/933#pullrequestreview-356890960

* diff: update autocomplete scripts
per https://github.com/iterative/dvc/pull/3308#issuecomment-585220300

* diff: update help output again
per https://github.com/iterative/dvc.org/pull/933#pullrequestreview-356890960

* diff/metrics diff: update help output yet again
per https://github.com/iterative/dvc.org/pull/933#discussion_r378654316

* term" under X control -> tracked by X
 2020-02-17 17:24:30 +02:00
Saugat Pachhai 46e5053705 update: support specifying revision when updating (#3337) 
* update: work with rev

* completion: update --rev completion scripts

* test: fix unit test assumption

* test: use fspath_py35
 2020-02-15 17:55:41 +02:00
Guro Bokum 7e6e8ee169 api: add list command (#3246) 
Close #2509
 2020-02-12 22:10:48 +02:00
Ruslan Kuprieiev de1f67afbd Merge pull request #3244 from jorgeorpinel/master 
diff/metics diff: update argument descriptions, et al.
 2020-02-03 02:15:54 +02:00
Casper da Costa-Luis b0b69ce819 lint/pre-commit/ci: replace shfmt => beautysh 2020-01-31 19:14:51 +00:00
Casper da Costa-Luis 4e87bf9e0e try beautysh 2020-01-31 19:07:53 +00:00