Commit Graph
323 Commits
Author SHA1 Message Date
Ruslan Kuprieiev ba38bb300c sign: increase timeout to 180 sec 2021-04-02 15:56:56 +03:00
Ruslan Kuprieiev b2503d5178 pyinstaller: hooks: bring back hook-googleapiclient.model.py
Got deleted by mistake in https://github.com/iterative/dvc/pull/5682/files
2021-04-01 21:34:26 +03:00
Ruslan Kuprieiev c4622c6311 sign: increase timeout to 90 2021-04-01 02:11:28 +03:00
Ruslan Kuprieiev 2f759d60be sign: increase timeout 2021-04-01 01:41:36 +03:00
Peter Rowlands (변기호) e91168812a build: fix fpm build zsh-completion path (#5743) 2021-03-31 16:45:34 +09:00
Ruslan Kuprieiev 98edde06a4 sign: increase timeout 2021-03-29 16:27:05 +03:00
Ruslan Kuprieiev 30f9ad4b54 sign: add timeout and verbosity 2021-03-29 15:54:32 +03:00
Ruslan Kuprieiev 1a25ebe3bd osxpkg: notarize: fix typo 2021-03-29 01:07:06 +03:00
Ruslan Kuprieiev 4ffa0675b4 build: fix --notarize 2021-03-26 03:47:34 +02:00
Ruslan Kuprieiev 49cdc2bea3 build: add osx signing/notarization (#5695)
* build: add osx signing/notarization

* gha: temporarily enable pkg builds
2021-03-26 02:31:32 +02:00
Ruslan Kuprieiev 8abffbf68b scripts: fpm: install zsh competion to install / "share"
Currently we create /usr/share/... which is not allowed by macos:

> the package is trying to install content to the system volume
2021-03-25 02:21:11 +02:00
Ruslan Kuprieiev f2ba67aece build: upgrade pyinstaller to 4.2 (#5682)
* build: upgrade pyinstaller to 4.2

* pyinstaller: hooks: remove hook-googleapiclient.model.py

https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py

* gha: use python 3.8 for pyinstaller
2021-03-24 09:43:05 +02:00
Ruslan Kuprieiev 19cdef1675 build: separate building and packaging (#5675)
* build: separate pyinstaller and fpm/innosetup

* gha: upload deb/rpm/osxpkg/exe artifacts

* build: dynamically generate dvc version
2021-03-23 19:55:44 +02:00
Helio MachadoandPeter Rowlands 4ed55d0051 build: add pyinstaller hook for googleapiclient (#5619)
* build: add pyinstaller hook for googleapiclient

The Google API Discovery Service JSON files from
googleapiclient/discovery_cache/documents were being
excluded due to a bug and, particularly, drive.v2.json
is essential for our gdrive backend. Now we're explicitly
asking PyInstaller to embed these files on the generated packages.

Fixes #5618

* Suppress linter error for unrecognized imports

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Apply black style after manual code review

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-03-14 12:56:49 +02:00
Ruslan Kuprieiev a72cea67f1 build: require git >= 1.7.0 (#5467) 2021-02-15 20:54:07 +02:00
Asif Saif UddinandRuslan Kuprieiev f8c9797f2a add psutil to install requirement (#5232)
* add  psutil to install requirement

* info: remove unneded checks

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-01-18 12:30:38 +02:00
Saugat Pachhai 995287bbba Pin numpy for windows build (#5055) 2020-12-08 18:07:14 +02:00
Clément WalterandRuslan Kuprieiev 94f2a1d3be Allow user to pass a list of commands for one single stage (#4934)
* Allow list type in STAGE_DEFINITION scheme and join(cmd) if list

* Remove join in Stage.__init__ and update test

* Loop over commands in cmd_run

* Improve logging of stage.run with detailed by command

* Fix test based on log message

* Update command logging with >

* Add tests for multi lines command

* Update > to $ in stage command log to prevent confusion with > operator

* Fix PR review about test: single quote and read_text

* tests: unify/clarify tests

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-12-03 12:49:30 +02:00
Ruslan Kuprieiev 685e76cfbe gitignore: add config.ini
Generated by innosetup.
2020-11-09 13:25:45 +02:00
Ruslan Kuprieiev 7da3de451f build: test built binary (#4839) 2020-11-03 23:30:41 +02:00
Ruslan Kuprieiev 21de21fe37 gha: adjust the test workflow (#4615)
* gha: move lint check to tests workflow

* gha: tests: don't fail fast

* scripts: remove redundant check
2020-09-24 16:13:03 +03:00
Ruslan Kuprieiev 2457633990 gha: build packages (#4607) 2020-09-24 14:48:04 +03:00
Ruslan Kuprieiev f52763a0b3 dvc: use flatten-dict instead of flatten-json (#4500)
* dvc: use flatten-dict instead of flatten-json

Fixes #4499

* utils: add [un]flatten

* pyinstaller: add flatten-dict hook
2020-08-31 03:15:52 +03:00
Ruslan Kuprieiev f44727a4d0 build: fix pyinstaller build (#4264)
https://github.com/pypa/setuptools/issues/1963
2020-07-23 20:25:39 +03:00
Casper da Costa-Luis 4edf368d68 snap: deploy: fix major channels not being pushed (#4222) 2020-07-17 05:05:50 +03:00
Ruslan Kuprieiev 4c9245a3b3 tests: remove real SSH (#4200)
* tests: remove real SSH

No longer needed, all tests are using mocked ssh already.

* travis: don't setup ssh
2020-07-14 13:55:14 +03:00
Ruslan Kuprieiev 375998c664 tests: hdfs: use hadoop/docker fixtures (#4183)
Fixed #4054
2020-07-10 21:38:48 +03:00
Ruslan Kuprieiev 0899b277c0 tests: launch oss/azure containers from fixtures (#4178)
* tests: launch oss/azure containers from fixtures

Related to #4054

* ci: hdfs: remove unneeded fix

We are using pyarrow 0.17.0 already.
2020-07-08 19:54:17 +03:00
Saugat Pachhai b81ace54df Enable pylint checks (#4092)
* pylint: enable checks for imports

* ci: add github action for pylint

* enable pylint on pre-commit

* Install requirements

* use local pylint installation rather than of pre-commit's

* fix import errors

* more checks

* Silence unused imports

* Make a copy of dict
2020-06-24 18:02:39 +03:00
Ruslan Kuprieiev 8495b0521b rpm/deb/osxpkg: add url and license (#4098)
We are currently missing url and license metadata:
```
Name        : dvc
Arch        : x86_64
Version     : 0.94.0
Release     : 1
Size        : 172 M
Repo        : installed
From repo   : dvc
Summary     : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
URL         : http://example.com/no-uri-given
License     : unknown
Description : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
```

https://github.com/iterative/dvc/issues/4088#issuecomment-648270797
2020-06-23 21:46:39 +03:00
Casper da Costa-Luis fa0f182028 completion: autogenerate (#3921)
* cli: split main_parser

* cli: basic parser tree printing

* cli: major completion tidy

* completion: more missing root commands

* completion: much automagic such beauty

- fixes #2985 automatically generate bash completion

* tests: bash completion

* tests: completion: tidy script a little

* minor tidy

* command: positional name consistency

* completion: bash: add logging

* double-check renames and consistency

* cli: add completion subcommand

* ci: completion: update test

* minor logging

* more completion logging verbosity

* fix completion of files

* add shtab

* purge unneeded completion => shtab

* completion: neaten UI

* completion: use `shtab>=0.0.2`

* completion: purge hardcoded completions!

* revert unneeded renames for now

* fix isort

* completion: tidy CLI API

* fix snap build

* fix snap build again

* snap: internally build completion

* allow repo-less completion

* command: tidy headless logic

* use CmdBaseNoRepo

* better DVC-file completion

* merge command.choices -> command.completion

* drop choices class

* remove completion optionals

* tidy docs
2020-06-19 04:54:20 +03:00
Ruslan Kuprieiev 15c787f2ac innosetup: commit images 2020-06-18 23:39:33 +03:00
Casper da Costa-Luis 2ba73daedc snap: make future major version upgrades easier (#4038)
Related: #3999
Related: #4037
2020-06-13 23:27:03 +03:00
Saugat PachhaiandRuslan Kuprieiev 9e7dc2640e schema: provide a jsonschema for dvc.yaml (#4028)
* schema: provide a jsonschema for dvc.yaml

* Update scripts/schema/dvc-yaml.json

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update scripts/schema/dvc-yaml.json

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-12 20:09:59 +05:45
Casper da Costa-Luis 5efc3bbee2 snap: build fixes (#3990)
* snap: drop sudo

Fixes #3989

* snap: ci: fix builds

* snap: ci: developer notifications

* snap: minimise size

* debug commit

* Revert "snap: minimise size"

This reverts commit 77b41a96f68b2c8df833803d0a644172b7db4779.

* fix image info

* Revert "debug commit"

This reverts commit 232baf0ad78335b0f3806456a3f20b82cdf0f24c.

* snap: v1 channels

Related: #3872
2020-06-10 00:00:15 +03:00
Saugat Pachhai b47b2c5673 tests: update mock-ssh-server (#3986) 2020-06-09 12:39:10 +03:00
Ruslan Kuprieiev 791ce22759 pyinstaller: add hook for google-api-python-client (#3965)
Fixes #3913
2020-06-08 11:53:12 +03:00
Ruslan Kuprieiev 036ce9398e tests: move mock-ssh-server git dependency (#3953)
This one doesn't allow us to release new pre-releases, because pypi
forbids that:

https://travis-ci.com/github/iterative/dvc/jobs/343343967#L2447

Let's install it separately for now, until there is a new official
mock-ssh-server release.
2020-06-04 05:54:29 +03:00
Ruslan Kuprieiev 13012f9c0c plots: completion/spelling fixups (#3952)
* completion: don't forget --show-vega

Fixes #3890

* plots: VEGA -> Vega
2020-06-04 05:49:20 +03:00
Alexander Schepanovski 9a55b909f8 Rename --xlab to --x-label, X_TITLE to X_LABEL, same for y (#3948)
Renamed:
- cli options
- dvcfile plot props
- template anchors
- variable names throughout the code
2020-06-03 20:30:17 +03:00
Ruslan Kuprieiev ac4f351daf dvc: require --external for external outputs (#3929)
Fixes #1545
2020-06-03 06:13:09 +03:00
Ruslan Kuprieiev 86f3457dec dvc: remove dvc metrics add/remove (#3928)
We will be re-introducing `metrics modify` in the future and
will likely revisit `add/remove` too, so it is better to get
rid of these for now and introduce later with a clear scneario
in mind.

Fixes #3849
Fixes #3907
2020-06-02 00:17:03 +03:00
Ruslan Kuprieiev d1e08b6157 dvc: replace pipeline with dag (#3905)
`pipeline list` is no more, it had some very questionable applications
(none to be precise) and was duplicating old `pipeline show` functionality.
We might introduce a different command to assist in listing (e.g. for
shell completion) in the future.

New `dvc dag` replaces `dvc pipeline show` and by-default will show
all pipelines in the project in ASCII (so there is no more --ascii)
and also `--dot` support, just like before. When given a target,
`dvc dag` (in any mode) will show DAG that leads to the target,
skipping the descendants, unless `--full` is specified.
2020-06-01 02:35:36 +03:00
Ruslan Kuprieiev 9ee62ef8f0 dvc: rename "working tree" to "workspace" (#3914)
Fixes #3892
2020-05-31 17:39:04 +03:00
Ruslan Kuprieiev 0bb49d2549 dvc: rename dvc lock -> dvc freeze (#3915)
Fixes #3901
2020-05-31 03:38:21 +03:00
Ruslan Kuprieiev f504245f35 pre-commit: don't run dvc in parallel (#3912)
* pre-commit: use local hooks

* pre-commit: don't run dvc in parallel

By default pre-commit will try to run hooks in parallel, which results
in some dvc instances failing to acquire repo locks.

Fixes #3751
2020-05-29 21:20:46 +03:00
karajan1001 c5920b00ab Auto-completion of the dvc remote rename command (#3868)
fix #3847
2020-05-24 20:41:35 +03:00
Casper da Costa-Luis ce8036e535 snap: release non-stable-semver tags to beta (#3865)
* snap: release non-stable-semver tags to beta

- fixes #3860

* snap: document --beta
2020-05-24 20:40:53 +03:00
Ruslan Kuprieiev b7ad2dc841 test: fix hdfs/azurite/oss tests (#3864)
* travis: fix typo

* tests: azure: fix expected etag format

* test: ssh: increase the number of tries

* travis: don't install docker

* dvc: update pyarrow

* tests: ssh: don't leak connections

* remote: don't fail when cache/remote doesn't have prefix subdir
2020-05-23 23:46:40 +03:00
Ruslan Kuprieiev 865d355554 params/metrics: introduce --no-path and --old (#3845)
* metrics: diff: add --no-path

Part of #3690

* params: diff: introduce --no-path

* metrics: diff: introduce --old

Part of #3690
2020-05-22 04:35:42 +03:00