Ruslan Kuprieiev
4ffa0675b4
build: fix --notarize
2021-03-26 03:47:34 +02:00
Ruslan Kuprieiev
49cdc2bea3
build: add osx signing/notarization ( #5695 )
...
* build: add osx signing/notarization
* gha: temporarily enable pkg builds
2021-03-26 02:31:32 +02:00
Ruslan Kuprieiev
8abffbf68b
scripts: fpm: install zsh competion to install / "share"
...
Currently we create /usr/share/... which is not allowed by macos:
> the package is trying to install content to the system volume
2021-03-25 02:21:11 +02:00
Ruslan Kuprieiev
f2ba67aece
build: upgrade pyinstaller to 4.2 ( #5682 )
...
* build: upgrade pyinstaller to 4.2
* pyinstaller: hooks: remove hook-googleapiclient.model.py
https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py
* gha: use python 3.8 for pyinstaller
2021-03-24 09:43:05 +02:00
Ruslan Kuprieiev
19cdef1675
build: separate building and packaging ( #5675 )
...
* build: separate pyinstaller and fpm/innosetup
* gha: upload deb/rpm/osxpkg/exe artifacts
* build: dynamically generate dvc version
2021-03-23 19:55:44 +02:00
Helio Machado and Peter Rowlands
4ed55d0051
build: add pyinstaller hook for googleapiclient ( #5619 )
...
* build: add pyinstaller hook for googleapiclient
The Google API Discovery Service JSON files from
googleapiclient/discovery_cache/documents were being
excluded due to a bug and, particularly, drive.v2.json
is essential for our gdrive backend. Now we're explicitly
asking PyInstaller to embed these files on the generated packages.
Fixes #5618
* Suppress linter error for unrecognized imports
Co-authored-by: Peter Rowlands (변기호) <[email protected] [email protected]
2021-03-14 12:56:49 +02:00
Ruslan Kuprieiev
a72cea67f1
build: require git >= 1.7.0 ( #5467 )
2021-02-15 20:54:07 +02:00
Asif Saif Uddin and Ruslan Kuprieiev
f8c9797f2a
add psutil to install requirement ( #5232 )
...
* add psutil to install requirement
* info: remove unneded checks
Co-authored-by: Ruslan Kuprieiev <[email protected]
2021-01-18 12:30:38 +02:00
Saugat Pachhai
995287bbba
Pin numpy for windows build ( #5055 )
2020-12-08 18:07:14 +02:00
Clément Walter and Ruslan Kuprieiev
94f2a1d3be
Allow user to pass a list of commands for one single stage ( #4934 )
...
* Allow list type in STAGE_DEFINITION scheme and join(cmd) if list
* Remove join in Stage.__init__ and update test
* Loop over commands in cmd_run
* Improve logging of stage.run with detailed by command
* Fix test based on log message
* Update command logging with >
* Add tests for multi lines command
* Update > to $ in stage command log to prevent confusion with > operator
* Fix PR review about test: single quote and read_text
* tests: unify/clarify tests
Co-authored-by: Ruslan Kuprieiev <[email protected]
2020-12-03 12:49:30 +02:00
Ruslan Kuprieiev
685e76cfbe
gitignore: add config.ini
...
Generated by innosetup.
2020-11-09 13:25:45 +02:00
Ruslan Kuprieiev
7da3de451f
build: test built binary ( #4839 )
2020-11-03 23:30:41 +02:00
Ruslan Kuprieiev
21de21fe37
gha: adjust the test workflow ( #4615 )
...
* gha: move lint check to tests workflow
* gha: tests: don't fail fast
* scripts: remove redundant check
2020-09-24 16:13:03 +03:00
Ruslan Kuprieiev
2457633990
gha: build packages ( #4607 )
2020-09-24 14:48:04 +03:00
Ruslan Kuprieiev
f52763a0b3
dvc: use flatten-dict instead of flatten-json ( #4500 )
...
* dvc: use flatten-dict instead of flatten-json
Fixes #4499
* utils: add [un]flatten
* pyinstaller: add flatten-dict hook
2020-08-31 03:15:52 +03:00
Ruslan Kuprieiev
f44727a4d0
build: fix pyinstaller build ( #4264 )
...
https://github.com/pypa/setuptools/issues/1963
2020-07-23 20:25:39 +03:00
Casper da Costa-Luis
4edf368d68
snap: deploy: fix major channels not being pushed ( #4222 )
2020-07-17 05:05:50 +03:00
Ruslan Kuprieiev
4c9245a3b3
tests: remove real SSH ( #4200 )
...
* tests: remove real SSH
No longer needed, all tests are using mocked ssh already.
* travis: don't setup ssh
2020-07-14 13:55:14 +03:00
Ruslan Kuprieiev
375998c664
tests: hdfs: use hadoop/docker fixtures ( #4183 )
...
Fixed #4054
2020-07-10 21:38:48 +03:00
Ruslan Kuprieiev
0899b277c0
tests: launch oss/azure containers from fixtures ( #4178 )
...
* tests: launch oss/azure containers from fixtures
Related to #4054
* ci: hdfs: remove unneeded fix
We are using pyarrow 0.17.0 already.
2020-07-08 19:54:17 +03:00
Saugat Pachhai
b81ace54df
Enable pylint checks ( #4092 )
...
* pylint: enable checks for imports
* ci: add github action for pylint
* enable pylint on pre-commit
* Install requirements
* use local pylint installation rather than of pre-commit's
* fix import errors
* more checks
* Silence unused imports
* Make a copy of dict
2020-06-24 18:02:39 +03:00
Ruslan Kuprieiev
8495b0521b
rpm/deb/osxpkg: add url and license ( #4098 )
...
We are currently missing url and license metadata:
```
Name : dvc
Arch : x86_64
Version : 0.94.0
Release : 1
Size : 172 M
Repo : installed
From repo : dvc
Summary : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
URL : http://example.com/no-uri-given
License : unknown
Description : Data Version Control - datasets, models, and experiments
versioning for ML or data science projects
```
https://github.com/iterative/dvc/issues/4088#issuecomment-648270797
2020-06-23 21:46:39 +03:00
Casper da Costa-Luis
fa0f182028
completion: autogenerate ( #3921 )
...
* cli: split main_parser
* cli: basic parser tree printing
* cli: major completion tidy
* completion: more missing root commands
* completion: much automagic such beauty
- fixes #2985 automatically generate bash completion
* tests: bash completion
* tests: completion: tidy script a little
* minor tidy
* command: positional name consistency
* completion: bash: add logging
* double-check renames and consistency
* cli: add completion subcommand
* ci: completion: update test
* minor logging
* more completion logging verbosity
* fix completion of files
* add shtab
* purge unneeded completion => shtab
* completion: neaten UI
* completion: use `shtab>=0.0.2`
* completion: purge hardcoded completions!
* revert unneeded renames for now
* fix isort
* completion: tidy CLI API
* fix snap build
* fix snap build again
* snap: internally build completion
* allow repo-less completion
* command: tidy headless logic
* use CmdBaseNoRepo
* better DVC-file completion
* merge command.choices -> command.completion
* drop choices class
* remove completion optionals
* tidy docs
2020-06-19 04:54:20 +03:00
Ruslan Kuprieiev
15c787f2ac
innosetup: commit images
2020-06-18 23:39:33 +03:00
Casper da Costa-Luis
2ba73daedc
snap: make future major version upgrades easier ( #4038 )
...
Related: #3999
Related: #4037
2020-06-13 23:27:03 +03:00
Saugat Pachhai and Ruslan Kuprieiev
9e7dc2640e
schema: provide a jsonschema for dvc.yaml ( #4028 )
...
* schema: provide a jsonschema for dvc.yaml
* Update scripts/schema/dvc-yaml.json
Co-authored-by: Ruslan Kuprieiev <[email protected] [email protected]
2020-06-12 20:09:59 +05:45
Casper da Costa-Luis
5efc3bbee2
snap: build fixes ( #3990 )
...
* snap: drop sudo
Fixes #3989
* snap: ci: fix builds
* snap: ci: developer notifications
* snap: minimise size
* debug commit
* Revert "snap: minimise size"
This reverts commit 77b41a96f68b2c8df833803d0a644172b7db4779.
* fix image info
* Revert "debug commit"
This reverts commit 232baf0ad78335b0f3806456a3f20b82cdf0f24c.
* snap: v1 channels
Related: #3872
2020-06-10 00:00:15 +03:00
Saugat Pachhai
b47b2c5673
tests: update mock-ssh-server ( #3986 )
2020-06-09 12:39:10 +03:00
Ruslan Kuprieiev
791ce22759
pyinstaller: add hook for google-api-python-client ( #3965 )
...
Fixes #3913
2020-06-08 11:53:12 +03:00
Ruslan Kuprieiev
036ce9398e
tests: move mock-ssh-server git dependency ( #3953 )
...
This one doesn't allow us to release new pre-releases, because pypi
forbids that:
https://travis-ci.com/github/iterative/dvc/jobs/343343967#L2447
Let's install it separately for now, until there is a new official
mock-ssh-server release.
2020-06-04 05:54:29 +03:00
Ruslan Kuprieiev
13012f9c0c
plots: completion/spelling fixups ( #3952 )
...
* completion: don't forget --show-vega
Fixes #3890
* plots: VEGA -> Vega
2020-06-04 05:49:20 +03:00
Alexander Schepanovski
9a55b909f8
Rename --xlab to --x-label, X_TITLE to X_LABEL, same for y ( #3948 )
...
Renamed:
- cli options
- dvcfile plot props
- template anchors
- variable names throughout the code
2020-06-03 20:30:17 +03:00
Ruslan Kuprieiev
ac4f351daf
dvc: require --external for external outputs ( #3929 )
...
Fixes #1545
2020-06-03 06:13:09 +03:00
Ruslan Kuprieiev
86f3457dec
dvc: remove dvc metrics add/remove ( #3928 )
...
We will be re-introducing `metrics modify` in the future and
will likely revisit `add/remove` too, so it is better to get
rid of these for now and introduce later with a clear scneario
in mind.
Fixes #3849
Fixes #3907
2020-06-02 00:17:03 +03:00
Ruslan Kuprieiev
d1e08b6157
dvc: replace pipeline with dag ( #3905 )
...
`pipeline list` is no more, it had some very questionable applications
(none to be precise) and was duplicating old `pipeline show` functionality.
We might introduce a different command to assist in listing (e.g. for
shell completion) in the future.
New `dvc dag` replaces `dvc pipeline show` and by-default will show
all pipelines in the project in ASCII (so there is no more --ascii)
and also `--dot` support, just like before. When given a target,
`dvc dag` (in any mode) will show DAG that leads to the target,
skipping the descendants, unless `--full` is specified.
2020-06-01 02:35:36 +03:00
Ruslan Kuprieiev
9ee62ef8f0
dvc: rename "working tree" to "workspace" ( #3914 )
...
Fixes #3892
2020-05-31 17:39:04 +03:00
Ruslan Kuprieiev
0bb49d2549
dvc: rename dvc lock -> dvc freeze ( #3915 )
...
Fixes #3901
2020-05-31 03:38:21 +03:00
Ruslan Kuprieiev
f504245f35
pre-commit: don't run dvc in parallel ( #3912 )
...
* pre-commit: use local hooks
* pre-commit: don't run dvc in parallel
By default pre-commit will try to run hooks in parallel, which results
in some dvc instances failing to acquire repo locks.
Fixes #3751
2020-05-29 21:20:46 +03:00
karajan1001
c5920b00ab
Auto-completion of the dvc remote rename command ( #3868 )
...
fix #3847
2020-05-24 20:41:35 +03:00
Casper da Costa-Luis
ce8036e535
snap: release non-stable-semver tags to beta ( #3865 )
...
* snap: release non-stable-semver tags to beta
- fixes #3860
* snap: document --beta
2020-05-24 20:40:53 +03:00
Ruslan Kuprieiev
b7ad2dc841
test: fix hdfs/azurite/oss tests ( #3864 )
...
* travis: fix typo
* tests: azure: fix expected etag format
* test: ssh: increase the number of tries
* travis: don't install docker
* dvc: update pyarrow
* tests: ssh: don't leak connections
* remote: don't fail when cache/remote doesn't have prefix subdir
2020-05-23 23:46:40 +03:00
Ruslan Kuprieiev
865d355554
params/metrics: introduce --no-path and --old ( #3845 )
...
* metrics: diff: add --no-path
Part of #3690
* params: diff: introduce --no-path
* metrics: diff: introduce --old
Part of #3690
2020-05-22 04:35:42 +03:00
Ruslan Kuprieiev
130b76c7a5
plots: remove --select CLI option ( #3840 )
...
* plots: remove --select from CLI
It needs reconsideration, we can't release it as is right now. E.g. it
expects xpath to start with `$`, but that will get interpreted by your
shell, which is bad.
* completion: update plots
2020-05-21 17:57:27 +03:00
Shankho Boron Ghosh and Restyled.io
19eaced304
dvc : renamed --outs-only to --dvc-only ( #3797 )
...
* Renamed --outs-only to --dvc-only
* Changed outs_only to dvc_only
* replace output_only with dvc_only
* Restyled by black
Co-authored-by: Restyled.io <[email protected]
2020-05-20 18:24:16 +03:00
Ruslan Kuprieiev
cec4b4e003
dvc: rename plot to plots ( #3802 )
...
* dvc: rename plot to plots
Follows `dvc params/metrics` convention and is needed in preparation for new
commands and some refactoring.
Part of #3409
* completion: rename plot -> plots
2020-05-14 22:28:22 +03:00
Ruslan Kuprieiev
974cf097dd
run/repro: rename "build cache" -> "run cache" ( #3788 )
...
Part of #3777
2020-05-14 02:27:18 +03:00
Saugat Pachhai
cc16b28a26
diff: autocompletion script for params, and show-md ( #3786 )
2020-05-12 16:41:09 +03:00
Paweł Redzyński
af8ac006ad
plot: completion for zsh ( #3733 )
2020-05-04 15:01:29 +03:00
Paweł Redzyński and Ruslan Kuprieiev
e553511085
Metrics - plotting for multiple revisions initial ( #3577 )
...
* init
* rename to plot data insertion basig on dicts update
* revision support
* roll back revision
* plot makedirs for backward compatibility
* log path
* pretty plot link to visualization page
* make target default title
* efiop review
* efiop review
* plot multiple initial
* add some missing metric file tests
* proper id generation
* proper id generation
* add confusion matrix template
* refactor tests
* plot from dvct file
* plot from dvct
* brush up commands
* fix confusion matrix multiple plot
* plot: change confusion matrix data schema
* should be working as intended
* support for src file in dvct files
* minor fixes
* plot: support json templates
* plot: rename confusion template
* plot: polish command behaviour
* fix test for json
* plot: test command
* some minor fixes for tests
* plot: unit test loading
* plot: unit test loading
* plot: handle TODOS
* cleanup
* use mocker
* plot: support tsv
* plot: command refactoring
* plot: fix windows issues with tests
* plot: test: some more windows fixes
* plot: _load_from_revisions complexity fix
* plot: reduce complexity
* plot: complexity reduction
* plot: deepsource suggestions
* plot: move template path evaluation
* fixup
* fixup
* exception on no datafile and no template
* json metric load with OrderedDict
* plot: improve handling non-existing files on revisions
* plot: improve handling non-existing files on revisions
* change default plot path
* some exceptions and fixes
* add yaml metrics support
* fixup
* some more suggestions
* default filename fix
* efiop review requests
* log exception on failur
* move revisions deduction to commands
* json templates
* extract template filling to separate method
* some parsing improvements
* add columns functionality
* extract default data transformation to separate method
* plot: initial support for jsonpath
* plot: rename columns to filters, tests are dict based
* plot: fixups
* plot: refactoring
* repo: plot: convert to package
* plot: data loading refactor, support searching for data
* plot: raise if wrong fields provided
* plot: command description
* plot: default: pass y axis info for default plot
* plot: get rid of fieldnames, expect ordered data
* plot: handle default plot in separate method
* plot: fix default
* plot: command option names fixes
* refactoring
* fixes
* plot: provide option for stdout redirection
* plot: rename show-json to no-html
* plot: add no-csv-header option
* plot: improve error message for wrongly structured metric
* plot: match template name exactly, whit suffix appended only
* plot: dmpetrov and ivan review
* plot: refactor --stdout help message
* plot: move template to repo/plot
* plot: add -x and -y options
* plot: add -x and -y options
* plot: command: order change
* plot: scatter
* plot: rename confusion matrix template, new name generation format
* plot: add title anchor
* plot: review from jorgeorpinel
* plot: rename filter and result options to select and file
* plot: add --title, --x-title, --y-title
* plot: xlab ylab
* Update dvc/repo/plot/template.py
Co-authored-by: Ruslan Kuprieiev <[email protected] [email protected] [email protected]
2020-05-01 22:45:03 +03:00
Ruslan Kuprieiev
a7b77672e6
innosetup: rename exe app name
...
Fixes https://github.com/iterative/dvc/issues/3687
2020-04-28 16:08:17 +03:00
Ruslan Kuprieiev
Helio MachadoandPeter Rowlands
Asif Saif UddinandRuslan Kuprieiev
Saugat Pachhai
Clément WalterandRuslan Kuprieiev
Casper da Costa-Luis
Alexander Schepanovski
karajan1001
Shankho Boron GhoshandRestyled.io
Paweł Redzyński