Alexander Schepanovski
7230d87137
Refactor plots ( #3994 )
...
- separate data collection and rendering
- provide a way to use custom template loader
- drop hard-coding datafiles in template feature
2020-06-09 22:47:17 +03:00
Paweł Redzyński
56833387ce
plots: find metric data in yaml dictionary ( #3992 )
2020-06-09 16:42:28 +03:00
Saugat Pachhai
593e67318b
do not consider stage cached if checksum/cmd has changed ( #3993 )
2020-06-09 18:46:49 +05:45
Saugat Pachhai
afc183f4e7
tests serializing of stage to lockfile ( #3988 )
...
* tests: serialize stage for lockfile
* config: make isort/black compatible
* convert class based tests to pytest
2020-06-09 15:48:17 +03:00
Saugat Pachhai
d201fad1e3
tests: serialize pipeline file ( #3985 )
...
serialize: make params ordered
2020-06-09 14:22:55 +03:00
Ruslan Kuprieiev
5be12b5aac
plots: fix help message ( #3983 )
...
Per https://github.com/iterative/dvc/pull/3911#discussion_r436238442
2020-06-09 14:15:15 +03:00
Saugat Pachhai
c1e5b10cbd
remove unnecessary dependencies ( #3987 )
...
Fix #3971
2020-06-09 12:43:34 +03:00
Saugat Pachhai
b47b2c5673
tests: update mock-ssh-server ( #3986 )
2020-06-09 12:39:10 +03:00
Ruslan Kuprieiev
939f78948a
remote: tree: don't ignore exceptions ( #3977 )
...
Moving ignore logic to the remote itself, so that tree could behave
properly.
2020-06-09 11:25:13 +03:00
Saugat Pachhai
215ea4944c
log: show modifying if entry already exists in dvcfile ( #3978 )
2020-06-08 18:56:00 +03:00
Saugat Pachhai
4e12e76e19
tests: unit testing for StageLoader for Pipelinefile ( #3976 )
2020-06-08 18:55:16 +03:00
Saugat Pachhai
33092e8614
issue template: do not auto-add labels on new issues ( #3979 )
2020-06-08 19:47:42 +05:45
Saugat Pachhai
ae7b604531
Fix loading of params from stage cache ( #3972 )
2020-06-08 12:43:13 +03:00
Ruslan Kuprieiev
b7c174641e
tests: mark s3 test as flaky
2020-06-08 12:27:25 +03:00
Ruslan Kuprieiev
cb43f9cdda
dvc: bump to 1.0.0a10
2020-06-08 11:56:58 +03:00
Ruslan Kuprieiev
791ce22759
pyinstaller: add hook for google-api-python-client ( #3965 )
...
Fixes #3913
2020-06-08 11:53:12 +03:00
Casper da Costa-Luis
c70adebc75
snap: restrict site-package checks ( #3878 )
...
* snap: restrict site-package checks
Fixes #3514
* add comment
2020-06-07 07:50:15 +03:00
2c9cb87679
Subcommand help ( #3956 )
...
* change DvcParser to print subcommand help
* add tests for subcommand help
* docstring refactoring
* remote: finish separating RemoteTree methods (#3946 )
* remote: move upload()/download() into tree
* remote: move path_info/path_cls into tree
* remote.azure: finish moving methods into tree
* remote.gs: finish moving methods into tree
* remote.hdfs: finish moving methods to tree
* remote.http: finish moving methods into tree
* remote.oss: finish moving methods into tree
* remote.s3: finish moving methods into tree
* remote.ssh: finish moving methods into tree
* remote.gdrive: finish moving methods into tree
* tests: update remote unit tests
* tests: update func tests
* remote: use walk_files() for all remotes
* list_cache_paths() now uses tree.walk_files() for all remotes except
local/ssh
* fix DS warnings
* bugfixes
* Unify metrics behavior with dirs (#3935 )
* run: disable directories for -m/-M flags
* run: reformatted string literals
* run: simplified metrics check
* tests: func: run: dirs as metrics outs
* tests: func: run: reformatted string literals
* tests: use pytest
Co-authored-by: Ruslan Kuprieiev <[email protected] #3949 )
* refactor multistage load for params and outputs
* tests: load params
* tests: output loading from pipeline file
* fix test
* fix typo in name
* split params load
* rename params func s/inject_values/fill_values
* fix tests
* simplify loads_params and output.load_from_pipeline
* address @pared's suggestions for tests
* remote: fix build errors (#3957 )
* remote: fix remaining gdrive build issues (#3959 )
* tests: fix gdrive build
* remote.gdrive: fix walk_files prefix relpath
* change DvcParser to print subcommand help
* add tests for subcommand help
* docstring refactoring
* change tests to pytest-style
* remove gettext
Co-authored-by: Peter Rowlands (변기호) <[email protected] [email protected] [email protected] [email protected]
2020-06-05 23:44:02 +03:00
Peter Rowlands (변기호)
b57077af11
remote: fix remaining gdrive build issues ( #3959 )
...
* tests: fix gdrive build
* remote.gdrive: fix walk_files prefix relpath
2020-06-05 15:36:47 +09:00
Peter Rowlands (변기호)
803bb3eb61
remote: fix build errors ( #3957 )
2020-06-05 05:14:34 +03:00
Saugat Pachhai
99b58d3d05
refactor/test multistage load for params and outputs ( #3949 )
...
* refactor multistage load for params and outputs
* tests: load params
* tests: output loading from pipeline file
* fix test
* fix typo in name
* split params load
* rename params func s/inject_values/fill_values
* fix tests
* simplify loads_params and output.load_from_pipeline
* address @pared's suggestions for tests
2020-06-04 23:35:07 +03:00
nik123 and Ruslan Kuprieiev
5bbaa28cae
Unify metrics behavior with dirs ( #3935 )
...
* run: disable directories for -m/-M flags
* run: reformatted string literals
* run: simplified metrics check
* tests: func: run: dirs as metrics outs
* tests: func: run: reformatted string literals
* tests: use pytest
Co-authored-by: Ruslan Kuprieiev <[email protected]
2020-06-04 23:32:00 +03:00
Peter Rowlands (변기호)
74bd8137ba
remote: finish separating RemoteTree methods ( #3946 )
...
* remote: move upload()/download() into tree
* remote: move path_info/path_cls into tree
* remote.azure: finish moving methods into tree
* remote.gs: finish moving methods into tree
* remote.hdfs: finish moving methods to tree
* remote.http: finish moving methods into tree
* remote.oss: finish moving methods into tree
* remote.s3: finish moving methods into tree
* remote.ssh: finish moving methods into tree
* remote.gdrive: finish moving methods into tree
* tests: update remote unit tests
* tests: update func tests
* remote: use walk_files() for all remotes
* list_cache_paths() now uses tree.walk_files() for all remotes except
local/ssh
* fix DS warnings
* bugfixes
2020-06-04 23:22:31 +03:00
Ruslan Kuprieiev
72a576cefe
dvc: bump to 1.0.0a9
2020-06-04 06:06:25 +03:00
Ruslan Kuprieiev
036ce9398e
tests: move mock-ssh-server git dependency ( #3953 )
...
This one doesn't allow us to release new pre-releases, because pypi
forbids that:
https://travis-ci.com/github/iterative/dvc/jobs/343343967#L2447
Let's install it separately for now, until there is a new official
mock-ssh-server release.
2020-06-04 05:54:29 +03:00
Ruslan Kuprieiev
13012f9c0c
plots: completion/spelling fixups ( #3952 )
...
* completion: don't forget --show-vega
Fixes #3890
* plots: VEGA -> Vega
2020-06-04 05:49:20 +03:00
Ruslan Kuprieiev
a65c7989c8
tests: don't cache gdrive url ( #3941 )
...
This makes `GDrive.get_url` behaviour consistent with the rest of the
test remotes.
Related to #3916
2020-06-04 04:01:12 +03:00
Ruslan Kuprieiev
3590fe8414
dvc: bump to 1.0.0a8
2020-06-04 00:10:10 +03:00
Ruslan Kuprieiev
05100122e1
plots: remove print
...
Leftover from https://github.com/iterative/dvc/pull/3948
FIxes #3951
2020-06-04 00:09:16 +03:00
Alexander Schepanovski
9a55b909f8
Rename --xlab to --x-label, X_TITLE to X_LABEL, same for y ( #3948 )
...
Renamed:
- cli options
- dvcfile plot props
- template anchors
- variable names throughout the code
2020-06-03 20:30:17 +03:00
Ruslan Kuprieiev
ff54ac1bfb
plots: fix error when there are no plots in repo ( #3940 )
...
Consistent with `dvc metrics show/diff` behaviour.
Fixes #3925
2020-06-03 08:54:48 +03:00
Ruslan Kuprieiev
57e920aa39
tests: don't forget --external when adding external files
2020-06-03 08:38:51 +03:00
Ruslan Kuprieiev
ac4f351daf
dvc: require --external for external outputs ( #3929 )
...
Fixes #1545
2020-06-03 06:13:09 +03:00
Peter Rowlands (변기호)
2c0116ccf9
remote: separate tree functions from main remote classes ( #3931 )
...
* BaseRemote: start moving tree functions into BaseRemoteTree
* remote: add LocalRemoteTree
* remote: add cloud remote trees
* remote: move remove(), makedirs() into tree
* remote: move move(), copy(), symlink(), hardlink(), reflink() into tree
* fix attributes for moved remote methods
* tests: update unit tests for moved remote tree methods
* tests: mv unit/remote/test_remote_dir.py unit/remote/test_remote_tree.py
* fix attributes for moved remote tree functions
* tests: update func tests for moved remote tree methods
* test fixes
* fix DS warnings
2020-06-03 06:04:25 +03:00
Ruslan Kuprieiev
c03ce4aa0f
dag: fix help
...
per https://github.com/iterative/dvc.org/pull/1383#discussion_r433397776
2020-06-03 04:25:15 +03:00
Ruslan Kuprieiev
72dcbc0a9e
remote: modify: fix --unset ( #3939 )
...
https://discordapp.com/channels/485586884165107732/563406153334128681/717474697721413686
Fixes https://github.com/iterative/dvc/issues/3938
2020-06-03 02:09:08 +03:00
Alexander Schepanovski
fd65f44b7f
Fix quadratic behavior in Repo._collect_stages() ( #3932 )
2020-06-02 16:56:49 +03:00
Ruslan Kuprieiev
7504b68c12
dvc: bump to 1.0.0a7
2020-06-02 03:34:04 +03:00
Ruslan Kuprieiev
86f3457dec
dvc: remove dvc metrics add/remove ( #3928 )
...
We will be re-introducing `metrics modify` in the future and
will likely revisit `add/remove` too, so it is better to get
rid of these for now and introduce later with a clear scneario
in mind.
Fixes #3849
Fixes #3907
2020-06-02 00:17:03 +03:00
Saugat Pachhai
219662597b
refactor: rename utils.stage to utils.yaml ( #3923 )
2020-06-01 16:49:46 +03:00
Alexander Schepanovski
536966f9d1
Fix modifying plot ( #3926 )
...
- no error when modifying plot with no props yet
- do not turn into non-plot if all props are unset
2020-06-01 16:19:11 +03:00
Saugat Pachhai
076eadceba
bugfix: rerun stage when command is changed ( #3922 )
...
Previously, the stage would not rerun if the deps/outs are cached
even if the cmd has changed.
Versions affected: >0.93.0
Fixes #3918
2020-06-01 14:04:31 +05:45
Ruslan Kuprieiev
d1e08b6157
dvc: replace pipeline with dag ( #3905 )
...
`pipeline list` is no more, it had some very questionable applications
(none to be precise) and was duplicating old `pipeline show` functionality.
We might introduce a different command to assist in listing (e.g. for
shell completion) in the future.
New `dvc dag` replaces `dvc pipeline show` and by-default will show
all pipelines in the project in ASCII (so there is no more --ascii)
and also `--dot` support, just like before. When given a target,
`dvc dag` (in any mode) will show DAG that leads to the target,
skipping the descendants, unless `--full` is specified.
2020-06-01 02:35:36 +03:00
Saugat Pachhai
edfa1741db
implement removing stage ( #3881 )
...
This should allow us to be able to remove a stage from a dvcfile,
(if it's a single stage file, it removes it), and make us able to
delete the overwrite the stage on run as well.
This commit:
* changes `run` `--overwrite-dvcfile` flag to `--overwrite`
* removes prompt in `run` for overwrite and fails hard
* changes `remove`: now it by default removes stage entry and unprotect outs
* `remove` has `--outs` to remove outputs instead of unprotecting them
2020-05-31 17:41:19 +03:00
Ruslan Kuprieiev
9ee62ef8f0
dvc: rename "working tree" to "workspace" ( #3914 )
...
Fixes #3892
2020-05-31 17:39:04 +03:00
Ruslan Kuprieiev
0bb49d2549
dvc: rename dvc lock -> dvc freeze ( #3915 )
...
Fixes #3901
2020-05-31 03:38:21 +03:00
Alexander Schepanovski and Ruslan Kuprieiev
516d82e411
Add plots props and plots modify params ( #3911 )
...
* Add plots props and `plots modify` params
* Update dvc/command/plots.py
Co-authored-by: Ruslan Kuprieiev <[email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
2020-05-30 14:37:11 +03:00
Saugat Pachhai
ec312d8601
use fork of mockssh-server ( #3910 )
2020-05-30 08:31:59 +05:45
Ruslan Kuprieiev
f504245f35
pre-commit: don't run dvc in parallel ( #3912 )
...
* pre-commit: use local hooks
* pre-commit: don't run dvc in parallel
By default pre-commit will try to run hooks in parallel, which results
in some dvc instances failing to acquire repo locks.
Fixes #3751
2020-05-29 21:20:46 +03:00
Saugat Pachhai
bb50fb8664
use higher level API selectors instead of select ( #3899 )
2020-05-29 09:25:07 +05:45
Alexander Schepanovski
Paweł Redzyński
Saugat Pachhai
Ruslan Kuprieiev
Casper da Costa-Luis
