Commit Graph
5562 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 73e60a8c13 Delete pre-commit.yaml 2020-06-15 23:35:10 +03:00
Ruslan Kuprieiev 9d1dde4051 Create pre-commit.yaml 2020-06-15 23:32:39 +03:00
Ruslan Kuprieiev 38dc0f85d4 dvc: bump to 1.0.0b3 1.0.0b3 2020-06-15 22:15:57 +03:00
Paweł Redzyński d2dbe9649c plots: add smooth template (#4046) 2020-06-15 22:15:27 +03:00
Ruslan Kuprieiev ece6f2d3ee dvc: bump to 1.0.0b2 1.0.0b2 2020-06-15 21:36:05 +03:00
Ruslan Kuprieiev b672622324 plots: make sure template is using data anchor (#4045)
Fixes #3969
2020-06-15 21:14:07 +03:00
Ruslan Kuprieiev c9391df313 plots: modify: handle --unset for nonexistent options (#4044)
* tests: plots: reorginize the structure

* plots: modify: handle --unset for nonexistent options

Fixes #3933
2020-06-15 20:31:08 +03:00
Peter Rowlands (변기호) a0c229cf9b remote: continue remote/cache/tree refactoring (#4041)
* remote: move constants into trees

* remote: replace Remote() Cache() helper functions

* output: update for remote changes

* cache/data_cloud: update for remote changes

* dependency: update for remote changes

* stage: update for remote changes

* remote: move remaining non-index methods into tree

* tests: update remote unit tests

* remote: LocalCache needs to lock remote index, not cache index

* fix bugs from moved PARAM_CHECKSUM/PARAM_RELPATH

* tests: repo tree open needs state context

* remote: make Remote.gc a class method

* operates on obj.tree, so that GC can be done with both remote and
  cache instances

* bug fixes

* tests: update test_data_cloud

* bug fixes

* tests: update func tests
2020-06-15 16:22:55 +03:00
karajan1001 a59f90fac9 Performance improvement of dvcignore (#3967)
fix#3869
1.Use big regex.

* Solve windows

* add rule order test

* Solve ignore order

* remove list comprehensions
2020-06-15 11:37:01 +02:00
Ruslan Kuprieiev bba502314d dvc: bump to 1.0.0b1 1.0.0b1 2020-06-14 20:44:44 +03:00
Saugat Pachhai bc4bf66d17 (bugfix) get/import: fix broken progress bar (#4039)
PR #3672 (6d8499e) extended `LocalRemote::_get_plans` to return one
`checksums` too. As all of the args from `_get_plans` was passed
down to `download()`, it recognized extra arg of checksum as
`no_progress_bar` due to which it became True and stopped showing
progress bar at all.

Fix #3874
2020-06-14 20:42:53 +03:00
f7348af1e0 metrics & plots: help output improvements (#3963)
* metrics & plots: help output improvements

* metrics & plots: ui: updated help messages

* refactor: removed redundant parentheses

* metrics: ui: updated 'targets' description in 'metrics show'

* metrics: ui: reordered params to match the order in cmd ref

* plots: ui: params modified to match cmd ref

* plots: ui: updated diff help

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

* Update dvc/command/plots.py

Co-authored-by: Jorge Orpinel <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Jorge Orpinel <[email protected]>
2020-06-14 20:39:33 +03:00
Casper da Costa-Luis 4d140a9944 gdrive: add open (#3916)
* gdrive: add open

Fixes #3408
Related #2865
Fixes #3897

* dependency: add gdrive

* test: api: open: gdrive

* rollback pydrive dependency

* Revert "dependency: add gdrive"

This reverts commit fd333261d2e9fe1364cc278138c79890a1f5fe7a.

* tests: remotes: GDrive: fix get_url

* tests: api: fully test GDrive

* minor typo fix

* attempt gdrive tests fix

* fix gdrive credentials config

* replace GDrivePathNotFound => FileMissingError

* fix test_open_external[GDrive]

* add ensure_dir_scm

* minor exception fix
2020-06-13 23:53:25 +03:00
Casper da Costa-Luis 2ba73daedc snap: make future major version upgrades easier (#4038)
Related: #3999
Related: #4037
2020-06-13 23:27:03 +03:00
Ruslan Kuprieiev 7a31a35cbf dvc: bump to 1.0.0b0 1.0.0b0 2020-06-13 00:56:14 +03:00
Saugat Pachhai 24a14563cc run: preserve order when -f/--force overwrite stage (#4035)
This commit preserves comments, meta and order inside a dvc.yaml
when `-f/--force` is used. But, it does not maintain comments/meta
for .dvc files.

Related: #4014
2020-06-13 00:55:15 +03:00
Ruslan Kuprieiev a1d3440f85 metrics: diff: introduce --precision (#4026)
Rounding to 5 digits by default.

Fixes #3904
2020-06-13 00:54:42 +03:00
Saugat Pachhai 45dcb7c27d Update dag.py (#4036) 2020-06-12 20:33:19 +03:00
Saugat Pachhai 9ff6fef53a bugfix: get dump folder contents in respective output folder (#4034)
The relative path for external git directory was computed relative to
the root of the repo itself, which created a relative path which is
current working directory. So, all of the contents of the erepo to download
was dumped in the pwd rather than inside of it's own directory.

Fix #4033
2020-06-12 20:31:18 +03:00
Saugat PachhaiandRuslan Kuprieiev 9e7dc2640e schema: provide a jsonschema for dvc.yaml (#4028)
* schema: provide a jsonschema for dvc.yaml

* Update scripts/schema/dvc-yaml.json

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update scripts/schema/dvc-yaml.json

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-12 20:09:59 +05:45
Paweł Redzyński 584c97a58d plots: fix handling of multiple plots from single revision (#4031) 2020-06-12 16:08:08 +03:00
Saugat Pachhai b94a18cfb9 tests: tests for dvcfile (#4032) 2020-06-12 15:54:05 +03:00
Peter Rowlands (변기호) 9ead641c95 remote: separate cloud remote and cloud cache classes (#4019)
* remote: move get_file_checksum() into tree

* makes RemoteTree and RepoTree consistent with regard to checksum
  calculation

* remote: save() now takes explicit tree parameter

* when tree is remote.tree, save will be a move + link operation
  (same as default existing behavior)
* when saving path from a different tree, save will be a copy operation

* tests: update for moved remote/tree functions

* remote: move get_checksum into tree

* remote: separate cloud remote and cache classes

* tests: update unit tests for remote/cache separation

* remote: cloud cache should extend cloud remote

* remote: move LocalRemote.get

* tests: update func tests

* dependency: update for moved get_checksum

* remote: fix state lookup bug
2020-06-12 15:52:47 +03:00
Peter Rowlands (변기호) 652f5ab439 remote: use separate working tree instance for local cache (#3991)
* dvcignore: ignore paths outside of CleanTree root

* remote: local remote/cache should always use its own working tree

* repo tree (CleanTree) instance should not be relevant to local cache
  operations. When the dvcignore CleanTree is relevant, repo.tree should
  be used. For local remote/cache operations, the local remote work_tree
  should be used.

* state: tie state to local cache working tree

* tests: update tests

* state: use os.path.exists instead of WorkingTree.exists in some cases
2020-06-12 12:52:38 +03:00
bjonnhandJonathan Bisson 28a123e727 Remove a remnant umask to solve #4021 (#4022)
Co-authored-by: Jonathan Bisson <[email protected]>
2020-06-11 19:47:25 +03:00
Ruslan Kuprieiev 566472f088 dvc: bump to 1.0.0a11 1.0.0a11 2020-06-11 19:09:51 +03:00
Saugat PachhaiandRuslan Kuprieiev 2e12a31c5a tests: cover run cache with functional tests (#4018)
* tests: cover run cache with functional tests

* run-cache: use _log_exceptions wrapper

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-06-11 18:44:21 +03:00
Alexander Schepanovski d7788887f0 plots: fix a couple of bugs (#4017)
- running .plots.collect() not from repo root used to skip plot files
- make Plots.render() be more convenient to be called from outside
2020-06-11 18:20:12 +07:00
Ruslan Kuprieiev 87276faef2 RepoTree: work with dirty repos (#4005)
Currently `RepoTree.open` will throw FileNotFoundError if you have a
file that is tracked by dvc, but which has no md5 or a cache for which
is missing, even though you have the file itself in your workspace.

This corresponds very nicely to the `--no-commit` or `--no-exec`
scenario, where you might not want to commit your artifacts to cache
just yet.

Fixes #3974
2020-06-11 09:50:27 +03:00
Saugat Pachhai 4042d5232d run: use -f/--force to overwrite, fix msg (#4011)
Also, makes fixes for `--overwrite` rename to `--force`
and also remove `remove_with_prompt` which was only being
used with `force\=True` anyway.

Fix #4003
2020-06-10 20:17:27 +03:00
Saugat Pachhai fda14c82d3 plots modify: do not convert output types to plot on unset (#4009)
Fix #4006
2020-06-10 15:31:17 +03:00
Saugat Pachhai 64be262d18 dvc.yaml: preserve meta (#4008) 2020-06-10 14:37:21 +03:00
Saugat Pachhai 1d6c80001d plots modify: complain if template does not exist (#3995) 2020-06-10 16:12:56 +05:45
Ruslan Kuprieiev 91a7194e4f add/run/import-url: don't use -f as --file (#4004)
If anything, `-f` is closer to `--force` than to `--file`. So let's set
this straight.
2020-06-10 05:04:55 +03:00
Ruslan Kuprieiev afe539aaba plots: rename --no-csv-header to --no-header (#4001)
* plots: rename --no-csv-header to --no-header

Fixes #3996

* fix typo
2020-06-10 03:34:25 +03:00
Ruslan Kuprieiev f259bc52cd config/remote/cache: make --system/--global/--local mutually exclusive (#4000) 2020-06-10 02:32:18 +03:00
Ruslan Kuprieiev db69def834 status: introduce --show-json (#3998)
Fixes #1406
2020-06-10 01:51:30 +03:00
Casper da Costa-Luis 5efc3bbee2 snap: build fixes (#3990)
* snap: drop sudo

Fixes #3989

* snap: ci: fix builds

* snap: ci: developer notifications

* snap: minimise size

* debug commit

* Revert "snap: minimise size"

This reverts commit 77b41a96f68b2c8df833803d0a644172b7db4779.

* fix image info

* Revert "debug commit"

This reverts commit 232baf0ad78335b0f3806456a3f20b82cdf0f24c.

* snap: v1 channels

Related: #3872
2020-06-10 00:00:15 +03:00
Alexander Schepanovski 7230d87137 Refactor plots (#3994)
- separate data collection and rendering
- provide a way to use custom template loader
- drop hard-coding datafiles in template feature
2020-06-09 22:47:17 +03:00
Paweł Redzyński 56833387ce plots: find metric data in yaml dictionary (#3992) 2020-06-09 16:42:28 +03:00
Saugat Pachhai 593e67318b do not consider stage cached if checksum/cmd has changed (#3993) 2020-06-09 18:46:49 +05:45
Saugat Pachhai afc183f4e7 tests serializing of stage to lockfile (#3988)
* tests: serialize stage for lockfile

* config: make isort/black compatible

* convert class based tests to pytest
2020-06-09 15:48:17 +03:00
Saugat Pachhai d201fad1e3 tests: serialize pipeline file (#3985)
serialize: make params ordered
2020-06-09 14:22:55 +03:00
Ruslan Kuprieiev 5be12b5aac plots: fix help message (#3983)
Per https://github.com/iterative/dvc/pull/3911#discussion_r436238442
2020-06-09 14:15:15 +03:00
Saugat Pachhai c1e5b10cbd remove unnecessary dependencies (#3987)
Fix #3971
2020-06-09 12:43:34 +03:00
Saugat Pachhai b47b2c5673 tests: update mock-ssh-server (#3986) 2020-06-09 12:39:10 +03:00
Ruslan Kuprieiev 939f78948a remote: tree: don't ignore exceptions (#3977)
Moving ignore logic to the remote itself, so that tree could behave
properly.
2020-06-09 11:25:13 +03:00
Saugat Pachhai 215ea4944c log: show modifying if entry already exists in dvcfile (#3978) 2020-06-08 18:56:00 +03:00
Saugat Pachhai 4e12e76e19 tests: unit testing for StageLoader for Pipelinefile (#3976) 2020-06-08 18:55:16 +03:00
Saugat Pachhai 33092e8614 issue template: do not auto-add labels on new issues (#3979) 2020-06-08 19:47:42 +05:45