Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-10 07:18:27 +01:00
Code Issues Packages Projects Releases Wiki Activity
5,427 Commits 7 Branches 579 Tags
aaca1401eab6fc81e07281c6c7683ff1577ea2ed
Commit Graph
5427 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
nik123 aaca1401ea Fixes #3814 (#3828) 2020-05-20 04:14:04 +03:00
Saugat Pachhai 2aaf065ea3 default target: reduce loglevel to debug (#3822) 2020-05-19 16:35:13 +03:00
Saugat Pachhai d4ec871a76 repro: do not log when stage is locked and is repo import (#3823) 2020-05-19 16:23:28 +03:00
Paweł Redzyński f44ddcf64e plot: dont zero y axis (#3813) 2020-05-19 16:20:21 +03:00
Peter Rowlands (변기호) b88b3b81c6 utils: support use of (GitTree) tree file objects in utils (#3819) 
* GitTree: add stat() function

* BaseTree: make walk_files a base tree method

* WorkingTree: add stat() method (wrapper for os.stat)

* utils.fs: support non-working tree for get_mtime_and_size

* utils.fs: add copy_fobj_to_file

* utils: support using tree.open in file_md5
 2020-05-19 16:20:04 +03:00
Ruslan Kuprieiev 81d04f887a stage: fix commit (#3816) 2020-05-19 03:30:52 +03:00
Peter Rowlands (변기호) afbb3a0add repo: Support streaming and pulling files on RepoTree/DvcTree.open() (#3810) 
* DvcTree: support stream or fetch on open()

* tests: test DvcTree dir cache support for walk()

* RepoTree: finish implementation of BaseTree functions

* repo: use RepoTree/DvcTree.open() for repo.open_by_relpath

* repo: only use RepoTree when needed for now

* fix docstring

* RepoTree: only use DvcTree.open for DVC outs
 2020-05-18 15:16:11 +03:00
Saugat Pachhai b8aa8ad951 pyupgrade: 3.6+ (#3805) 
* pyupgrade: 3.6+

* tests: pyupgrade 3.6+
 2020-05-15 16:48:02 +03:00
Saugat Pachhai 9897f7c470 test: metric/cache/persist keys (#3785) 2020-05-15 13:58:50 +03:00
Saugat Pachhai cf7f125c14 refactor: simplify checkouts and run, get rid of check_can_commit (#3800) 
* stage: split checkouts

* stage: simplify can check commit

* split shell executable expansion to separate function

* refactor changed_entries for better message on pipeline file

* fix commit msg entry for stage

* reduce complexity of run_stage

* simplify prompt for commit

* fixes

* refactor commit

* refactor run_stage

* commit refactor
 2020-05-15 10:08:32 +05:45
Ruslan Kuprieiev cec4b4e003 dvc: rename plot to plots (#3802) 
* dvc: rename plot to plots

Follows `dvc params/metrics` convention and is needed in preparation for new
commands and some refactoring.

Part of #3409

* completion: rename plot -> plots
 2020-05-14 22:28:22 +03:00
Ruslan Kuprieiev ff4fcc937b stage: fix --dry-run (#3799) 
* tests: unit: move stage tests into a separate dir

* stage: fix --dry-run

Part of #3777
 2020-05-14 20:19:34 +05:45
Saugat Pachhai 1676a7a1cd stage: moving things around, refactor (#3793) 
* stage: extract unnecessary methods out of Stage class

* stage: change naming of functions

* stage: extract dump equal to utils

* stage: extract computation of md5 to utils

* stage: extract command run related logic in run.py

* stage: reorganize run()

* stage: boolean cosmetic changes

* stage: reorder save() and extract some into func

* stage: refactor status() into smaller functions

* extract resolve_wdir into utils

* merge _changed() and changed()

* cosmetic changes

* extract dump to utils

* extract import updates/sync to imports.py

* remove meta from PipelineStage

* fix imports, params load

* make run_stage a function, fix mocks in tests

* use original cmd

* rollback imports on decorators

* fix ds and cc
 2020-05-14 16:43:02 +03:00
Ruslan Kuprieiev 974cf097dd run/repro: rename "build cache" -> "run cache" (#3788) 
Part of #3777
 2020-05-14 02:27:18 +03:00
Ruslan Kuprieiev 597ab98d0e diff: hanle repo with no commits (#3784) 
`git diff` doesn't throw errors, so we could do the same.

Fixes #3398
 2020-05-14 02:14:37 +03:00
Peter Rowlands (변기호) 833e82a7ea Drop Python 3.5 support (#3792) 
* drop python 3.5 support

* dvc: remove fspath() use inside os.path calls

* dvc: replace dvc.compat.fspath with os.fspath

* tests: replace dvc.compat.fspath with os.fspath

* utils: relpath() should use fspath/str

* utils: os.readlink doesn't support path-like obj on Windows until 3.8
 2020-05-14 01:46:42 +03:00
Peter Rowlands (변기호) 3916d4fa07 tests: mark test_data_cloud::test_pull_git_imports as flaky (#3790) 2020-05-14 00:16:37 +09:00
Paweł Redzyński d81b56978f plot: add doctype at the beggining of html embedding (#3787) 2020-05-13 00:17:05 +03:00
Jorge Orpinel 1e7efffe42 plot: update some output strings (#3759) 
* plot: update datafile desc.

* plot: link to cmd ref in output

* plot: update --stdout desc
per https://github.com/iterative/dvc/pull/3577#pullrequestreview-407036181

* plot: a few more output string updates
per https://github.com/iterative/dvc/pull/3759#pullrequestreview-409011291
and https://github.com/iterative/dvc.org/pull/1283
 2020-05-12 16:52:06 +03:00
Saugat Pachhai cc16b28a26 diff: autocompletion script for params, and show-md (#3786) 2020-05-12 16:41:09 +03:00
Paweł Redzyński 451c7aac04 tests: erepo_dir: extract remote setup to separate method (#3748) 2020-05-12 02:05:22 +03:00
Ruslan Kuprieiev 7b4466c982 run-cache: support uncached files (#3778) 
Currently we only restore `--outs`, but `--outs-no-cache` are also
useful, especially when they are metrics files
(`-M|--metrics-no-cache`). So this PR makes run-cache save uncached
files to our cache and also tries to checkout them back during the
`restore` phase.
 2020-05-12 02:02:53 +03:00
Jack WellsandRuslan Kuprieiev 7ca3a1119c Added S3 SSEKMSKeyId config option (#3762) 
* Added S3 SSEKMSKeyId config option

* test: unit: s3: add SSEKMSKeyId test

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-05-11 22:52:55 +03:00
Saugat Pachhai 2c026066db pipeline file: disallow punctuation characters in stage name (#3767) 
except hyphen and underscore
 2020-05-11 19:11:47 +03:00
nik123 7f17c6aa03 refactor: check if targets is instances of str (#3779) 2020-05-11 19:10:14 +03:00
Ned Letcher 9f015fac1d import: check erepo is None before modifying url - fixes #3780 (#3781) 
* import: check erepo is None before modifying url - fixes #3780

* import: add test to cover this local import
 2020-05-11 18:29:35 +03:00
Saugat PachhaiandRuslan Kuprieiev 666e203bbb params/metrics: diff: implement tabulated markdown output (--show-md) (#3757) 
* params/metrics: diff: implement tabulated markdown output

Via `--show-md` on `diff`s.

* table: show None on missing val

* tests: use `dedent`, add tests for markdown

* diff: add --show-md

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-05-10 18:39:51 +03:00
Karthikeyan Singaravelan ce946d1160 Import ABC from collections.abc for Python 3 compatibility. (#3772) 2020-05-10 10:49:32 +05:45
karajan1001 76b31ac282 fix #3588 (#3770) 
* fix #3588

remove improper edges from output DAG.

* less code modification

* deepsource required

* deepsource required

* refactoring for reducing cognitive complexity.
 2020-05-09 21:00:46 +03:00
karajan1001karajan1001Ruslan Kuprieiev
 		6093a1fe97 Dvc remote default in list validation (#3715) 
* fix #3470

Add validation to restrict default remote repo in list of remote repos.

* fix #3470

1. add two tests
2. add one validation
3. modified remote remove to satithe validation change

* Pass DeepSource

* deepsource required

* Redesigned.

* fix typo

* Revert "deepsource required"

This reverts commit 1ae458c34f49e8d7b35c39c1e6c592b423493a97.

* Revert "Pass DeepSource"

This reverts commit b254d104df65295e0a1156f3979b365be9d38a3d.

Co-authored-by: karajan1001 <[email protected]>
Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2020-05-09 20:59:17 +03:00
Ruslan Kuprieiev d62a54c7b0 push/pull: properly collect run cache (#3768) 
Fixes #1234
 2020-05-08 23:34:35 +03:00
nik123 c922da4f3f update: --recursive flag (#3760) 
* update: --recursive flag

* refactor: applied isort changes to dvc/repo/update.py

* update: removed default values, check for str

* update: fixed code climate cognitive complexity issues
 2020-05-08 23:23:48 +03:00
Ruslan Kuprieiev 2a16b021ad stage: hide unwanted warnings (#3763) 
These messages all correpond to expected behaviour, so there is no need
to polute the output with unnecessary warnings.
 2020-05-08 15:52:09 +03:00
Ruslan Kuprieiev fc42ca721c dvc: bump to 1.0.0a1 1.0.0a1 2020-05-07 19:33:30 +03:00
Ruslan Kuprieiev 785dcfeef0 push/pull: naively transfer run cache (#3746) 
This is the most simple (aka dumb) implementation for this
functionality that is needed to unblock CICD development.
 2020-05-07 16:55:10 +03:00
Paweł Redzyński 9714b14456 plot: fix help messages (#3734) 2020-05-07 13:22:55 +03:00
Ivan Shcheklein 6a406abde8 trying to enable the sponsorship button 2020-05-06 21:09:45 -07:00
Ivan Shcheklein 7ca055e868 fix: bump PyDrive2 to fix Gdrive remote download 2020-05-06 20:11:33 -07:00
Ruslan Kuprieiev 6a64f86178 pre-commit: use isort seeder 2020-05-07 00:25:41 +03:00
Ruslan Kuprieiev 227dbf44d9 isort: fix config 2020-05-07 00:17:38 +03:00
Casper da Costa-Luis 2bb7257a06 lint: isort pre-commit & misc improvements (#3749) 
* test: clean flake8 config root clutter

* test: pre-commit: add isort

Compatible with black (see https://github.com/timothycrosley/isort/issues/694)

* lint: reorder imports

* lint: more complex imports

* lint: fix deepsource

* config: isort: add first party dvc

* tests: restyled: add isort

* temp commit to revert before merge

* Revert "temp commit to revert before merge"

This reverts commit 11db0fd8e95fe891597cbb9bf804480909b19e24.
 2020-05-06 23:56:07 +03:00
Saugat Pachhai e3831066de repro: use dvc.yaml by default (#3747) 
Note: This does not change default for `pipeline` as
it cannot show multiple DAGs at the same time. If we
are able to show multiple DAGs, we can just default to
`dvc.yaml`.
 2020-05-06 18:33:42 +03:00
Paweł Redzyński aa99cc9934 plot: check field exists (#3742) 2020-05-06 18:33:10 +03:00
Casper da Costa-Luis 26e203242f gdrive: download: stream & add progress (#3722) 
* wip

* gdrive: add progress

Part of #2865
See https://github.com/iterative/dvc/issues/2865#issuecomment-622984785

* gdrive: move towards next pydrive2 release

- depends on https://github.com/iterative/PyDrive2/pull/30

* update to latest pydrive>=1.4.11

* avoid unneeded API call

* progress: gdrive: ensure proper bar_format
 2020-05-06 17:02:16 +03:00
Saugat Pachhai d00bd4dfe5 pull/fetch/push: add stats (#3735) 
* pull/fetch/push: add stats

* fix typo: s/logging/logger

* pull: make stats detailed, with summary at the bottom

* tests: mock properly, set return_value

* add a line space
 2020-05-05 16:10:50 +03:00
Casper da Costa-Luis 795f001929 Snap version (#3741) 
* snap: potential build fix

* snap: fix travis
 2020-05-05 12:48:19 +03:00
Ruslan Kuprieiev 93d7b9173b tests: use --single-stage 1.0.0a0 2020-05-04 20:23:49 +03:00
Ruslan Kuprieiev bc7f177c7a run: use multistage dvcfiles by default (#3740) 
Adding hidden `--single-stage` to simplify the test migration.
 2020-05-04 19:55:39 +03:00
Ruslan Kuprieiev 6f73a67c60 dvc: bump to 1.0.0a0 2020-05-04 19:46:49 +03:00
Ruslan Kuprieiev 68a0c39ad1 run: remove deprecated -y|--yes (#3737) 2020-05-04 17:51:07 +03:00