karajan1001
76b31ac282
fix #3588 ( #3770 )
...
* fix #3588
remove improper edges from output DAG.
* less code modification
* deepsource required
* deepsource required
* refactoring for reducing cognitive complexity.
2020-05-09 21:00:46 +03:00
Casper da Costa-Luis
2bb7257a06
lint: isort pre-commit & misc improvements ( #3749 )
...
* test: clean flake8 config root clutter
* test: pre-commit: add isort
Compatible with black (see https://github.com/timothycrosley/isort/issues/694 )
* lint: reorder imports
* lint: more complex imports
* lint: fix deepsource
* config: isort: add first party dvc
* tests: restyled: add isort
* temp commit to revert before merge
* Revert "temp commit to revert before merge"
This reverts commit 11db0fd8e95fe891597cbb9bf804480909b19e24.
2020-05-06 23:56:07 +03:00
Ruslan Kuprieiev
bc7f177c7a
run: use multistage dvcfiles by default ( #3740 )
...
Adding hidden `--single-stage` to simplify the test migration.
2020-05-04 19:55:39 +03:00
Ruslan Kuprieiev
f9088e1c67
dvc: rename pipelines.yaml -> dvc.yaml ( #3710 )
2020-04-30 18:19:51 +03:00
Saugat Pachhai and Ruslan Kuprieiev
1937527249
dvc: implement multistage dvcfile ( #3676 )
...
* dvc: use lockfile as yaml instead of json
* refactor Dvcfile into Pipeline file and Single stage file
* fix tests
* dvc: fix outputs
* add more tests for collection of outputs
* add tests for data cloud/get/import/ls
* tests: test for checkouts
* Allow other checksums other than md5
* tests: use iterdir instead of os.listdir
* cleanup errors reported by cc and ds
* utils: throw DvcException instead of plain Exception
* tests: use yaml.load instead of json.load
* run: split assignments
* Update dvc/stage/exceptions.py
Co-authored-by: Ruslan Kuprieiev <[email protected] >
2020-04-28 13:36:05 +03:00
Saugat Pachhai
128fa7f887
dvc: add a support for multistage dvcfile. ( #3584 )
...
Now, pipeline stage files can house multiple stages, and separate lockfiles
are created which has the checksums, whereas Dvcfile will be clean and human
readable and editable. The *.dvc files will be generated for output files.
It is available via a hidden flag: -n.
Fixes #1871
PR: #3584
2020-04-17 22:58:36 +03:00
Ruslan Kuprieiev
7edee656da
pipeline: fix --outs output ( #3507 )
...
Fixes #3497
2020-03-18 23:03:41 +02:00
Ruslan Kuprieiev
06c634adee
pipelines: fix edges gathering ( #3421 )
2020-03-05 14:16:57 +02:00
Fábio Santos
758205a52a
PR feedback
2020-02-03 18:45:44 +00:00
Fábio Santos
11ad0a9803
remove __
2020-02-03 18:41:55 +00:00
Fábio Santos
a7bad88159
remove unnecessary chdir
2020-01-28 11:49:24 +00:00
Fábio Santos
233790ced5
add test and traverse nodes when showing outs
2020-01-23 17:54:00 +00:00
Paweł Redzyński
945eb181bc
pipeline: test: migrate to dir helpers
2020-01-08 11:23:43 +01:00
Mr. Outis
fd82402d34
py3: use new magic from PEP 3135
2020-01-02 16:24:31 -06:00
Ruslan Kuprieiev
95fe1578bb
dvc: automatically reorder imports
2019-11-05 02:34:40 +02:00
Ruslan Kuprieiev
381601f4ee
dvc: cache graph/stages/pipelines ( #2529 )
...
* dvc: cache graph/stages/pipelines
Signed-off-by: Ruslan Kuprieiev <[email protected] >
* repo: check_dag: don't reset the cache
https://github.com/iterative/dvc/pull/2529#discussion_r327711928
Signed-off-by: Ruslan Kuprieiev <[email protected] >
* repo: rename check_dag(stages) -> check_modified_graph(new_stages)
https://github.com/iterative/dvc/pull/2529#discussion_r327708439
Signed-off-by: Ruslan Kuprieiev <[email protected] >
* repo: refactor "active" logic and remove it from Repo class
* repo: use tree.setter to automate cache reset
As noted by @Suor's , this is much less prone to errors.
* repo: move graph-related staticmethods into graph.py
As per @Suor's request.
* dvc: remove unused from_directory
https://github.com/iterative/dvc/pull/2529#discussion_r328079258
* repo: use _tree instead of __tree
https://github.com/iterative/dvc/pull/2529#discussion_r328071115
* repo: add a note about cache invalidation on repo.lock release
https://github.com/iterative/dvc/pull/2529#discussion_r328071635
* repo: rename reset() -> _reset()
https://github.com/iterative/dvc/pull/2529#discussion_r328079899
* pipeline: use graph helpers
https://github.com/iterative/dvc/pull/2529#discussion_r327502293
* repo: use list comprehension for collect()
https://github.com/iterative/dvc/pull/2529#discussion_r328076651
* repo: graph: only use first cycle
https://github.com/iterative/dvc/pull/2529#discussion_r328081346
* dvc: don't use G.nodes()
https://github.com/iterative/dvc/pull/2529#discussion_r328089671
https://github.com/iterative/dvc/pull/2529#discussion_r328089878
2019-09-26 07:42:29 +03:00
Ruslan Kuprieiev
918ef646dc
appveyor: test py2 and py3
...
Related to #2166
Signed-off-by: Ruslan Kuprieiev <[email protected] >
2019-06-23 13:33:47 +03:00
TomHortons
e3de8aac65
pipeline: show: dot: print dot
2019-05-22 11:11:23 +09:00
Ruslan Kuprieiev
6f11f1f51f
test: move func tests to tests/func
...
Signed-off-by: Ruslan Kuprieiev <[email protected] >
2019-04-16 20:19:22 +03:00