Commit Graph
3 Commits
Author SHA1 Message Date
Saugat PachhaiandRuslan Kuprieiev 1937527249 dvc: implement multistage dvcfile (#3676)
* dvc: use lockfile as yaml instead of json

* refactor Dvcfile into Pipeline file and Single stage file

* fix tests

* dvc: fix outputs

* add more tests for collection of outputs

* add tests for data cloud/get/import/ls

* tests: test for checkouts

* Allow other checksums other than md5

* tests: use iterdir instead of os.listdir

* cleanup errors reported by cc and ds

* utils: throw DvcException instead of plain Exception

* tests: use yaml.load instead of json.load

* run: split assignments

* Update dvc/stage/exceptions.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-04-28 13:36:05 +03:00
Saugat Pachhai 63c466d821 refactor: extract stage loading outside of dvcfile (#3655)
* refactor: extract stage loading outside of dvcfile

* repo: simplify stage collection

* fix typing
2020-04-20 23:55:02 +03:00
Saugat Pachhai 128fa7f887 dvc: add a support for multistage dvcfile. (#3584)
Now, pipeline stage files can house multiple stages, and separate lockfiles
are created which has the checksums, whereas Dvcfile will be clean and human
readable and editable. The *.dvc files will be generated for output files.

It is available via a hidden flag: -n.

Fixes #1871
PR: #3584
2020-04-17 22:58:36 +03:00