diff --git a/README.rst b/README.rst index ef53728e9..0f51228e7 100644 --- a/README.rst +++ b/README.rst @@ -182,6 +182,12 @@ Fedora / CentOS (rpm) Comparison to related technologies ================================== +#. *Data Engineering* tools such as `AirFlow `_, `Luigi `_ and + others - in DVC data, model and ML pipelines represent a single ML project and focuses on data scientist' experience while + data engineering pipeline orchestrates multiple data projects and focuses on efficent execution. DVC project can be used + from data pipeline as a single execution step. `AirFlow DVC `_ is an example + of such integration. + #. `Git-annex `_ - DVC uses the idea of storing the content of large files (which should not be in a Git repository) in a local key-value store, and uses file hardlinks/symlinks instead of copying/duplicating files. @@ -195,7 +201,8 @@ Comparison to related technologies #. `Workflow Management Systems `_ - DVC is a workflow management system designed specifically to manage machine learning experiments. DVC is built on top of Git. -#. `DAGsHub `_ - This is a Github equivalent for DVC. Pushing Git+DVC based repositories to DAGsHub will produce in a high level project dashboard; including DVC pipelines and metrics visualizations, as well as links to any DVC-managed files present in cloud storage. +#. `DAGsHub `_ - online service of Git+DVC repositories with pipeline and metrics visualization and + DVC-specific cloud storage. Contributing ============