mirror of
https://github.com/Stone-Red-Code/dvc.git
synced 2026-11-02 07:18:25 +01:00
129 lines
5.3 KiB
ReStructuredText
129 lines
5.3 KiB
ReStructuredText
.. raw:: html
|
|
|
|
<p align="center"><img src="https://dvc.org/static/img/logo-github-readme.png" height="161px" alt="DVC"></p>
|
|
|
|
.. image:: https://img.shields.io/travis/iterative/dvc/master.svg?label=Linux%20%26%20Mac%20OS
|
|
:target: https://travis-ci.org/iterative/dvc
|
|
|
|
.. image:: https://img.shields.io/appveyor/ci/iterative/dvc/master.svg?label=Windows
|
|
:target: https://ci.appveyor.com/project/iterative/dvc/branch/master
|
|
|
|
.. image:: https://codeclimate.com/github/iterative/dvc/badges/gpa.svg
|
|
:target: https://codeclimate.com/github/iterative/dvc
|
|
|
|
.. image:: https://codecov.io/gh/iterative/dvc/branch/master/graph/badge.svg
|
|
:target: https://codecov.io/gh/iterative/dvc
|
|
|
|
Data Science Version Control or DVC is an open source tool for data science projects.
|
|
It helps data scientists manage their code and data together in a simple form of Git-like commands.
|
|
|
|
Get started
|
|
===========
|
|
+-----------------------------------+-------------------------------------------------------------------+
|
|
| Step | Command |
|
|
+===================================+===================================================================+
|
|
| Track code and data together | | ``$ git add train.py`` |
|
|
| | | ``$ dvc add images.zip`` |
|
|
+-----------------------------------+-------------------------------------------------------------------+
|
|
| Connect code and data by commands | | ``$ dvc run -d images.zip -o images/ unzip -q images.zip`` |
|
|
| | | ``$ dvc run -d images/ -d train.py -o model.p python train.py`` |
|
|
+-----------------------------------+-------------------------------------------------------------------+
|
|
| Make changes and reproduce | | ``$ vi train.py`` |
|
|
| | | ``$ dvc repro model.p.dvc`` |
|
|
+-----------------------------------+-------------------------------------------------------------------+
|
|
| Share code | | ``$ git add .`` |
|
|
| | | ``$ git commit -m 'The baseline model'`` |
|
|
| | | ``$ git push`` |
|
|
+-----------------------------------+-------------------------------------------------------------------+
|
|
| Share data and ML models | | ``$ dvc remote add myremote s3://mybucket/image_cnn`` |
|
|
| | | ``$ dvc core.remote myremote`` |
|
|
| | | ``$ dvc push`` |
|
|
+-----------------------------------+-------------------------------------------------------------------+
|
|
|
|
See more in `tutorial <https://dvc.org/doc/get-started>`_.
|
|
|
|
Installation
|
|
============
|
|
|
|
Packages
|
|
--------
|
|
|
|
Operating system dependent packages are the recommended way to install DVC.
|
|
The latest version of the packages can be found at GitHub releases page: https://github.com/iterative/dvc/releases
|
|
|
|
Python Pip
|
|
----------
|
|
|
|
DVC could be installed via the Python Package Index (PyPI).
|
|
|
|
.. code-block:: bash
|
|
|
|
pip install dvc
|
|
|
|
Homebrew (Mac OS)
|
|
-----------------
|
|
|
|
Formula:
|
|
^^^^^^^^
|
|
|
|
.. code-block:: bash
|
|
|
|
brew install iterative/homebrew-dvc/dvc
|
|
|
|
Cask:
|
|
^^^^^
|
|
|
|
.. code-block:: bash
|
|
|
|
brew cask install iterative/homebrew-dvc/dvc
|
|
|
|
Links
|
|
=====
|
|
|
|
Website: https://dvc.org
|
|
|
|
Documentation: http://dvc.org/doc
|
|
|
|
Tutorial: https://dvc.org/doc/tutorial
|
|
|
|
Discussion: https://discuss.dvc.org/
|
|
|
|
Related technologies
|
|
====================
|
|
|
|
#. `Git-annex <https://git-annex.branchable.com/>`_ - DVC uses the idea of storing the content of large files (that you
|
|
don't want to see in your Git repository) in a local key-value store and uses file hardlinks/symlinks instead of the
|
|
copying actual files.
|
|
#. `Git-LFS <https://git-lfs.github.com/>`_. - DVC is compatible with any remote storage (S3, Google Cloud, Azure, SSH,
|
|
etc). DVC utilizes reflinks or hardlinks to avoid copy operation on checkouts which makes much more efficient for
|
|
large data files.
|
|
#. Makefile (and it's analogues). DVC tracks dependencies (DAG).
|
|
#. `Workflow Management Systems <https://en.wikipedia.org/wiki/Workflow_management_system>`_. DVC is a workflow
|
|
management system designed specifically to manage machine learning experiments. DVC is built on top of Git.
|
|
|
|
DVC is compatible with Git for storing code and the dependency graph (DAG), but not data files cache.
|
|
To store and share data files cache DVC supports remotes - any cloud (S3, Azure, Google Cloud, etc) or any on-premise
|
|
network storage (via SSH. for example).
|
|
|
|
How DVC works
|
|
=============
|
|
|
|
|
|
.. image:: https://dvc.org/static/img/flow.png
|
|
:target: https://dvc.org/static/img/flow.png
|
|
:alt: how_dvc_works
|
|
|
|
|
|
Contributing
|
|
=========
|
|
Contributions are welcome! Please see our `Contributing Guide <https://dvc.org/doc/user-guide/contributing/>`_ for more
|
|
details.
|
|
|
|
Copyright
|
|
=========
|
|
|
|
This project is distributed under the Apache license version 2.0 (see the LICENSE file in the project root).
|
|
|
|
By submitting a pull request for this project, you agree to license your contribution under the Apache license version
|
|
2.0 to this project.
|