Commit Graph
749 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 5941ace270 dvc: initial 'dvc pkg' implementation
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-17 18:12:36 +03:00
Ruslan Kuprieiev ade4261cf5 Merge pull request #2133 from Suor/push-pull-error-code
dvc: return non zero error code on push/pull/fetch error
2019-06-14 20:43:02 +03:00
Alexander Schepanovski db75693fe1 dvc: return non zero error code on push/pull/fetch error
Closes #2038
2019-06-14 23:25:39 +07:00
Ruslan Kuprieiev 0d330c1b9b dvc: move Updater out of Repo()
Fixes #2039

Required by #2012 to be able to fetch data through the API without
interuption from updater.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-14 00:54:36 +03:00
Ruslan Kuprieiev b1eebe2f86 diff: don't use self
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-14 00:20:56 +03:00
Ruslan Kuprieiev 16e162941e system: handle PurePath
------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/efiop/git/dvc/dvc/remote/local/__init__.py", line 280, in upload
    no_progress_bar=no_progress_bar,
  File "/Users/efiop/git/dvc/dvc/utils/__init__.py", line 129, in copyfile
    System.reflink(src, dest)
  File "/Users/efiop/git/dvc/dvc/system.py", line 125, in reflink
    ret = System._reflink_darwin(source, link_name)
  File "/Users/efiop/git/dvc/dvc/system.py", line 89, in _reflink_darwin
    ctypes.c_char_p(src.encode("utf-8")),
AttributeError: 'PosixPathInfo' object has no attribute 'encode'
------------------------------------------------------------

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-13 11:51:14 +03:00
Aman Sharma 11bafe4b7d Check whether package is installed from source or binary 2019-06-13 10:38:37 +05:30
pawel 974bc39ae5 scm: git: end gitignore with newline 2019-06-12 16:52:36 +02:00
Ruslan Kuprieiev 39a8cab49d test: fix s3 test with extra args
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-11 16:28:11 +03:00
Alexander Schepanovski d80293c44d dvc: path info refactoring
Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.

Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous

Both families might had caused undefined surprising behavior when local
and remote paths clash.
2019-06-11 19:19:17 +07:00
pawel fe3f83ee05 remote: local: test: refactor unpacked dir tests 2019-06-11 10:21:57 +02:00
pawel ac5ca4656a remote: progress callback for unpacked dir creation 2019-06-10 20:19:47 +02:00
pawel cfbd381d42 remote: create unpacked dir after status check if needed 2019-06-10 14:44:03 +02:00
pawel b5464cbc59 remote: local: check if status verification dir already exists 2019-06-10 14:44:03 +02:00
pawel 5d25d5ce96 remote: local: refactor link method 2019-06-10 14:44:03 +02:00
pared 90ca5bd35a remote: local: add cache status check hardlink optimization 2019-06-10 14:44:03 +02:00
Ruslan Kuprieiev f69836c2b3 tests: remove fixme msg
https://github.com/iterative/dvc/issues/2086

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-06 14:41:02 +03:00
Ruslan Kuprieiev 9e9e078de3 config: move logic from CLI to API
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-06 14:41:02 +03:00
Ruslan Kuprieiev f41dcea248 tests: don't use var before assignment
```
repo_dir = <tests.basic_env.TestDirFixture object at 0x7f36c3f3d650>                  
git = <git.Repo "/tmp/dvc-test.2894.0LMoaT.eefefa48-172d-44e4-8c3d-4f65f76c78ee/.git">
                                                                                      
    @pytest.fixture                                                                   
    def dvc_repo(repo_dir, git):                                                      
        try:                                                                          
            dvc = DvcRepo.init(repo_dir._root_dir)                                    
            dvc.scm.commit("init dvc")                                                
            yield dvc                                                                 
        finally:                                                                      
>           dvc.scm.git.close()                                                       
E           UnboundLocalError: local variable 'dvc' referenced before assignment      
                                                                                      
/home/efiop/git/dvc/tests/conftest.py:76: UnboundLocalError                           
```
2019-06-06 13:44:13 +03:00
Andrew Grigorev 5fce9a01e4 tests: fix for pytest>=4.6.0 caplog 2019-06-02 15:41:21 +03:00
Aman Sharma 688bcaa0e3 Add info for link type and filesystem name (#1997)
* Add info for link and filesystem type

* Remove unnecessary `repo_dir` fixture

* Add cache info

* Classify `test_version.py` as functional test

* Remove redundancy

* Optimize some functions involved

* Add pathlib2 to install_requires

* Display fs type for cache directory

* Use `dvc.utils.compat` to import Path

* Remove redundancy in tests and make logs more descriptive
2019-06-01 14:40:59 +03:00
Ruslan Kuprieiev 0935cfae2a tests: show how long tests took 2019-05-31 11:19:18 +03:00
Mr. Outis e3b3f3f371 progress: bug fix with --quiet
Related to #1377
2019-05-29 16:45:10 -07:00
Ruslan Kuprieiev 31848d5c7f remote: support remote:// notation for push/pull/gc
Fixes #2066

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-29 12:54:31 +03:00
Aman Sharma 55ad81bf9d Coalesce requirements.txt into setup.py (#2017)
* Coalesce `requirements.txt` and `tests/requirements.txt` into `setup.py`

* Remove requirements for tests

* Parse tests requirements in setup.py

* Append extra dependenices in test_requirements in setup.py

* Modify CI build to pass the tests

* Correctly install black requirements

* Install all requirements to pass the tests
2019-05-29 10:23:01 +03:00
pawel fae55c2ba2 remote: local: roll back cache dir metadata storing 2019-05-28 13:39:48 -07:00
Ruslan Kuprieiev 29fd8e2e41 Merge pull request #2030 from prihoda/recursive-repro
Add repro -R, support dirs & stage files when running pull/push/repro…
2019-05-27 23:09:02 +03:00
David Příhoda d072c2bd79 Fix after renaming dvc fixture to dvc_repo. 2019-05-27 14:14:44 +02:00
Alexander Schepanovski ca93dbcf13 dvc: add get_url()/open() API
As a part of this:
- DataCloud._cloud was refactored into `.get_remote()`
- data stopped using internal _cloud
- added a parametrization for remote tests
2019-05-27 17:56:31 +07:00
David Příhoda e11c3cbbc5 Add repro -R, support dirs & stage files when running pull/push/repro -R. 2019-05-27 12:48:46 +02:00
Alexander Schepanovski 7ddbbbc80f test: simplify google creds handling 2019-05-25 17:55:23 +07:00
Jorge OrpinelandRuslan Kuprieiev ce83231128 Updates to 1. zsh autocomplete script and 2. slow link type warning message (#2003)
* Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes in dvc.scm.git::Git::get_diff_trees

* diff: Changes to dvc.scm.git so `dvc diff` outputs short SHAs

- Docstring for Git class constructor
- Renames self.git for self.repo throughout module (involves several test changes)
- Better doc and logical refactor in Git._get_diff_trees
- `dvc diff` now outputs short SHA strings (Git._get_diff_trees) - hard coded to 7 length str
  (involves some test changes in tests/func/test_diff.py)

For #1902

* Undoes "Renames self.git for self.repo throughout module" introduced in 3d2dc83.

* autocomplete: Brings scripts/completion/dvc.zsh up-to-date

- Reorganizes commands in alphabetical order;
- Updates the command dscriptions according to their current usage (-h) output;
- Adds comments for MISSING command entries

* Update scripts/completion/dvc.zsh

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* Update scripts/completion/dvc.zsh

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* Update scripts/completion/dvc.zsh

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* Update scripts/completion/dvc.zsh

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* autocomplete: Adds missing commands to Zsh script

* cache: updates slow link type warning message (and test)

in accordance with https://github.com/iterative/dvc.org/pull/331

* Fixes accidental docstring rewrap

per https://github.com/iterative/dvc/pull/2003#discussion_r286206870

* Actually fixes accidental docstring rewrap...

per https://github.com/iterative/dvc/pull/2003#pullrequestreview-240994592

* Really fixes accidental docstring rewrap this time...
2019-05-25 00:00:47 +03:00
Ruslan Kuprieiev 8a4e5f937b Merge pull request #2034 from pared/rename_dvc_fixture
[RFC] tests: rename dvc repo fixture
2019-05-24 03:14:46 +03:00
Tadas Audinis ee6a0b2875 tests: resolve deprecation warnings (#2028)
* tests: resolve deprecation warnings

Fixes #2021

* tests: resolve deprecation warnings

Fixes #2021
2019-05-24 03:12:36 +03:00
TomHortons e3de8aac65 pipeline: show: dot: print dot 2019-05-22 11:11:23 +09:00
pawel 01c2f6c174 tests: rename dvc repo fixture 2019-05-21 15:04:43 -07:00
Ruslan Kuprieiev 3e461f8f11 Merge pull request #2007 from Suor/test-remotes
test: update/add helpers to run tests against remotes
2019-05-21 15:44:23 +03:00
Alexander Schepanovski 0e1f20f0f5 test: update/add helpers to run tests against remotes 2019-05-21 17:38:53 +07:00
prd 1cb4e5233e path: local: str return full path on value error 2019-05-14 16:56:24 -07:00
Ruslan Kuprieiev 1d5d608c21 test: make sure checksums are not recomputed on checkout
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-13 21:07:12 +03:00
Ruslan Kuprieiev b4143b19dd remote: unify class names
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-12 17:29:38 +03:00
Ruslan Kuprieiev a593a9ce2e dvc: refactor PathInfos
Following same conventions from remotes/outsputs/dependencies.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-12 17:18:46 +03:00
Ruslan Kuprieiev d4ef461fce version: use CmdBaseNoRepo
Currently `dvc version` outside of dvc repo raises "not dvc repo" error,
even though it is not really using repo in any way.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-11 17:20:30 +03:00
pared 3ff10005fa path_info: remove dict path_info, introduce PathInfo class 2019-05-10 22:25:50 -07:00
Mr. Outis c39c2f087a install: append commands if hook already exists
Close #1964
2019-05-10 16:20:01 -07:00
Ruslan Kuprieiev 45f94e36b2 Merge pull request #1963 from algomaster99/dvc_version
Add `dvc version` command
2019-05-10 22:20:53 +03:00
Aman Sharma 46325808b8 Change regex pattern and move test under unit/command 2019-05-11 18:29:03 +05:30
Ruslan Kuprieiev 1322a5b2d8 Merge pull request #1978 from mroutis/fix-1973
logger: fix bug that ignored the progress bar
2019-05-10 21:25:15 +03:00
Jorge Orpinel b09bd0426e Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes for dvc diff (#1906)
* Document dvc.scm.git::Git::_get_diff_trees and shorten commit hashes in dvc.scm.git::Git::get_diff_trees

* diff: Changes to dvc.scm.git so `dvc diff` outputs short SHAs

- Docstring for Git class constructor
- Renames self.git for self.repo throughout module (involves several test changes)
- Better doc and logical refactor in Git._get_diff_trees
- `dvc diff` now outputs short SHA strings (Git._get_diff_trees) - hard coded to 7 length str
  (involves some test changes in tests/func/test_diff.py)

For #1902

* Undoes "Renames self.git for self.repo throughout module" introduced in 3d2dc83.

* diff: `dvc diff` uses code for `git rev_parse --short` instead of hard `[:7]` (for short commit hashes).

- Tests updated too.

Per https://github.com/iterative/dvc/pull/1906#discussion_r278510340

* tests: Reformatted with `black tests/func/test_diff.py`

* tests: Remove uneccessary `str()` casting form `dvc diff` tests (and reformat)
2019-05-10 15:30:36 +03:00
Aman Sharma db07f3a08d Modify tests to use pytest fixture 2019-05-10 13:00:42 +05:30