Commit Graph
61 Commits
Author SHA1 Message Date
Ruslan Kuprieiev 07da9f1e5c tests: remotes: use TmpDir-like fixtures (#4140) 2020-07-02 04:46:40 +03:00
Saugat Pachhai 047352027f pylint: enable attribute-outside-init check (#4119)
* pylint: enable attribute-outside-init check

* fix tests

* Fix mock for time

This was an interesting one. I was mocking `time` on function definition
and then, using those later on test run. When all the tests were run, there
would be some delay that'd make tests fail. Hmmm ...
2020-06-26 17:22:38 +03:00
Saugat Pachhai ad306d7bf3 Enable most of the pylint checks (#4111) 2020-06-25 18:10:14 +03:00
Saugat PachhaiandRuslan Kuprieiev 2fecc66cb9 addressing: allow stage name addressing without ":" (#3842)
* addressing: allow stage name addressing without colon

* repo: reorganize imports

* refactor/reorganize collect_granular

* add tests for stage addressing

* fix ds and cc issues

* fix unpack error in test

* tests: stage string representations

* exception: add error message

Also remove `cause` when throwing NoOutputOrStage, as
CheckoutErrorSuggestGit will start chaining all the messages
in cause of the exception.

* rename NoOutputOrStage NoOutputOrStageError

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-05-26 03:51:51 +03:00
Peter Rowlands (변기호) b88b3b81c6 utils: support use of (GitTree) tree file objects in utils (#3819)
* GitTree: add stat() function

* BaseTree: make walk_files a base tree method

* WorkingTree: add stat() method (wrapper for os.stat)

* utils.fs: support non-working tree for get_mtime_and_size

* utils.fs: add copy_fobj_to_file

* utils: support using tree.open in file_md5
2020-05-19 16:20:04 +03:00
Saugat Pachhai b8aa8ad951 pyupgrade: 3.6+ (#3805)
* pyupgrade: 3.6+

* tests: pyupgrade 3.6+
2020-05-15 16:48:02 +03:00
Peter Rowlands (변기호) 833e82a7ea Drop Python 3.5 support (#3792)
* drop python 3.5 support

* dvc: remove fspath() use inside os.path calls

* dvc: replace dvc.compat.fspath with os.fspath

* tests: replace dvc.compat.fspath with os.fspath

* utils: relpath() should use fspath/str

* utils: os.readlink doesn't support path-like obj on Windows until 3.8
2020-05-14 01:46:42 +03:00
Casper da Costa-Luis 2bb7257a06 lint: isort pre-commit & misc improvements (#3749)
* test: clean flake8 config root clutter

* test: pre-commit: add isort

Compatible with black (see https://github.com/timothycrosley/isort/issues/694)

* lint: reorder imports

* lint: more complex imports

* lint: fix deepsource

* config: isort: add first party dvc

* tests: restyled: add isort

* temp commit to revert before merge

* Revert "temp commit to revert before merge"

This reverts commit 11db0fd8e95fe891597cbb9bf804480909b19e24.
2020-05-06 23:56:07 +03:00
Saugat Pachhai d00bd4dfe5 pull/fetch/push: add stats (#3735)
* pull/fetch/push: add stats

* fix typo: s/logging/logger

* pull: make stats detailed, with summary at the bottom

* tests: mock properly, set return_value

* add a line space
2020-05-05 16:10:50 +03:00
Ruslan Kuprieiev f9088e1c67 dvc: rename pipelines.yaml -> dvc.yaml (#3710) 2020-04-30 18:19:51 +03:00
Ruslan Kuprieiev 18e8f07572 dvc: introduce local build cache (#3603)
This patch introduces `.dvc/cache/stages` that is used to store previous
runs and their results, which could then be reused later when we stumble
upon the same command with the same deps and outs.

Format of build cache entries is single-line json, which is readable by
humans and might also be used for lock files discussed in #1871.

Related to #1871
Local part of #1234
2020-04-30 00:22:32 +03:00
Saugat Pachhai 907853b985 tag: getting rid of it (#3699)
* tags: get rid of it

* tests: adjust for tags and allow @ in stage name

* disallow @ on stage name

* fix ds issue, leftover tag in Stage
2020-04-29 16:57:31 +03:00
Saugat PachhaiandRuslan Kuprieiev 1937527249 dvc: implement multistage dvcfile (#3676)
* dvc: use lockfile as yaml instead of json

* refactor Dvcfile into Pipeline file and Single stage file

* fix tests

* dvc: fix outputs

* add more tests for collection of outputs

* add tests for data cloud/get/import/ls

* tests: test for checkouts

* Allow other checksums other than md5

* tests: use iterdir instead of os.listdir

* cleanup errors reported by cc and ds

* utils: throw DvcException instead of plain Exception

* tests: use yaml.load instead of json.load

* run: split assignments

* Update dvc/stage/exceptions.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-04-28 13:36:05 +03:00
Saugat Pachhai d2734dfe1e tests: remove use of mocker as context-manager (#3480) 2020-03-16 08:25:22 +05:45
Ruslan Kuprieiev 49039968f7 Merge pull request #3208 from fabiosantoscode/feature/3198-remove-spy-method
tests: replace spy() with pytest-mock spy
2020-01-23 10:52:43 +02:00
Fábio Santos fc92266114 address comments 2020-01-22 17:31:46 +00:00
Fábio Santos bdf665a34a don't use patch.object() 2020-01-21 20:52:48 +00:00
Fábio Santos 5dfcda54f9 tests: replace spy() with pytest-mock spy
Fixes #3198
2020-01-21 14:47:49 +00:00
Paweł Redzyński cf08529362 utils: add tests for resolve out 2020-01-17 09:51:07 +01:00
Ruslan Kuprieiev b865a48b92 Merge pull request #3146 from pared/2896_fs_repo
fs: tests: remove repo_dir
2020-01-14 16:49:23 +02:00
pawel cb5b5ed334 ssh: test: unit, utils: test: unit: remove repo_dir 2020-01-14 15:32:48 +01:00
pawel 144f10d8cb fs: tests: remove repo_dir 2020-01-14 15:25:55 +01:00
Aman Sharma 74850fe728 Move walk_files to fs.py 2020-01-09 14:55:49 +05:30
Aman Sharma a79816920d Move copyfile to fs.py 2020-01-09 04:57:53 +05:30
Aman Sharma eca5d2dde4 Move makedirs to fs.py 2020-01-09 04:30:51 +05:30
Aman Sharma 13ca5bdf28 Write test for walk_files to check if it accepts Path-like and str objects 2020-01-07 05:50:48 +05:30
Aman Sharma f08c90fc6e Ensure relpath accepts str and Path-like objects 2020-01-02 02:17:02 +05:30
Mr. Outis 6d36b85736 py3: remove string compat related code 2019-12-30 23:49:19 -06:00
Mr. Outis 94a0914bba py3: get rid of future imports 2019-12-30 19:35:48 -06:00
pawel cf2d349bd7 tree: move CleanTree to ignore 2019-12-30 14:19:43 +01:00
pawel d8c0ae6909 remove most occurences of dvc_walk 2019-12-30 14:19:43 +01:00
pawel f4ee01cffe no need for dvcignore in walk args 2019-12-30 14:19:43 +01:00
Saugat Pachhai 83c8ab5c26 test: Refactor tests 2019-11-19 21:55:13 +05:45
Saugat Pachhai 68c7366ef0 utils: implement path_isin to compare child and parent paths 2019-11-19 05:34:33 +05:45
Aman Sharma 3edad21b41 Write test for tmp_fname to check if it accepts Path-like and str objects 2019-11-18 11:49:40 +05:30
Aman Sharma 636d850c84 Ensure remove accepts str and Path-like objects 2019-11-15 01:54:40 +05:30
Aman Sharma 9427bbe8d0 Move move to filesystem utility 2019-11-09 23:57:57 +05:30
Aman Sharma 44268251b4 Ensure move accepts str and Path-like objects 2019-11-09 18:47:01 +05:30
Aman Sharma 204cfcb879 Ensure makedirs accepts str and Path-like objects 2019-11-08 20:37:32 +05:30
Aman Sharma d4aa27d6ab Ensure copyfile accepts str and Path-like objects 2019-11-08 01:51:49 +05:30
pawel f14aca630f ignore: make DvcIgnoreFilter use tree 2019-11-07 15:37:24 +01:00
Aman Sharma 34a868b420 Ensure file_md5 accept str and Path-like objects 2019-11-06 09:25:35 +05:30
Ruslan Kuprieiev 5e2fa3742b Merge pull request #2741 from algomaster99/remove-get-parent-dirs
Remove `get_parent_dirs_up_to`
2019-11-05 23:01:31 +02:00
Aman Sharma 8491ddf43f Remove get_parent_dirs_up_to 2019-11-06 02:03:47 +05:30
Aman Sharma f642ff76a5 Ensure contains_symlink_up_to accept both str and Path-like objects 2019-11-06 00:24:10 +05:30
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Aman Sharma d454cb8637 Modify test to check for directory as well 2019-11-03 02:23:46 +05:30
Aman Sharma b9a17752e3 Test acceptable path types of get_mtime_and_size 2019-10-31 00:30:54 +05:30
Aman Sharma 5f644745c7 Write test for get_inode (#2673) 2019-10-28 16:43:47 +02:00
Ruslan Kuprieiev 132de572af run: improve pyenv PATH modification criteria
https://discordapp.com/channels/485586884165107732/485596304961962003/628766062472724510

Related to #2506
2019-10-02 06:33:01 +03:00