Commit Graph
34 Commits
Author SHA1 Message Date
Peter Rowlands (변기호) 686898240c Revert "repro/exp run: use git status when reminding to git-track files (#5544)" (#5596)
This reverts commit ce2d7c65db.
2021-03-12 11:50:13 +09:00
Peter Rowlands (변기호) d68f6d3696 pygit/dulwich: implement stash functionality (#5286)
* pygit2: disable set_ref with reflog message

* git: implement is_dirty in dulwich

* pygit2: implement _stash_apply

* pygit: update to 1.5.0

* enable set_ref(message=...) for pygit2

* git: replace backend reflog_delete in favor of _stash_drop

* pygit: support stash_push with custom refs

* update tests for backend stash_drop behavior

* pygit: raise MergeConflictError on failed stash apply

* pygit: reload index from disk before stash apply

* update stash_apply checkout strategy

* mark flaky test
2021-03-08 17:24:35 +02:00
Peter Rowlands (변기호) ce2d7c65db repro/exp run: use git status when reminding to git-track files (#5544)
* repro: skip DVC deps in SCM track stage

(DVC deps will be tracked when their own stage is repro'd)

* scm: use git status when reminding to git-track files

* sort names in reminder instead of random set order

* update unit test
2021-03-04 17:48:12 +09:00
Peter Rowlands (변기호) 9909ee85a1 git: implement dulwich add --update (#5491)
* dulwich: implement add with --update

* add git backend test for add()

* fix list comprehension in dulwich status()

* allow adding single string path instead of list

* handle windows paths in add(update=...)
2021-02-18 17:56:56 +09:00
Ruslan Kuprieiev b0fd40f12c dvc: rename tree -> fs (#5461) 2021-02-13 15:40:44 +02:00
RogerandPeter Rowlands e97f8fe19c pygit2, gitpython: handle relpaths in reset/checkout_index backends (#5414)
* Make minor change that fixes things for some reason

While working on vscode-dvc, both mattseddon and I found that this change allows
us to run `dvc exp run`, and we cannot otherwise.

I don't exactly know what this change does, and I'm hoping someone more
knowledgable does- after which we can give this PR a better name.

* test relpaths in git backend funcs which take path lists

* gitpython: handle relpaths in reset and checkout_index

* use / for git relpaths on windows

Co-authored-by: Peter Rowlands <[email protected]>
2021-02-05 12:05:51 +09:00
Saugat Pachhai 9f0b86f221 unit.scm.test_git: use pytest instead of unittest (#5325) 2021-01-25 10:23:46 +05:45
Saugat Pachhai 3741f730e0 show git-add hint on stage:add, refactor scm_context (#5290)
* Refactor scm_context, show git-add hint on stage:add

* move tests
2021-01-24 22:43:38 +02:00
Peter Rowlands (변기호) 407f5607a2 scm: implement pygit2 resolve rev and native checkout (#5270)
* pygit2: implement get/set/remove refs

* pygit: implement iter_refs

* pygit: implement resolve_rev

* pygit2: implement working tree checkout

* pygit2: implement reset, explicitly read/write index when used

* pygit2: checkout does not need to reset index

* pygit2: handle "-" branch shorthand in checkout

* tests: install post-checkout hook should be run via gitpython

* exp apply: use stricter exception handling for merge conflict

* pygit: fix index read/write usage

* gitpython: handle untracked file conflicts in stash apply

* fix flaky/race index issue
2021-01-18 17:41:25 +02:00
Peter Rowlands (변기호) 1ab1abf62a git: implement merge/checkout via pygit2 (#5245)
* git: implement merge (gitpython, pygit2)

* pylint: add pygit2 as C extension

* git: s/checkout_paths/checkout_index/ and add pygit implementation

* git: support basic checkout/merge conflict strategies in checkout_index

* fix relpath issue in pygit merge

* git: add stubs for add --update

* experiments: drop remaining gitpython specific scm usage
2021-01-12 13:26:27 +02:00
Peter Rowlands (변기호) 7a92ff5e5d experiments: commit with --no-verify in executors (#5078)
* git: add no_verify flag for commit()

* experiments: commit with --no-verify in executors

* git: wrap hook exceptions with SCMError

* update hook file perms

* update dulwich to 0.20.15

* update pygit2 commit def

* skip hook tests on windows
2020-12-28 17:35:35 +09:00
devramx 0bb3f40be8 remove: delete .gitignore file if empty (#5148)
* remove: delete .gitignore file if empty

modified behavior to unlink empty .gitignore files.
added corresponding unit & func test cases.
updated failing test cases.

Fixes #4962

* fix failing lint checks

* integrating feedback from the code review
2020-12-23 23:04:19 +02:00
Peter Rowlands (변기호) 7bf9a0e805 experiments: support experiments cache collection for exp push/pull/gc (#5064)
* experiments: add utils.exp_commits for iterating over experiment commits

* repo: add support for all_experiments in brancher and used_cache

* git: make all_commits return commits in standard refs instead of --all

* gc: add --all-experiments to preserve cache for experiments

* update gc tests

* repo: support pushing and fetching specific revs internally

* exp push: push cache for experiment by default

* exp pull: pull cache for experiment by default

* fix empty git repo bug
2020-12-09 22:35:06 +09:00
Peter Rowlands (변기호) 51fa0f5c38 experiments: add dvc exp push/pull/list (#5042)
* git: add support for on_diverged in push_refspec

* experiments: update executor for push-refspec changes

* experiments: implement push()

* tests: add tests for experiments.push

* git: add iter_remote_refs

* roughly equivalent to `git ls-remote --refs`

* experiments: add experiments.list for listing local and remote
experiments

* refactor git upstream/downstream in tests

* experiments: add experiments.pull

* experiments: add CLI commands for push/pull/list

* exp list: drop short options

* experiments: s/list/ls/

* dulwich: catch not repo errors when creating remote client

* fix remote rev resolution for list

* fix command test

* add tests for push/pull with ambiguous experiment names
2020-12-08 21:20:03 +09:00
Peter Rowlands (변기호) 821ec0a934 git: drop scm requirement from backends (#5049)
* git: remove scm from backends, make backends configurable

* git: implement get/set/remove ref in gitpython

* git: support reflog --rewrite for stash

* tests: support testing individual backends in unit/scm/test_git

* stash: create reflog if it does not exist for gitpython backend
2020-12-08 20:37:16 +09:00
Ruslan Kuprieiev 5152866634 scm: move logic to gitpython backend (#5022) 2020-12-03 20:25:20 +02:00
Peter Rowlands (변기호) 1e52bc94a3 git: add support for working with refspecs and stashes (#4952)
* git: add support for dulwich Stash

* git: start adding support for custom refs

* git: add Stash and helper context managers for preserving
workspace state

* scm: cleanup refs/stash and add push_refspec

* tests: add tests for git refs and stash

* fix push_refspec

* tests: add tests for git push_refspec

* scm: add git fetch-refspecs

* add test for fetch refspecs

* scm: add test for git refs containing

* remove extraneous log msgs

* fix typing issues

* stash: only try to push if workspace is dirty

* update dulwich to 0.20.14

* scm: move dulwich into scm.backend

* move stash internals to backends

* scm: move git code into dvc/scm/git/... module

* git: use ABC for BaseGitBackend

* fix rebase issue

* git: fix refspec tests on windows

- git file:// URLs need posix style paths

* git: move get_refs_containing to backend

* scm: add callback for handling diverged refs during fetch
2020-12-01 16:56:50 +09:00
Peter Rowlands (변기호) 3d58f4b8ce git: move branch iteration functionality from experiments into scm.git (#4810)
* git: move branch iteration functionality from experiments into scm.git

* add branch_revs unit test
2020-10-30 19:06:35 +09:00
Robert LowryandRuslan Kuprieiev 15488ebefd scm: add config option to stage files automatically (#4543)
* scm: add "autostage" config option to stage files automatically

Fixes #4330

* Update tests/unit/scm/test_scm.py

* Update tests/unit/scm/test_scm.py

* scm_context, git: call reset_tracked_files from scm_context, remove output message

* Update dvc/repo/scm_context.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-09-21 11:59:25 +03:00
Saugat Pachhai c4bc4c83e8 upgrade to isort5 (#4399)
* Imports are no longer moved to top.
* `seed-isort-config` is no longer needed.
* Has a `black`-compatible profile/config.
* Also `isorts` the imports anywhere in the code
  (not just at the top).

Also bumps `restyled-isort` to newest version
2020-08-17 19:59:40 +05:45
Peter Rowlands (변기호) f516dd252a trees: handle tree.walk errors the same way as os.walk (#4059)
* trees: BaseTree.walk should handle errors like os.walk

* use onerror callback if exceptions should be logged or raised

* repo.ls: use onerror for tree.walk
2020-06-17 13:25:14 +03:00
Ruslan Kuprieiev 597ab98d0e diff: hanle repo with no commits (#3784)
`git diff` doesn't throw errors, so we could do the same.

Fixes #3398
2020-05-14 02:14:37 +03:00
Peter Rowlands (변기호) 833e82a7ea Drop Python 3.5 support (#3792)
* drop python 3.5 support

* dvc: remove fspath() use inside os.path calls

* dvc: replace dvc.compat.fspath with os.fspath

* tests: replace dvc.compat.fspath with os.fspath

* utils: relpath() should use fspath/str

* utils: os.readlink doesn't support path-like obj on Windows until 3.8
2020-05-14 01:46:42 +03:00
Casper da Costa-Luis 2bb7257a06 lint: isort pre-commit & misc improvements (#3749)
* test: clean flake8 config root clutter

* test: pre-commit: add isort

Compatible with black (see https://github.com/timothycrosley/isort/issues/694)

* lint: reorder imports

* lint: more complex imports

* lint: fix deepsource

* config: isort: add first party dvc

* tests: restyled: add isort

* temp commit to revert before merge

* Revert "temp commit to revert before merge"

This reverts commit 11db0fd8e95fe891597cbb9bf804480909b19e24.
2020-05-06 23:56:07 +03:00
Ruslan Kuprieiev e2981d2743 git: is_tracked: use ls_files instead of walking the index (#3677)
Current implementation doesn't work with directories, which results in
hard-to-debug bugs like #3561. The bug that made us walk through index
manually is now fixed in gitpython, so we can simply use `ls_files` from
now on. Added some tests to prevent this from happening in the future.

Fixes #3561
2020-04-24 19:18:58 +03:00
Ramón Valles ee26afe65e git: quote filenames @ remind_to_track (#3315)
Fix #3303
2020-03-23 18:22:51 +02:00
Ruslan Kuprieiev eae432818a git: tree: use item.path instead of item.name (#3505)
We are currently using this [0] hack to work around the fact that
item.name
is not always a basename. Even though `IndexObject` has it defined that
way
[1], but `Submodule` - doesn't [2]. So using `item.name` might cause
issues
such as [3], because GitPython doesn't pass name parameter when simply
going through objects [4]. In other places, when working specifically
with
submodules, it sets `_name` attribute explicitly [5].

Note that we still don't walk into submodules when walking through a git
repo. Might want to reconsider that later as for now we haven't heard
anything about it from actual users.

Fixes #3481

[0] https://github.com/iterative/dvc/blob/0.90.0/dvc/scm/git/tree.py#L15
[1]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/base.py#L170
[2]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/submodule/base.py#L1123
[3] https://github.com/gitpython-developers/GitPython/issues/597
[4]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/tree.py#L237
[5]
https://github.com/gitpython-developers/GitPython/blob/3.1.0/git/objects/submodule/base.py#L357
2020-03-18 19:18:58 +02:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Ruslan Kuprieiev 918ef646dc appveyor: test py2 and py3
Related to #2166

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-23 13:33:47 +03:00
Andrew Grigorev 7d79043bbb dvc.scm.scm_context -> dvc.repo.scm_context.scm_context
The `dvc/scm/__init__.py:scm_context` works with `Repo` objects and
is used mostly in `dvc/repo/...`, so it is better to put it in
`dvc/repo/scm_context.py`.

Fix #1769
2019-03-22 19:53:41 +03:00
Andrew Grigorev 9e6c52079b Abstract access to file objects
Abstract access to filesystem and its implementation to access files and
particulary Stage objects in different git branches without `git
checkout` execution.
2019-03-21 19:47:42 +03:00
Ivan Shcheklein 75421f0f21 migrate to pytest 2019-03-17 14:49:49 -07:00
pared 4897a6a44f add: prevent adding git files 2019-03-13 14:11:59 +01:00
pared f8711b3faa repo/scm: move tracking reminder from repo to scm 2019-03-11 23:13:34 +01:00