git: use dulwich for .gitignore (#4809)

In my tests `Git._ignore` is taking around ~65seconds for the whole test
suit(~50K invocations). The reason is that `gitpython` is launching a real
`git check-ignore` process, that takes time and resources.

Switching to `pygit2` or `dulwich` makes it drop to ~5seconds, saving us
almost a minute of the test run time and a lot of resources by not
launching any processes. I've chosen `dulwich` over `pygit2` here
because it is much more pythonic and as long as the performance is
on-par with `pygit2` it is better for us to use `dulwich`.

dvc/scm/git.py

@@ -183,13 +183,12 @@ class Git(Base):
         return entry, gitignore
 
     def _ignored(self, path):
-        from git.exc import GitCommandError
+        from dulwich import ignore
+        from dulwich.repo import Repo
 
-        try:
-            self.repo.git.check_ignore(path)
-            return True
-        except GitCommandError:
-            return False
+        repo = Repo(self.root_dir)
+        manager = ignore.IgnoreFilterManager.from_repo(repo)
+        return manager.is_ignored(relpath(path, self.root_dir))
 
     def ignore(self, path):
         entry, gitignore = self._get_gitignore(path)

setup.py

@@ -50,6 +50,7 @@ install_requires = [
     "colorama>=0.3.9",
     "configobj>=5.0.6",
     "gitpython>3",
+    "dulwich>=0.20.6",
     "setuptools>=34.0.0",
     "nanotime>=0.5.2",
     "pyasn1>=0.4.1",