* pygit2: disable set_ref with reflog message
* git: implement is_dirty in dulwich
* pygit2: implement _stash_apply
* pygit: update to 1.5.0
* enable set_ref(message=...) for pygit2
* git: replace backend reflog_delete in favor of _stash_drop
* pygit: support stash_push with custom refs
* update tests for backend stash_drop behavior
* pygit: raise MergeConflictError on failed stash apply
* pygit: reload index from disk before stash apply
* update stash_apply checkout strategy
* mark flaky test
* gdrive: sync with pydrive's save backend changes
* bump pydrive version
* better temporary credentials managements for service accounts
* proper temp var management
delete is a direct alias to rm(), though apparently it is kind of
broken on the latest release of gcsfs. This doesn't change any behavior
but only uses rm() directly instead of the given alias.
* state: only load when needed
* state: don't use separate load/dump
* checkout: group link removal logic together
* state: use diskcache.Cache
* staging: use pbar and save to state
* state: don't pass the whole repo to __init__
* staging: explicitly pass state object
* fs: get rid of odb property
* state: don't close the connection to cache
* repo: close: close state
When trying to publish to pypi, we get:
```
Invalid value for requires_dist. Error: Can't have direct dependency:
"gdrivefs @ git+https://github.com/intake/gdrivefs.git ; extra ==
'tests'"
```
* git: add support for dulwich Stash
* git: start adding support for custom refs
* git: add Stash and helper context managers for preserving
workspace state
* scm: cleanup refs/stash and add push_refspec
* tests: add tests for git refs and stash
* fix push_refspec
* tests: add tests for git push_refspec
* scm: add git fetch-refspecs
* add test for fetch refspecs
* scm: add test for git refs containing
* remove extraneous log msgs
* fix typing issues
* stash: only try to push if workspace is dirty
* update dulwich to 0.20.14
* scm: move dulwich into scm.backend
* move stash internals to backends
* scm: move git code into dvc/scm/git/... module
* git: use ABC for BaseGitBackend
* fix rebase issue
* git: fix refspec tests on windows
- git file:// URLs need posix style paths
* git: move get_refs_containing to backend
* scm: add callback for handling diverged refs during fetch
* s3: use mocked s3 instead of the real one
Not removing the fixtures just yet
* Update moto
Tests was failing, because of a change in `responses` package, which
`moto` was mocking. This was fixed on
https://github.com/spulec/moto/pull/3466 which is only available on
`dev122` release.
* Use mocked s3 in checkout tests
Via pyparsing. The error message is not that great, as it's greedy.
But, it does point out where the mistakes are, correctly most of the
time. :)
Also raises appropriate error messages for ParseError
* Add support for WebHDFS
* Style and linting fixes
* Implement abstract methods and disable no-member pylint errors
* Add init unit test
* Yield client and fix redefinition of hdfs
* Get dynamic hdfs Docker port
* Add support for open in WebHDFSTree
* Remove hdfscli_config fallbacks in WebHDFSTree
* Use port 50070 for webhdfs
* WebHDFSTree supports http URL and proper walking
* Fix failing unit test after refactoring
* Add overwrite argument to upload
* Add remaining func tests for webhdfs
* Remove print statements
* Allow dynamic attributes in pylint for hdfs library
* Comment error on instantiating HDFS client from config
* Remove depth argument from call to walk
* Update dvc/tree/webhdfs.py
* Remove unnecessary deque
* Check exception messages hdfs client failure from config
* Return client directly
Co-authored-by: Ruslan Kuprieiev <[email protected]>
* Split long line
* Add size to HashInfo
* Implement move via hdfs.rename
* Add missing line
* Make parent directories before moving file in WebHDFS
* Add total to progress bar
Co-authored-by: Ruslan Kuprieiev <[email protected]>
```
c:\hostedtoolcache\windows\python\3.6.8\x64\lib\site-packages\moto\core\models.py:17: in <module>
import responses
c:\hostedtoolcache\windows\python\3.6.8\x64\lib\site-packages\responses.py:29: in <module>
from urllib3.connection import HTTPHeaderDict
E ImportError: cannot import name 'HTTPHeaderDict'
```
moto has aws-sam-translator as an indirect dependency that since 1.29.0
started to have a public package
that causes conflicts in flufl.lock when it tries to use atpublic
package.
```
tests/func/test_add.py:21: in <module>
from dvc.main import main
dvc/main.py:7: in <module>
from dvc import analytics
dvc/analytics.py:16: in <module>
from dvc.lock import Lock, LockError
dvc/lock.py:8: in <module>
import flufl.lock
/opt/hostedtoolcache/Python/3.6.12/x64/lib/python3.6/site-packages/flufl/lock/__init__.py:3: in <module>
from flufl.lock._lockfile import (
/opt/hostedtoolcache/Python/3.6.12/x64/lib/python3.6/site-packages/flufl/lock/_lockfile.py:42: in <module>
from public import public
E ImportError: cannot import name 'public'
```
Oddly enough, this seems to also been causing pylint issues. public is a
terrible name for a package SMH
In my tests `Git._ignore` is taking around ~65seconds for the whole test
suit(~50K invocations). The reason is that `gitpython` is launching a real
`git check-ignore` process, that takes time and resources.
Switching to `pygit2` or `dulwich` makes it drop to ~5seconds, saving us
almost a minute of the test run time and a lot of resources by not
launching any processes. I've chosen `dulwich` over `pygit2` here
because it is much more pythonic and as long as the performance is
on-par with `pygit2` it is better for us to use `dulwich`.
Saugat Pachhai
nik123
Peter Rowlands (변기호)
Batuhan Taskaya
Paweł Redzyński
Corentin Hembise
Asif Saif UddinandRuslan Kuprieiev
João Rafael
Tomás Farías SantanaandRuslan Kuprieiev
Sami Jawhar
Tomás Farías Santana