* git: add support for dulwich Stash
* git: start adding support for custom refs
* git: add Stash and helper context managers for preserving
workspace state
* scm: cleanup refs/stash and add push_refspec
* tests: add tests for git refs and stash
* fix push_refspec
* tests: add tests for git push_refspec
* scm: add git fetch-refspecs
* add test for fetch refspecs
* scm: add test for git refs containing
* remove extraneous log msgs
* fix typing issues
* stash: only try to push if workspace is dirty
* update dulwich to 0.20.14
* scm: move dulwich into scm.backend
* move stash internals to backends
* scm: move git code into dvc/scm/git/... module
* git: use ABC for BaseGitBackend
* fix rebase issue
* git: fix refspec tests on windows
- git file:// URLs need posix style paths
* git: move get_refs_containing to backend
* scm: add callback for handling diverged refs during fetch
Having that logic in `create_stage` is unnatural and also has some
side-effects for unsuspecting users. For example, it will check the
filesystem for existing dvcfile and will try to load a corresponding old
stage. I've adjusted it with minimal changes to better reflect the logic
behind it.
Pre-requisite for #4907
* repro: support regex/foreach-group to run at once
Fixes#4912Fixes#4886Fixes#4958
* Use `tree.isdir` rather than `os.path.isdir`
* Use glob rather than regex
* Update dvc/command/repro.py
* s/regex/glob
* disable glob on `collect_granular`
There's no need for a glob here
* add tests for `collect` and `collect_granular`
* s3: use mocked s3 instead of the real one
Not removing the fixtures just yet
* Update moto
Tests was failing, because of a change in `responses` package, which
`moto` was mocking. This was fixed on
https://github.com/spulec/moto/pull/3466 which is only available on
`dev122` release.
* Use mocked s3 in checkout tests
`Azure` tests are flaky for some reason. When I investigated one
of the job [1], something seemed wrong with the Azurite or
the client lib.
On `AzureTree.blob_service`, we do the following to check if the
container exists:
```python
try:
container_client.get_container_properties()
except ResourceNotFoundError:
container_client.create_container()
```
When looking at the test log, I found out that the
`get_container_properties` was failing with
`ErrorCode:ContainerNotFound` whereas `create_container`
successively was failing with `ErrorCode:ContainerAlreadyExists`.
`create_container` is correct, but `get_container_properties` is
incorrect, as many of the Azure tests have already run successfully
before.
So, either the Azurite has/had bug or the client lib has a bug.
Anyway, we'll find out after updating the Azurite.
[1]: https://github.com/iterative/dvc/runs/1448864706#step:7:4029
* parametrization: prevent dumping parametrized stages
This is a bit hackish, but does the job done.
It feels like we need some unified way to be able to load
and dump, but not sure how. Till then, this dataclass
will be helpful in communicating between those.
* s/Meta/RawData
* Update dvc/stage/loader.py
Some of the tests are run by default, other requires
`--enable-{remote}` to run. The remotes that are run by default
also have a `--disable-{remote}` flag. All of the remotes can be
enabled using `--all` flag.
To use this functionality in the fixture,
`test_config` fixture has to be used, and requires to call
`test_config.requires(remote)`.
In an individual test, you can simply use `pytest.mark.remote`
to make them skip or run.
Via pyparsing. The error message is not that great, as it's greedy.
But, it does point out where the mistakes are, correctly most of the
time. :)
Also raises appropriate error messages for ParseError
The schema is changed of `vars` to accomodate the both use of
`imports` and setting variables locally.
Now:
```
vars:
- params.yaml
- foo: foo
bar: bar
```
Also, same distinction occurs for the stages.
Paweł Redzyński
Saugat Pachhai
Ruslan Kuprieiev
Peter Rowlands (변기호)
Artem Dudarev
Jorge Orpinel
Sami Jawhar