* api: add support for simple wildcards
Related to #4816.
Signed-off-by: Ioana Grigoropol <[email protected]>
* api: make wildcard interpretation optional
Adds a new argument for the add command `glob` that is disabled by default and when enabled it passes
the input targets through glob filtering.
Related: #4816
Signed-off-by: Ioana Grigoropol <[email protected]>
* Update dvc/repo/add.py
* Update dvc/repo/add.py
* Update dvc/repo/add.py
* Update dvc/repo/add.py
* Update dvc/repo/add.py
Co-authored-by: Ruslan Kuprieiev <[email protected]>
```
c:\hostedtoolcache\windows\python\3.6.8\x64\lib\site-packages\moto\core\models.py:17: in <module>
import responses
c:\hostedtoolcache\windows\python\3.6.8\x64\lib\site-packages\responses.py:29: in <module>
from urllib3.connection import HTTPHeaderDict
E ImportError: cannot import name 'HTTPHeaderDict'
```
* experiments: refactor regular/checkpoint experiment separation
* fix scm_checkout pull bug
* experiments: return stages from executor.reproduce
* since executor has to move to threadpool execution anyways, we no
longer have to work around the pickle repro result limitation
* refactor new/resume behavior
* refactor checkpoints tests
* use -f to force checkpoints reset/restart
* checkpoints: use 'exp run' and 'exp res[ume]' commands
* stage: all outs can be missing for checkpoint stages
* regular outputs may not be written at the time a checkpoint is
generated, but subsequent checkouts of that stage should not fail due
to the missing outs
* commit stage on checkpoint generation otherwise non-persist cached outs
are lost
* checkpoints: wait for stage proc to end after killing via terminate()
* stage: fix issues with killing checkpoint stage
* refactor signaling (use threading.Event instead of Condition)
* fix windows bug (kill all child processes via taskkill)
* experiments: refactor executor collection
* tests: move checkpoint tests into separate module
* refactor experiment tests to use stage fixture
We wrap every data to one of the `Node` object. But, since we were
passing `self.data` instead of `self`, it was skipping the conversion
and instead was just trying to set data as-is, which would have make
our latter assumptions incorrect, as other codebases assume it to be
a `Node`.
I have added asserts in a few places. Also, I noticed that, on a few
test assumptions, when `resolving`, we were not converting values
back to it's original form (`CtxDict` -> `dict`, `CtxList` -> `list`
and `Value` -> _). So, I have added `node.value` property, as it is
a complement of the above fix.
This caused a ripple effect for the changes in the API as `foreach`
now resolved and changed the data to the original form,
which prevents us from tracking the data later in the "field".
So, `resolve`/`select` got a real `unwrap` option that should
reliably work for all kinds of `Node` objects.
Currently we are converting dir_info to/from lists all the time.
The reason is that dir_info is stored as list of dicts in *.dir files,
but that makes it hard to work with. In addition to that, we will likely
be changing .dir file format in the near future #829, so we need to
abstract away dir_info into something that we won't care how it will be
stored on disk.
Related #3256
Related #4847
moto has aws-sam-translator as an indirect dependency that since 1.29.0
started to have a public package
that causes conflicts in flufl.lock when it tries to use atpublic
package.
```
tests/func/test_add.py:21: in <module>
from dvc.main import main
dvc/main.py:7: in <module>
from dvc import analytics
dvc/analytics.py:16: in <module>
from dvc.lock import Lock, LockError
dvc/lock.py:8: in <module>
import flufl.lock
/opt/hostedtoolcache/Python/3.6.12/x64/lib/python3.6/site-packages/flufl/lock/__init__.py:3: in <module>
from flufl.lock._lockfile import (
/opt/hostedtoolcache/Python/3.6.12/x64/lib/python3.6/site-packages/flufl/lock/_lockfile.py:42: in <module>
from public import public
E ImportError: cannot import name 'public'
```
Oddly enough, this seems to also been causing pylint issues. public is a
terrible name for a package SMH
* http: added check for HTTP status code 404
Fixes#4288
* http: add unit test and minor change
In tests_exists, changed to raise a HTTPError from dvc instead.
Fixes#4288
* http: removed fixtures in test for HTTPTree.exists
Fixes#4288
* test: add comments on why `raw` is required on the mock
Co-authored-by: Saugat Pachhai (सौगात) <[email protected]>
* command: add dvc push instruction after dvc repro
Adds a logger.info() statement to remind the user to run dvc push after
running dvc repro.
Fixes#2359
* command: change dvc push instruction for dvc repro
* command: Use backticks and else in dvc repro log
* add: warn on cache link failure instead of error
* add test case
* fix linted string
Co-authored-by: Paweł Redzyński <[email protected]>
Co-authored-by: Paweł Redzyński <[email protected]>
* metrics: include revision information when collecting metrics files
* plots: test warning message on no plot on revision
* plots/metrics: move to one package, introduce common tests
* tests: fix utils placement
In my tests `Git._ignore` is taking around ~65seconds for the whole test
suit(~50K invocations). The reason is that `gitpython` is launching a real
`git check-ignore` process, that takes time and resources.
Switching to `pygit2` or `dulwich` makes it drop to ~5seconds, saving us
almost a minute of the test run time and a lot of resources by not
launching any processes. I've chosen `dulwich` over `pygit2` here
because it is much more pythonic and as long as the performance is
on-par with `pygit2` it is better for us to use `dulwich`.
Saugat Pachhai
Sami Jawhar
Peter Rowlands (변기호)
Ioana GrigoropolandRuslan Kuprieiev
LucaButeraandLuca Butera
Michal NowikowskiandRuslan Kuprieiev
Paweł Redzyński
Liew XunandSaugat Pachhai
simonlousky
Kevin Lloyd Bernal
João Vitor
Juliano Efson Sales