Saugat Pachhai
a74855b9f7
fix empty string parsing for the command ( #6074 )
...
Fixes #6072
2021-05-28 06:51:27 +00:00
Saugat Pachhai
63dba45c14
ui: write gets a styled param for rich console printing ( #6037 )
2021-05-28 09:30:56 +05:45
dependabot[bot]
e96f0cfed5
build(deps): bump urllib3 from 1.26.4 to 1.26.5 ( #6069 )
...
Bumps [urllib3](https://github.com/urllib3/urllib3 ) from 1.26.4 to 1.26.5.
- [Release notes](https://github.com/urllib3/urllib3/releases )
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst )
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.4...1.26.5 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-27 13:07:14 +05:45
Saugat Pachhai
d818e70bf3
output/dep: specify schema as a dictionary ( #6061 )
2021-05-26 14:58:51 +03:00
Saugat Pachhai
e28726c057
fix mypy errors ( #6062 )
2021-05-26 06:01:04 +00:00
Ruslan Kuprieiev
8a7ccab82f
fs: deps/outs: get rid of types ( #6054 )
...
Pre-requisite for removing fs.path_info and switching to regular strings
as paths in filesystems.
With new filesystems we no longer need the zoo of different dependency
and output types, as most of them are now just handled by the underlying
filesystem. Though deps/outs are still due for a refactor in other
aspects.
2021-05-25 19:25:09 +03:00
Batuhan Taskaya
50ec64b841
setup: bump fsspec/s3fs/gcsfs to 2021.5.0 ( #6056 )
2021-05-25 14:54:56 +03:00
Ruslan Kuprieiev
aa672c5108
setup: bump dulwich to 0.20.23 ( #6053 )
...
Fixes #6052
2021-05-25 00:30:22 +03:00
Peter Rowlands (변기호)
9dda6bd32c
git: update dulwich to 0.20.22 ( #6049 )
...
* git: update dulwich to 0.20.22
* git: disable reading system config in CI
2021-05-24 17:40:14 +09:00
Saugat Pachhai
bbe06df18e
ui: migrate dag ( #6044 )
2021-05-21 07:35:49 +00:00
Batuhan Taskaya
960ece926c
fsspec: split the cloud-specific functionality from the base wrapper ( #6014 )
...
* fsspec: split the cloud-specific functionality from the base fs
* rename Cloud to Object
2021-05-19 18:12:07 +03:00
Ruslan Kuprieiev
c6b4c94fde
dvcfs: raise FileNotFoundError on missing version info ( #6017 )
2021-05-18 19:12:35 +03:00
Saugat Pachhai
b9000ed926
pre-commit: update black to 21.5b1 ( #6032 )
...
* pre-commit: update black to 21.5b1
also updated others, but black is more noticeable
* remove magic trailing comma
2021-05-18 15:51:15 +00:00
Saugat Pachhai
58c465ef13
ui: migrate completion to ui.write ( #6029 )
2021-05-18 12:02:18 +00:00
Saugat Pachhai
789faa4f11
ui: migrate checkout to ui.write ( #6028 )
2021-05-18 13:55:11 +02:00
dependabot[bot]
9a5b7251b3
build(deps): bump pydocstyle from 6.0.0 to 6.1.1 ( #6027 )
...
Bumps [pydocstyle](https://github.com/PyCQA/pydocstyle ) from 6.0.0 to 6.1.1.
- [Release notes](https://github.com/PyCQA/pydocstyle/releases )
- [Changelog](https://github.com/PyCQA/pydocstyle/blob/master/docs/release_notes.rst )
- [Commits](https://github.com/PyCQA/pydocstyle/compare/6.0.0...6.1.1 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-18 14:42:05 +05:45
Ivan Shcheklein
aab5dec7f5
rwlock: add missing space to the message ( #6026 )
2021-05-18 12:32:35 +09:00
Saugat Pachhai
e6a2b8a2ad
ui: migrate cache:dir and check-ignore to ui.write ( #6021 )
2021-05-17 18:50:58 +03:00
dependabot[bot]
9c4a8c384a
build(deps): bump pytest-cov from 2.11.1 to 2.12.0 ( #6022 )
...
Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov ) from 2.11.1 to 2.12.0.
- [Release notes](https://github.com/pytest-dev/pytest-cov/releases )
- [Changelog](https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/pytest-dev/pytest-cov/compare/v2.11.1...v2.12.0 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-17 15:26:41 +05:45
Peter Rowlands (변기호)
71ea12877d
stage: allow duplicate values in _read_env ( #6007 )
2021-05-14 15:49:41 +02:00
Saugat Pachhai
c6f49bd23d
ls: use ui.write ( #6004 )
2021-05-14 10:19:55 +05:45
Ruslan Kuprieiev
a5b080645c
dvcignore: refactor from_file(s) ( #6002 )
2021-05-13 12:23:58 +02:00
Ruslan Kuprieiev
6d941e6cb5
fs: get rid of fs_root ( #6000 )
...
* dvcignore: don't use fs_root
* fs: get rid of fs_root
2021-05-13 04:15:56 +03:00
Ruslan Kuprieiev
da0f0406fb
fs: fix typo
2021-05-12 22:28:58 +03:00
Ruslan Kuprieiev
89b40afee7
fs: make constructor closer to fsspec ( #5911 )
...
* fs: don't use repo for jobs/hash_jobs
* fs: local: exists: use fsspec's LocalFileSystem
* fs: make constructor comply with fsspec
2021-05-12 22:03:09 +03:00
dependabot[bot]
0ca08d2fea
build(deps): bump rtCamp/action-slack-notify from 2 to 2.1.3 ( #5993 )
...
Bumps [rtCamp/action-slack-notify](https://github.com/rtCamp/action-slack-notify ) from 2 to 2.1.3.
- [Release notes](https://github.com/rtCamp/action-slack-notify/releases )
- [Commits](https://github.com/rtCamp/action-slack-notify/compare/v2...v2.1.3 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-12 13:06:50 +05:45
dependabot[bot]
14e1511eee
build(deps): bump actions/setup-ruby from 1 to 1.1.3 ( #5994 )
...
Bumps [actions/setup-ruby](https://github.com/actions/setup-ruby ) from 1 to 1.1.3.
- [Release notes](https://github.com/actions/setup-ruby/releases )
- [Commits](https://github.com/actions/setup-ruby/compare/v1...v1.1.3 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-12 13:06:40 +05:45
b349b39d62
Decoupling dvcignore and fs ( #5812 )
...
* Decoupling dvcignore and fs
[x] remove dvcignore from fs objects
[x] attach dvcignore to repo
[x] add ignore check to functions(add, checkout, output, pipeline, etc)
[x] pass all tests
[x] final check
* Solve all of the arugments error.
* add dvcignore to dvc list
* Solve some test_ignore cases
* add dvcignore to checkout.
1. add dvcignore to 'checkout', 'add'
2. add ignore_filter to fs
* Solve two tests
* Get rid of ignore_filter
* Add ignore examine to download
* Solve a test issue
* Solve dvcignore and subrepo problems
* Solve sub repo error
* Revert some changes
* pass all tests
* Remove debug codes
* Some final exams
* dvcignore trie tree only update when needed
* reuse of the dir result in path walk.
* Some change after code review.
* Recover something from false `--force` push
* add abspath trans to dvcginore
* pass pylint
* pylint related
* Update dvc/fs/repo.py
Co-authored-by: Ruslan Kuprieiev <[email protected] [email protected] [email protected] https://github.com/iterative/dvc/pull/5812/files#r630604524
Co-authored-by: myname <[email protected] [email protected] [email protected] [email protected]
2021-05-12 05:29:04 +03:00
Saugat Pachhai
25eb9ad8c1
RFC: respect $PAGER env vars, make -FRSX default in less ( #5906 )
2021-05-11 16:35:18 +00:00
Saugat Pachhai
eab8146b76
ui: use sys.stdout and sys.stderr all the time ( #5991 )
...
We used to allow changing default output and error stream
but that was problematic for the testing. Now it just uses
sys, so capsys should work for the testing.
The proper way to change sys.stdout and sys.stderr is
by using 'redirect_stdout' now.
2021-05-11 11:33:22 +00:00
Batuhan Taskaya
e7571ab325
setup: fixate aiobotocore/botocore/boto3 versions ( #5987 )
2021-05-10 18:28:11 +03:00
Batuhan Taskaya
fc4e31bbf8
azure: use selector event loop for windows ( #5983 )
2021-05-10 16:50:58 +03:00
dependabot[bot]
674693bb86
build(deps): bump pytest-mock from 3.6.0 to 3.6.1 ( #5981 )
...
Bumps [pytest-mock](https://github.com/pytest-dev/pytest-mock ) from 3.6.0 to 3.6.1.
- [Release notes](https://github.com/pytest-dev/pytest-mock/releases )
- [Changelog](https://github.com/pytest-dev/pytest-mock/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pytest-dev/pytest-mock/compare/v3.6.0...v3.6.1 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-07 18:20:01 +05:45
Ruslan Kuprieiev
0ad1408da7
plots: handle NoRemoteError for dirs ( #5976 )
2021-05-07 14:35:35 +03:00
Saugat Pachhai
19f2fe52e6
tests: skip copying sqlite files in test directories ( #5977 )
2021-05-06 18:51:07 +05:45
Saugat Pachhai
77be01ee2d
try to close dvc.state on exit ( #5962 )
...
* try to close dvc.state on exit
I have a feeling that it is causing the issues with
shutil.copytree and some PermissionError during 'init',
as we are not closing it between multiple 'main()' calls.
But I am not quite sure, and this seems like a good thing to
do anyway.
* fixup
* close dvc on exit
* close cache on tests and on init
2021-05-06 14:32:25 +03:00
Saugat Pachhai
eabf6ddf27
run gdrive tests in a single job, run all on schedule ( #5974 )
2021-05-06 17:09:34 +05:45
Batuhan Taskaya
597f730e7d
s3: respect credentialpath/configpath ( #5973 )
2021-05-06 12:27:29 +03:00
ciaochiaociao and Chiao-Wei Hsu
3670bf4fc3
commit,completion: Add stages completion for dvc commit ( #5963 )
...
Fixes #5960
Co-authored-by: Chiao-Wei Hsu <[email protected]
2021-05-05 15:53:20 +03:00
Saugat Pachhai
c2f1a1e162
Notify CI failures to Slack ( #5970 )
2021-05-05 18:37:33 +05:45
Batuhan Taskaya
a234f53b76
azure: use a custom event loop for authentication ( #5958 )
...
* azure: use a custom event loop for authentication
* add test_temp_event_loop
* wrap open() too!
* ignore aiohttp warnings for normal level use-cases
* rename tests/unit/remote/test_azure to tests/unit/fs/test_azure
2021-05-05 15:41:55 +03:00
dependabot[bot]
0a0d146386
build(deps): bump pytest from 6.2.3 to 6.2.4 ( #5968 )
...
Bumps [pytest](https://github.com/pytest-dev/pytest ) from 6.2.3 to 6.2.4.
- [Release notes](https://github.com/pytest-dev/pytest/releases )
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst )
- [Commits](https://github.com/pytest-dev/pytest/compare/6.2.3...6.2.4 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-05 14:31:20 +05:45
Batuhan Taskaya
d4a3df73df
s3: fsspec migration ( #5683 )
...
* s3: migrate to s3fs
* s3: strip etag
* s3: disable instance cache
* tests: use the proper remote:// form in test_checkout
* tests: move/drop some tests
* setup: depend on s3fs
* tests: normalize s3 tests with some cleanups
* Use _download/_upload with old implementation
* fsspec: cache .exists() results
* setup: pin the boto version
* setup: install s3fs[boto3]
* s3: proper exception management
* s3: config improvements
* setup: reference without quotes
* setup: install boto separately
* setup: install moto server
* s3: mocking adjustments for moto_server
* s3: mark some fixtures with scope=session
* s3: s3_fake_creds_file scopes to func
* fsspec: add some docs regarding exists() optimization
* fsspec: exists() fallback
* fsspec: revert exists() optimization
* tests: ssl_verify => verify
* setup: temporarily fetch from master
* s3: add boto as a requirement
* tests: do an extra check regarding race conditions
* tests: use motoserver docker image
* tests: wrap config deletion with try/finally
* tests: disable s3 by default
* setup: upgrade fsspec
2021-05-05 11:36:10 +03:00
Peter Rowlands (변기호)
256cb06921
move: validate new stage before move ( #5953 )
...
* move: validate new stage before move
* verify that nothing was moved in tests
2021-05-05 13:09:55 +09:00
Saugat Pachhai
ff56379f8a
tests: rerun gdrive and gs on failures ( #5959 )
...
we just need to mark them with needs_internet marker.
Unfortunately, it's not possible to do that automatically
with the fixtures as fixtures are per-session and we need
something on more higher level (which the marker is not passed
through in the pytest).
2021-05-04 17:59:50 +03:00
Saugat Pachhai
4733757bb1
use realpath for is_in_repo calculation ( #5954 )
...
This is because, the repo root is used from a realpath, and the stage.wdir is abspath.
We might need to use same kinds of path, but this patch seems to be the easiest/fastest
way to fix the flaky tests (if all the tests agree).
2021-05-04 17:59:09 +03:00
dependabot[bot]
3e9fd100ea
build(deps): bump moto from 2.0.5 to 2.0.6 ( #5955 )
...
Bumps [moto](https://github.com/spulec/moto ) from 2.0.5 to 2.0.6.
- [Release notes](https://github.com/spulec/moto/releases )
- [Changelog](https://github.com/spulec/moto/blob/master/CHANGELOG.md )
- [Commits](https://github.com/spulec/moto/compare/2.0.5...2.0.6 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-04 17:45:30 +03:00
dependabot[bot]
c1a49b440d
build(deps): bump pygments from 2.8.1 to 2.9.0 ( #5956 )
...
Bumps [pygments](https://github.com/pygments/pygments ) from 2.8.1 to 2.9.0.
- [Release notes](https://github.com/pygments/pygments/releases )
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES )
- [Commits](https://github.com/pygments/pygments/compare/2.8.1...2.9.0 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-04 17:45:16 +03:00
dependabot[bot]
c2734a2cd7
build(deps): bump codecov/codecov-action from v1.4.1 to v1.5.0 ( #5957 )
...
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action ) from v1.4.1 to v1.5.0.
- [Release notes](https://github.com/codecov/codecov-action/releases )
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md )
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.1...a1ed4b322b4b38cb846afb5a0ebfa17086917d27 )
Signed-off-by: dependabot[bot] <[email protected]
2021-05-04 17:45:01 +03:00
Saugat Pachhai
71ab15771a
exp show: decoupling tree and styling from row collection ( #5903 )
...
* exp show: try to split presentation logic
Trying to remove tree and the bold/row-styling part even more
* fix naming of the experiment types
2021-05-04 18:15:55 +05:45
Saugat Pachhai
dependabot[bot]
Ruslan Kuprieiev
Batuhan Taskaya
Peter Rowlands (변기호)
Ivan Shcheklein
ciaochiaociaoandChiao-Wei Hsu