Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-06 15:28:28 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,569 Commits 7 Branches 579 Tags
fc4e31bbf8d4699ca71f11ea7479527f0a89336b
Commit Graph
6569 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Batuhan Taskaya fc4e31bbf8 azure: use selector event loop for windows (#5983) 2021-05-10 16:50:58 +03:00
dependabot[bot] 674693bb86 build(deps): bump pytest-mock from 3.6.0 to 3.6.1 (#5981) 
Bumps [pytest-mock](https://github.com/pytest-dev/pytest-mock) from 3.6.0 to 3.6.1.
- [Release notes](https://github.com/pytest-dev/pytest-mock/releases)
- [Changelog](https://github.com/pytest-dev/pytest-mock/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest-mock/compare/v3.6.0...v3.6.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-07 18:20:01 +05:45
Ruslan Kuprieiev 0ad1408da7 plots: handle NoRemoteError for dirs (#5976) 2021-05-07 14:35:35 +03:00
Saugat Pachhai 19f2fe52e6 tests: skip copying sqlite files in test directories (#5977) 2021-05-06 18:51:07 +05:45
Saugat Pachhai 77be01ee2d try to close dvc.state on exit (#5962) 
* try to close dvc.state on exit

I have a feeling that it is causing the issues with
shutil.copytree and some PermissionError during 'init',
as we are not closing it between multiple 'main()' calls.
But I am not quite sure, and this seems like a good thing to
do anyway.

* fixup

* close dvc on exit

* close cache on tests and on init
 2021-05-06 14:32:25 +03:00
Saugat Pachhai eabf6ddf27 run gdrive tests in a single job, run all on schedule (#5974) 2021-05-06 17:09:34 +05:45
Batuhan Taskaya 597f730e7d s3: respect credentialpath/configpath (#5973) 2021-05-06 12:27:29 +03:00
ciaochiaociaoandChiao-Wei Hsu 3670bf4fc3 commit,completion: Add stages completion for dvc commit (#5963) 
Fixes #5960

Co-authored-by: Chiao-Wei Hsu <[email protected]>
 2021-05-05 15:53:20 +03:00
Saugat Pachhai c2f1a1e162 Notify CI failures to Slack (#5970) 2021-05-05 18:37:33 +05:45
Batuhan Taskaya a234f53b76 azure: use a custom event loop for authentication (#5958) 
* azure: use a custom event loop for authentication

* add test_temp_event_loop

* wrap open() too!

* ignore aiohttp warnings for normal level use-cases

* rename tests/unit/remote/test_azure to tests/unit/fs/test_azure
 2021-05-05 15:41:55 +03:00
dependabot[bot] 0a0d146386 build(deps): bump pytest from 6.2.3 to 6.2.4 (#5968) 
Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.2.3 to 6.2.4.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/6.2.3...6.2.4)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-05 14:31:20 +05:45
Batuhan Taskaya d4a3df73df s3: fsspec migration (#5683) 
* s3: migrate to s3fs

* s3: strip etag

* s3: disable instance cache

* tests: use the proper remote:// form in test_checkout

* tests: move/drop some tests

* setup: depend on s3fs

* tests: normalize s3 tests with some cleanups

* Use _download/_upload with old implementation

* fsspec: cache .exists() results

* setup: pin the boto version

* setup: install s3fs[boto3]

* s3: proper exception management

* s3: config improvements

* setup: reference without quotes

* setup: install boto separately

* setup: install moto server

* s3: mocking adjustments for moto_server

* s3: mark some fixtures with scope=session

* s3: s3_fake_creds_file scopes to func

* fsspec: add some docs regarding exists() optimization

* fsspec: exists() fallback

* fsspec: revert exists() optimization

* tests: ssl_verify => verify

* setup: temporarily fetch from master

* s3: add boto as a requirement

* tests: do an extra check regarding race conditions

* tests: use motoserver docker image

* tests: wrap config deletion with try/finally

* tests: disable s3 by default

* setup: upgrade fsspec
 2021-05-05 11:36:10 +03:00
Peter Rowlands (변기호) 256cb06921 move: validate new stage before move (#5953) 
* move: validate new stage before move

* verify that nothing was moved in tests
 2021-05-05 13:09:55 +09:00
Saugat Pachhai ff56379f8a tests: rerun gdrive and gs on failures (#5959) 
we just need to mark them with needs_internet marker.
Unfortunately, it's not possible to do that automatically
with the fixtures as fixtures are per-session and we need
something on more higher level (which the marker is not passed
through in the pytest).
 2021-05-04 17:59:50 +03:00
Saugat Pachhai 4733757bb1 use realpath for is_in_repo calculation (#5954) 
This is because, the repo root is used from a realpath, and the stage.wdir is abspath.
We might need to use same kinds of path, but this patch seems to be the easiest/fastest
way to fix the flaky tests (if all the tests agree).
 2021-05-04 17:59:09 +03:00
dependabot[bot] 3e9fd100ea build(deps): bump moto from 2.0.5 to 2.0.6 (#5955) 
Bumps [moto](https://github.com/spulec/moto) from 2.0.5 to 2.0.6.
- [Release notes](https://github.com/spulec/moto/releases)
- [Changelog](https://github.com/spulec/moto/blob/master/CHANGELOG.md)
- [Commits](https://github.com/spulec/moto/compare/2.0.5...2.0.6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-04 17:45:30 +03:00
dependabot[bot] c1a49b440d build(deps): bump pygments from 2.8.1 to 2.9.0 (#5956) 
Bumps [pygments](https://github.com/pygments/pygments) from 2.8.1 to 2.9.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.8.1...2.9.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-04 17:45:16 +03:00
dependabot[bot] c2734a2cd7 build(deps): bump codecov/codecov-action from v1.4.1 to v1.5.0 (#5957) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.1 to v1.5.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.1...a1ed4b322b4b38cb846afb5a0ebfa17086917d27)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-04 17:45:01 +03:00
Saugat Pachhai 71ab15771a exp show: decoupling tree and styling from row collection (#5903) 
* exp show: try to split presentation logic

Trying to remove tree and the bold/row-styling part even more

* fix naming of the experiment types
 2021-05-04 18:15:55 +05:45
Peter Rowlands (변기호) 53414a0618 dvcfile/stage: check if dvcfile would be gitignored on stage creation (#5907) 
* dvcfile: add check for gitignored when validating filenames

* update tests

* update exception using in tests

* review fixes

- clarify method name
- use FileIsGitIgnored
 2021-05-04 15:14:55 +09:00
Saugat Pachhai 708587a159 tests: mark test_stash_pop as xfail on linux (#5952) 
I tried increasing max_run to very high number: 1000. Even in those
cases, it failed 947 times. So I don't really see a reason to mark
it as flaky. Anyway, we are treating it as xfail by keeping it failing
in the CI, so this PR does not change anything for us. :)
 2021-05-04 11:22:57 +05:45
Ruslan Kuprieiev 3db25ea67d dvc: bump to 2.1.0 2.1.0 2021-05-03 16:29:49 +03:00
Saugat Pachhai d6afd26bea metrics diff/show: rollback to rounding fractional digits with precision (#5904) 2021-05-02 16:56:00 +03:00
Saugat Pachhai dba0aaeab7 accept dict for header styling (#5905) 
I found that it's harder to style a header with the list using index,
they are to be applied by header name. Otherwise, we might end up using
a different header for coloring, or worse throw some error because the
header name and style don't match.
Row does have it's property of index, so it makes sense there.
 2021-04-30 18:43:38 +05:45
Paweł Redzyński d509d77193 list: add --show-json option (#5888) 
* list: add --show-json option

* list: add test for --show-json
 2021-04-30 11:40:44 +02:00
Saugat Pachhai 102817e53a simplify experiments row collection logic (#5901) 2021-04-30 15:11:57 +05:45
Peter Rowlands (변기호) f771b1d5b0 exp run: error out when trying to resume from intermediate checkpoint w/no modifications (#5829) 
* apply: allow applying exp with multiple possible branches

* default to new checkpoint branch after applying ambiguous commit

* output shas in addition to exp names by default

* update show tests

* git: allow force checkout in detach_head context

* exp run: error when trying to resume from a non-head checkpoint without
modifications

* add test for apply/resume from intermediate checkpoint

* list some possible heads when multiple matches
 2021-04-30 15:12:44 +09:00
Peter Rowlands (변기호) 753803c177 pygit: force release of odb contexts after checkout/merge (#5849) 2021-04-30 10:18:37 +09:00
Ruslan Kuprieiev c9a748421a dependabot: add "maintenance" label (#5898) 2021-04-29 16:59:53 +03:00
Ruslan Kuprieiev cab4126ed4 output: relative paths outside of repo (#5893) 
* output: relative paths outside of repo

Fixes #5870

* scm: enhance FileNotInRepo message
 2021-04-28 16:13:01 +03:00
Batuhan Taskaya 2172add905 fs: introduce tree.find() (#5879) 2021-04-28 13:39:19 +03:00
Saugat Pachhai b2f71d459b Revert "build(deps): bump pylint-pytest from 1.0.3 to 1.1.2 (#5850)" (#5890) 
This reverts commit 8930be06e8.
 2021-04-27 13:32:39 +00:00
Saugat Pachhai 111c282808 exp show: separate ui representation of exp names and identifiers (#5885) 2021-04-27 18:37:35 +05:45
Saugat Pachhai 42cf394fde diff: use ui.table for --show-md (#5887) 2021-04-27 17:10:08 +05:45
dependabot[bot] 9c94b6dcaa build(deps): bump google-cloud-storage from 1.37.1 to 1.38.0 (#5883) 
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 1.37.1 to 1.38.0.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/master/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v1.37.1...v1.38.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-27 16:03:08 +05:45
dependabot[bot] a05be55e7e build(deps): bump pylint from 2.8.1 to 2.8.2 (#5884) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.8.1 to 2.8.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.8.1...v2.8.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-27 16:02:45 +05:45
Saugat Pachhai 8fd2eba9a2 refactor exp show: use new abstractions (#5880) 2021-04-27 13:14:56 +05:45
Saugat Pachhai f38d163d5c refactor diff and show to use new table ui and new data structure (#5872) 
* use new table data structure for diff and show

* fix params diff showing changes

* metrics show: use default precision

* skip precision format for exp diff params

* exp diff: separate tables in --quiet mode

* change precision to signify significants digits

changed f specifier to g, so that we show significant digits
instead of fractional digits. This does changes how precision
worked for metrics/params diff/show.
 2021-04-27 11:05:52 +05:45
Saugat Pachhai 0860e419e4 ui: tabular data (#5852) 
* introduce tabular data structure

* ignore ellipsis
 2021-04-26 14:54:23 +03:00
Saugat Pachhai 2978a5d7e9 dagascii: prune unused pager (#5876) 2021-04-26 14:52:29 +03:00
dependabot[bot] af6921e9ac build(deps): bump pre-commit/action from v2.0.2 to v2.0.3 (#5856) 
Bumps [pre-commit/action](https://github.com/pre-commit/action) from v2.0.2 to v2.0.3.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/v2.0.2...9b88afc9cd57fd75b655d5c71bd38146d07135fe)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 14:08:40 +03:00
dependabot[bot] 8930be06e8 build(deps): bump pylint-pytest from 1.0.3 to 1.1.2 (#5850) 
Bumps [pylint-pytest](https://github.com/reverbc/pylint-pytest) from 1.0.3 to 1.1.2.
- [Release notes](https://github.com/reverbc/pylint-pytest/releases)
- [Changelog](https://github.com/reverbc/pylint-pytest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/reverbc/pylint-pytest/compare/pylint-pytest-1.0.3...pylint-pytest-1.1.2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 14:05:33 +03:00
dependabot[bot] 9eba910023 build(deps): bump pytest-mock from 3.5.1 to 3.6.0 (#5874) 
Bumps [pytest-mock](https://github.com/pytest-dev/pytest-mock) from 3.5.1 to 3.6.0.
- [Release notes](https://github.com/pytest-dev/pytest-mock/releases)
- [Changelog](https://github.com/pytest-dev/pytest-mock/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest-mock/compare/v3.5.1...v3.6.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 13:21:58 +03:00
dependabot[bot] 0728777494 build(deps): bump pylint from 2.7.4 to 2.8.1 (#5873) 
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.7.4 to 2.8.1.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.7.4...pylint-2.8.1)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 13:21:51 +03:00
Jan 54c0675fc6 Add periods (#5867) 2021-04-23 13:58:12 +03:00
Peter Rowlands (변기호) e0d90cdda1 exp run: dvc commit DVC-tracked data deps when stashing an experiment (#5859) 
* stage,commit: add support for only committing data stages from
experiments

* exp run: `dvc commit` data deps when stashing exp changes

* add test for modified DVC-tracked data when generating experiment

* test with and without params change

* handle run without target
 2021-04-23 11:59:15 +09:00
Batuhan Taskaya 17d7b06f7b output: introduce BaseOutput.transfer (#5861) 
* output: introduce BaseOutput.transfer

* add: use BaseOutput.transfer for to-cache
 2021-04-22 15:01:44 +03:00
dependabot[bot] 8ac26b58ee build(deps): bump codecov/codecov-action from v1.4.0 to v1.4.1 (#5857) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.0 to v1.4.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.0...967e2b38a85a62bd61be5529ada27ebc109948c2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-22 13:50:46 +03:00
Batuhan Taskaya a42e363217 objects: don't recompute hash on odb.add() if verify is False (#5860) 2021-04-22 13:50:24 +03:00
Paweł Redzyński d87ace0968 html: enable providing custom html page templates (#5694) 
* html: enable providing custom html page templates

* plots: add html template configuration option
 2021-04-20 11:02:14 +02:00