Commit Graph
222 Commits
Author SHA1 Message Date
Saugat Pachhai 1bb7f39e14 add support for Python 3.10 (#6422)
* add support for Python 3.10

pyarrow is not available, so we are skipping tests for it.
And, restricting `hdfs` extras to be not installed in Python3.10.

Pathlib removed `_init` method and `init` parameter, so just working
around that. It's not a big change.

* skip windows on 3.10; update pip/wheel/setuptools during install

pywin32 is not yet available for 3.10 Python, so skipping those for now

* install libgit2 with Python3.10 as pygit2 wheel is not available

* install libgit2

* only run on macOS, libgit2-1.1 not available on ubuntu
2021-08-16 16:24:28 +05:45
dependabot[bot] b294bca383 build(deps): bump codecov/codecov-action from 2.0.1 to 2.0.2 (#6361)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v2.0.1...v2.0.2)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-28 10:21:10 +03:00
dependabot[bot] 005bca1da6 build(deps): bump codecov/codecov-action from 1.5.2 to 2.0.1 (#6334)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1.5.2 to 2.0.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.5.2...v2.0.1)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-20 14:06:50 +03:00
GaoandRuslan Kuprieiev 93f36d9351 oss: migrate to ossfs (#6307)
* Migrate to ossfs

fix 6177

* Add verification to object stored on oss

* Change the oss remote's dependency.

pass all of the tests

* Performance optimaztion and ossfs version update

* Remove the argument to set the resumable get/put

1. Experiments shows resumable get/put has little influnence on performance.
2. Just use resumable ones as default

* Use a new api for performance optimization

* Without filesize progress bar version

* With progress bar version

* Bump and add comments

1. Bump ossfs from 2021.6.2 to 2021.7.0
2. add some comments to the verification trigger

* Change the name of kwrags following parent class.

* Bump to new ossfs and rewrite upload_fobj

Because some problem with oss-emulater
1. bump to ossfs 2021.7.1
2. rewrite upload_fobj

* Optimization on rm_files

* Some optimization follow the old version.

* Update dvc/fs/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/objects/db/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* bump into ossfs 2021.7.2

* Bump into ossfs 2021.7.3OC

* Switch to real_oss_server

* Switch to a real oss server

* Use the oss bucket located in US.

* Remove some emulator related code

* Add envs to ci

* remove some oss pytest XFAIL

Because we didn't use emulator, we can remove some pytest XFAILs

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2021-07-13 15:27:34 +03:00
Saugat Pachhai b7c6df0ece Create CODEOWNERS (#6213) 2021-06-23 11:56:37 +05:45
dependabot[bot] 57c728269d build(deps): bump codecov/codecov-action from 1.5.0 to 1.5.2 (#6139)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1.5.0 to 1.5.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.5.0...v1.5.2)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-09 23:30:31 +05:45
dependabot[bot] bf5c28b71a build(deps): bump rtCamp/action-slack-notify from 2.1.3 to 2.2.0 (#6128)
Bumps [rtCamp/action-slack-notify](https://github.com/rtCamp/action-slack-notify) from 2.1.3 to 2.2.0.
- [Release notes](https://github.com/rtCamp/action-slack-notify/releases)
- [Commits](https://github.com/rtCamp/action-slack-notify/compare/v2.1.3...v2.2.0)

---
updated-dependencies:
- dependency-name: rtCamp/action-slack-notify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-06-07 14:04:32 +05:45
Batuhan Taskaya 0e3db0d323 ci: move to ubuntu-18 from ubuntu-16 (#6101) 2021-06-02 17:31:28 +03:00
dependabot[bot] efda015de4 build(deps): bump actions/cache from 2.1.5 to 2.1.6 (#6073)
Bumps [actions/cache](https://github.com/actions/cache) from 2.1.5 to 2.1.6.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.5...v2.1.6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-28 14:43:23 +05:45
dependabot[bot] 0ca08d2fea build(deps): bump rtCamp/action-slack-notify from 2 to 2.1.3 (#5993)
Bumps [rtCamp/action-slack-notify](https://github.com/rtCamp/action-slack-notify) from 2 to 2.1.3.
- [Release notes](https://github.com/rtCamp/action-slack-notify/releases)
- [Commits](https://github.com/rtCamp/action-slack-notify/compare/v2...v2.1.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-12 13:06:50 +05:45
dependabot[bot] 14e1511eee build(deps): bump actions/setup-ruby from 1 to 1.1.3 (#5994)
Bumps [actions/setup-ruby](https://github.com/actions/setup-ruby) from 1 to 1.1.3.
- [Release notes](https://github.com/actions/setup-ruby/releases)
- [Commits](https://github.com/actions/setup-ruby/compare/v1...v1.1.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-12 13:06:40 +05:45
Saugat Pachhai eabf6ddf27 run gdrive tests in a single job, run all on schedule (#5974) 2021-05-06 17:09:34 +05:45
Saugat Pachhai c2f1a1e162 Notify CI failures to Slack (#5970) 2021-05-05 18:37:33 +05:45
dependabot[bot] c2734a2cd7 build(deps): bump codecov/codecov-action from v1.4.1 to v1.5.0 (#5957)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.1 to v1.5.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.1...a1ed4b322b4b38cb846afb5a0ebfa17086917d27)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-04 17:45:01 +03:00
Ruslan Kuprieiev c9a748421a dependabot: add "maintenance" label (#5898) 2021-04-29 16:59:53 +03:00
dependabot[bot] af6921e9ac build(deps): bump pre-commit/action from v2.0.2 to v2.0.3 (#5856)
Bumps [pre-commit/action](https://github.com/pre-commit/action) from v2.0.2 to v2.0.3.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/v2.0.2...9b88afc9cd57fd75b655d5c71bd38146d07135fe)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-04-26 14:08:40 +03:00
dependabot[bot] 8ac26b58ee build(deps): bump codecov/codecov-action from v1.4.0 to v1.4.1 (#5857)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.0 to v1.4.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.0...967e2b38a85a62bd61be5529ada27ebc109948c2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-04-22 13:50:46 +03:00
Ruslan Kuprieiev 3472cb1a6a tests: don't set sensitive env secrets for all steps (#5823)
Due to https://about.codecov.io/security-update/ , there is a potential that we've leaked these. This PR takes a more granular approach when setting secret env vars so that this doesn't happen in the future.

We'll change these secrets after this PR is merged.
2021-04-19 16:39:37 +03:00
dependabot[bot] 48890c22b7 build(deps): bump codecov/codecov-action from v1.3.2 to v1.4.0 (#5838)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.3.2 to v1.4.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.3.2...0e28ff86a50029a44d10df6ed4c308711925a6a8)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-04-17 11:20:46 +05:45
dependabot[bot] 2472c04873 build(deps): bump actions/cache from v2.1.4 to v2.1.5 (#5805)
Bumps [actions/cache](https://github.com/actions/cache) from v2.1.4 to v2.1.5.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.4...1a9e2138d905efd099035b49d8b7a3888c653ca8)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-04-13 14:23:05 +03:00
dependabot[bot] c71a576a7a build(deps): bump actions/setup-python from v2.2.1 to v2.2.2 (#5804)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from v2.2.1 to v2.2.2.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.2.1...dc73133d4da04e56a135ae2246682783cc7c7cb6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-04-13 14:22:57 +03:00
dependabot[bot] 02d9684c33 build(deps): bump codecov/codecov-action from v1.3.1 to v1.3.2 (#5770)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.3.1 to v1.3.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.3.1...9b0b9bbe2c64e9ed41413180dd7398450dfeee14)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-04-07 19:14:40 +05:45
Ruslan Kuprieiev 4f8cfc025b gha: remove leftovers 2021-04-06 14:19:25 +03:00
Ruslan Kuprieiev 14f05073ad gha: packages: don't build snap (#5774)
* Delete snap directory

* gha: packages: remove snap
2021-04-06 13:59:09 +03:00
dependabot[bot]andSaugat Pachhai 9e7e01ce9f build(deps): bump pre-commit/action from v2.0.0 to v2.0.2 (#5767)
* build(deps): bump pre-commit/action from v2.0.0 to v2.0.1

Bumps [pre-commit/action](https://github.com/pre-commit/action) from v2.0.0 to v2.0.1.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/v2.0.0...b54880c283dfc80a3fc1baf61807e8ae84471361)

Signed-off-by: dependabot[bot] <[email protected]>

* Update .github/workflows/tests.yaml

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai <[email protected]>
2021-04-05 22:58:34 +05:45
Saugat Pachhai cd9098211f remove pytest-tap (#5713) 2021-03-26 20:43:57 +05:45
dependabot[bot] c3bf0e1cb5 build(deps): bump actions/setup-python from v2.1.4 to v2.2.1 (#5702)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from v2.1.4 to v2.2.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.1.4...3105fb18c05ddd93efea5f9e0bef7a03a6e9e7df)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 18:40:55 +05:45
dependabot[bot] cdb5d668da build(deps): bump actions/upload-release-asset from v1.0.1 to v1.0.2 (#5704)
Bumps [actions/upload-release-asset](https://github.com/actions/upload-release-asset) from v1.0.1 to v1.0.2.
- [Release notes](https://github.com/actions/upload-release-asset/releases)
- [Commits](https://github.com/actions/upload-release-asset/compare/v1.0.1...e8f9f06c4b078e705bd2ea027f0926603fc9b4d5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:48:57 +00:00
dependabot[bot] f1375fd706 build(deps): bump actions/cache from v2 to v2.1.4 (#5699)
Bumps [actions/cache](https://github.com/actions/cache) from v2 to v2.1.4.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2...26968a09c0ea4f3e233fdddbafd1166051a095f6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:40:45 +00:00
dependabot[bot] d04aaf4ae1 build(deps): bump actions/checkout from v2.3.2 to v2.3.4 (#5698)
Bumps [actions/checkout](https://github.com/actions/checkout) from v2.3.2 to v2.3.4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.3.2...5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:40:26 +00:00
dependabot[bot] 191b8c1a6d build(deps): bump codecov/codecov-action from v1.0.13 to v1.3.1 (#5701)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.0.13 to v1.3.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.0.13...fcebab03f26c7530a22baa63f06b3e0515f0c7cd)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:40:17 +00:00
Saugat Pachhai 8c90771388 Add dependabot to update github-actions and python requirements (#5697) 2021-03-26 18:20:10 +05:45
Ruslan Kuprieiev c0671a2d9b gha: don't upload pkg artifacts 2021-03-26 03:18:29 +02:00
Ruslan Kuprieiev f8498d6340 gha: don't build packages for PRs 2021-03-26 03:17:22 +02:00
Ruslan Kuprieiev 49cdc2bea3 build: add osx signing/notarization (#5695)
* build: add osx signing/notarization

* gha: temporarily enable pkg builds
2021-03-26 02:31:32 +02:00
Ruslan Kuprieiev 8bebed93dd build: osxpkg: build lxml (#5687)
lxml wheels are built with an old SDK, which breaks osxpkg notarization:

```
	* error for path "dvc-2.0.7.pkg/dvc-2.0.7.pkg Contents/Payload/usr/local/lib/dvc/lxml/etree.cpython-37m-darwin.so": The binary uses an SDK older than the 10.9 SDK.
```
2021-03-24 20:30:03 +02:00
Ruslan Kuprieiev f2ba67aece build: upgrade pyinstaller to 4.2 (#5682)
* build: upgrade pyinstaller to 4.2

* pyinstaller: hooks: remove hook-googleapiclient.model.py

https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py

* gha: use python 3.8 for pyinstaller
2021-03-24 09:43:05 +02:00
Ruslan Kuprieiev 19cdef1675 build: separate building and packaging (#5675)
* build: separate pyinstaller and fpm/innosetup

* gha: upload deb/rpm/osxpkg/exe artifacts

* build: dynamically generate dvc version
2021-03-23 19:55:44 +02:00
Saugat Pachhai b691b15007 Add a link to help debugging more information regarding the issue (#5601)
* Add a link to help debugging more information regarding the issue

* Update .github/ISSUE_TEMPLATE/bug_report.md
2021-03-11 16:02:53 +02:00
Saugat Pachhai 6c00fedb46 remove flaky-service
Looks like the project is dead: https://github.com/GoogleCloudPlatform/flaky-service.

And, the site https://flaky.dev seems to be dead.
2021-03-11 15:21:12 +05:45
Ruslan Kuprieiev 4a75067f4c github: template: use dvc doctor instead of dvc version 2021-03-05 16:08:46 +02:00
Ruslan Kuprieiev 98a1a92fa3 snap: debug failing tests 2021-03-03 23:11:19 +02:00
Batuhan Taskaya 9ae8bcdf52 gdrive: support service auth with json (#5485)
* gdrive: support service auth with json

* retry logic
2021-02-18 17:10:35 +02:00
Ruslan Kuprieiev 282e83c481 tests: temporarily exclude gdrivefs from extras (#5483)
When trying to publish to pypi, we get:

```
Invalid value for requires_dist. Error: Can't have direct dependency:
"gdrivefs @ git+https://github.com/intake/gdrivefs.git ; extra ==
'tests'"
```
2021-02-17 05:49:34 +02:00
Ruslan Kuprieiev 51988475a4 gha: packages: update for 2.0 2021-02-17 04:38:45 +02:00
Ruslan Kuprieiev c635493104 tests: don't limit the number of pytest workers (#5441)
We used to do this because of gitpython leaks.
2021-02-11 12:27:00 +02:00
Ruslan Kuprieiev 4b938c5104 gha: temporarily disable macos 11
Those workers take a very long time to start up.
2021-02-06 03:07:17 +02:00
Batuhan Taskaya adedd19c52 gha: switch to pre-commit/action for caching (#5389) 2021-02-03 17:14:46 +02:00
Batuhan Taskaya b337396144 cache pip directory on github actions (#5365) 2021-02-01 13:48:03 +01:00
Saugat Pachhai 76c6c54278 Fix backport branch 2020-12-18 12:02:13 +05:45