Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
9,201 Commits 7 Branches 579 Tags
5be23cfd4eb1303342eed3cdf9b7b8ee72691e08
Commit Graph
229 Commits
Author SHA1 Message Date
Saugat Pachhai f9bb38b96f Set timeout for jobs in Github Actions (#6545) 
* Set timeout for jobs in Github Actions

Sometimes tests are getting frozen(most likely an issue with `pytest-xdist`), and sometimes the pip resolver is slow due to some weird constraints set by us or one of our dependencies.

The default timeout is 360 minutes (6hrs), but our test completes on ~18 minutes at maximum, so I think 30mins should be enough to account for occasional CI slowdown.

Also keeping the total job timeout at 45 mins, we usually require just 4 mins of extra time to do installation and setup but giving it 11 more minutes just in case.

Similarly, `lint` completes in <5 mins usually, but giving it a 10 mins timeout.

This should give us early feedback on the issue.

* Update setup.cfg

* Update setup.cfg
 2021-09-07 14:08:22 +05:45
Ruslan Kuprieiev 6287b13ba3 gha: allow running tests on other branches (#6520) 
Simplifies development for all contributors.
 2021-09-02 17:55:54 +03:00
Peter Rowlands (변기호) a69edcc2c8 tpi: lazily import python_terraform (#6526) 
* machine: lazy load tpi/python_terraform only when needed

* remove terraform from package installers

* update tpi test imports
 2021-09-02 17:09:31 +03:00
Ruslan Kuprieiev 5d7476a539 gha: packages: don't forget terraform 2021-08-27 23:43:28 +03:00
Ruslan Kuprieiev 341ab4fe9b gha: packages: don't forget terraform 2021-08-27 23:38:21 +03:00
Peter Rowlands (변기호) 4ee98299ee dvc machine: implement initial terraform support (#6462) 
* tpi: add naive terraform-provider-iterative wrapper

* executor: add naive backend/terraform wrapper

* setup: add dvc[terraform] install extras

* config: add feature.executor feature flag

* fix named executor config sections

* executor: add poc add/modify/remove/init/destroy commands

* relax python-terraform requirement

* ci: install terraform extras

* tpi: support .tf.json format

* s/executor/machine

* terraform: use separate dirs per machine

* s/init/create/

* reserve local machine name

* add initial ssh skeleton

* add initial machine unit tests

* add initial machine command tests

* add tests for add/remove

* enable feature by default in test env

* close temp pemfile object so it can be used on windows

* cleanup pemfile syntax
 2021-08-27 12:46:24 +03:00
dependabot[bot] ceb0964fcf build(deps): Bump codecov/codecov-action from 2.0.2 to 2.0.3 (#6487) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v2.0.2...v2.0.3)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-25 08:22:52 +05:45
Saugat Pachhai 1bb7f39e14 add support for Python 3.10 (#6422) 
* add support for Python 3.10

pyarrow is not available, so we are skipping tests for it.
And, restricting `hdfs` extras to be not installed in Python3.10.

Pathlib removed `_init` method and `init` parameter, so just working
around that. It's not a big change.

* skip windows on 3.10; update pip/wheel/setuptools during install

pywin32 is not yet available for 3.10 Python, so skipping those for now

* install libgit2 with Python3.10 as pygit2 wheel is not available

* install libgit2

* only run on macOS, libgit2-1.1 not available on ubuntu
 2021-08-16 16:24:28 +05:45
dependabot[bot] b294bca383 build(deps): bump codecov/codecov-action from 2.0.1 to 2.0.2 (#6361) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2.0.1 to 2.0.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v2.0.1...v2.0.2)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-07-28 10:21:10 +03:00
dependabot[bot] 005bca1da6 build(deps): bump codecov/codecov-action from 1.5.2 to 2.0.1 (#6334) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1.5.2 to 2.0.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.5.2...v2.0.1)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-07-20 14:06:50 +03:00
GaoandRuslan Kuprieiev 93f36d9351 oss: migrate to ossfs (#6307) 
* Migrate to ossfs

fix 6177

* Add verification to object stored on oss

* Change the oss remote's dependency.

pass all of the tests

* Performance optimaztion and ossfs version update

* Remove the argument to set the resumable get/put

1. Experiments shows resumable get/put has little influnence on performance.
2. Just use resumable ones as default

* Use a new api for performance optimization

* Without filesize progress bar version

* With progress bar version

* Bump and add comments

1. Bump ossfs from 2021.6.2 to 2021.7.0
2. add some comments to the verification trigger

* Change the name of kwrags following parent class.

* Bump to new ossfs and rewrite upload_fobj

Because some problem with oss-emulater
1. bump to ossfs 2021.7.1
2. rewrite upload_fobj

* Optimization on rm_files

* Some optimization follow the old version.

* Update dvc/fs/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/objects/db/oss.py

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* bump into ossfs 2021.7.2

* Bump into ossfs 2021.7.3OC

* Switch to real_oss_server

* Switch to a real oss server

* Use the oss bucket located in US.

* Remove some emulator related code

* Add envs to ci

* remove some oss pytest XFAIL

Because we didn't use emulator, we can remove some pytest XFAILs

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2021-07-13 15:27:34 +03:00
Saugat Pachhai b7c6df0ece Create CODEOWNERS (#6213) 2021-06-23 11:56:37 +05:45
dependabot[bot] 57c728269d build(deps): bump codecov/codecov-action from 1.5.0 to 1.5.2 (#6139) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 1.5.0 to 1.5.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.5.0...v1.5.2)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-09 23:30:31 +05:45
dependabot[bot] bf5c28b71a build(deps): bump rtCamp/action-slack-notify from 2.1.3 to 2.2.0 (#6128) 
Bumps [rtCamp/action-slack-notify](https://github.com/rtCamp/action-slack-notify) from 2.1.3 to 2.2.0.
- [Release notes](https://github.com/rtCamp/action-slack-notify/releases)
- [Commits](https://github.com/rtCamp/action-slack-notify/compare/v2.1.3...v2.2.0)

---
updated-dependencies:
- dependency-name: rtCamp/action-slack-notify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-07 14:04:32 +05:45
Batuhan Taskaya 0e3db0d323 ci: move to ubuntu-18 from ubuntu-16 (#6101) 2021-06-02 17:31:28 +03:00
dependabot[bot] efda015de4 build(deps): bump actions/cache from 2.1.5 to 2.1.6 (#6073) 
Bumps [actions/cache](https://github.com/actions/cache) from 2.1.5 to 2.1.6.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.5...v2.1.6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-28 14:43:23 +05:45
dependabot[bot] 0ca08d2fea build(deps): bump rtCamp/action-slack-notify from 2 to 2.1.3 (#5993) 
Bumps [rtCamp/action-slack-notify](https://github.com/rtCamp/action-slack-notify) from 2 to 2.1.3.
- [Release notes](https://github.com/rtCamp/action-slack-notify/releases)
- [Commits](https://github.com/rtCamp/action-slack-notify/compare/v2...v2.1.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-12 13:06:50 +05:45
dependabot[bot] 14e1511eee build(deps): bump actions/setup-ruby from 1 to 1.1.3 (#5994) 
Bumps [actions/setup-ruby](https://github.com/actions/setup-ruby) from 1 to 1.1.3.
- [Release notes](https://github.com/actions/setup-ruby/releases)
- [Commits](https://github.com/actions/setup-ruby/compare/v1...v1.1.3)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-12 13:06:40 +05:45
Saugat Pachhai eabf6ddf27 run gdrive tests in a single job, run all on schedule (#5974) 2021-05-06 17:09:34 +05:45
Saugat Pachhai c2f1a1e162 Notify CI failures to Slack (#5970) 2021-05-05 18:37:33 +05:45
dependabot[bot] c2734a2cd7 build(deps): bump codecov/codecov-action from v1.4.1 to v1.5.0 (#5957) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.1 to v1.5.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.1...a1ed4b322b4b38cb846afb5a0ebfa17086917d27)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-04 17:45:01 +03:00
Ruslan Kuprieiev c9a748421a dependabot: add "maintenance" label (#5898) 2021-04-29 16:59:53 +03:00
dependabot[bot] af6921e9ac build(deps): bump pre-commit/action from v2.0.2 to v2.0.3 (#5856) 
Bumps [pre-commit/action](https://github.com/pre-commit/action) from v2.0.2 to v2.0.3.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/v2.0.2...9b88afc9cd57fd75b655d5c71bd38146d07135fe)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-26 14:08:40 +03:00
dependabot[bot] 8ac26b58ee build(deps): bump codecov/codecov-action from v1.4.0 to v1.4.1 (#5857) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.4.0 to v1.4.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.4.0...967e2b38a85a62bd61be5529ada27ebc109948c2)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-22 13:50:46 +03:00
Ruslan Kuprieiev 3472cb1a6a tests: don't set sensitive env secrets for all steps (#5823) 
Due to https://about.codecov.io/security-update/ , there is a potential that we've leaked these. This PR takes a more granular approach when setting secret env vars so that this doesn't happen in the future.

We'll change these secrets after this PR is merged.
 2021-04-19 16:39:37 +03:00
dependabot[bot] 48890c22b7 build(deps): bump codecov/codecov-action from v1.3.2 to v1.4.0 (#5838) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.3.2 to v1.4.0.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.3.2...0e28ff86a50029a44d10df6ed4c308711925a6a8)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-17 11:20:46 +05:45
dependabot[bot] 2472c04873 build(deps): bump actions/cache from v2.1.4 to v2.1.5 (#5805) 
Bumps [actions/cache](https://github.com/actions/cache) from v2.1.4 to v2.1.5.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2.1.4...1a9e2138d905efd099035b49d8b7a3888c653ca8)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-13 14:23:05 +03:00
dependabot[bot] c71a576a7a build(deps): bump actions/setup-python from v2.2.1 to v2.2.2 (#5804) 
Bumps [actions/setup-python](https://github.com/actions/setup-python) from v2.2.1 to v2.2.2.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.2.1...dc73133d4da04e56a135ae2246682783cc7c7cb6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-13 14:22:57 +03:00
dependabot[bot] 02d9684c33 build(deps): bump codecov/codecov-action from v1.3.1 to v1.3.2 (#5770) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.3.1 to v1.3.2.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.3.1...9b0b9bbe2c64e9ed41413180dd7398450dfeee14)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-04-07 19:14:40 +05:45
Ruslan Kuprieiev 4f8cfc025b gha: remove leftovers 2021-04-06 14:19:25 +03:00
Ruslan Kuprieiev 14f05073ad gha: packages: don't build snap (#5774) 
* Delete snap directory

* gha: packages: remove snap
 2021-04-06 13:59:09 +03:00
dependabot[bot]andSaugat Pachhai 9e7e01ce9f build(deps): bump pre-commit/action from v2.0.0 to v2.0.2 (#5767) 
* build(deps): bump pre-commit/action from v2.0.0 to v2.0.1

Bumps [pre-commit/action](https://github.com/pre-commit/action) from v2.0.0 to v2.0.1.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/v2.0.0...b54880c283dfc80a3fc1baf61807e8ae84471361)

Signed-off-by: dependabot[bot] <[email protected]>

* Update .github/workflows/tests.yaml

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai <[email protected]>
 2021-04-05 22:58:34 +05:45
Saugat Pachhai cd9098211f remove pytest-tap (#5713) 2021-03-26 20:43:57 +05:45
dependabot[bot] c3bf0e1cb5 build(deps): bump actions/setup-python from v2.1.4 to v2.2.1 (#5702) 
Bumps [actions/setup-python](https://github.com/actions/setup-python) from v2.1.4 to v2.2.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.1.4...3105fb18c05ddd93efea5f9e0bef7a03a6e9e7df)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-26 18:40:55 +05:45
dependabot[bot] cdb5d668da build(deps): bump actions/upload-release-asset from v1.0.1 to v1.0.2 (#5704) 
Bumps [actions/upload-release-asset](https://github.com/actions/upload-release-asset) from v1.0.1 to v1.0.2.
- [Release notes](https://github.com/actions/upload-release-asset/releases)
- [Commits](https://github.com/actions/upload-release-asset/compare/v1.0.1...e8f9f06c4b078e705bd2ea027f0926603fc9b4d5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-26 12:48:57 +00:00
dependabot[bot] f1375fd706 build(deps): bump actions/cache from v2 to v2.1.4 (#5699) 
Bumps [actions/cache](https://github.com/actions/cache) from v2 to v2.1.4.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2...26968a09c0ea4f3e233fdddbafd1166051a095f6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-26 12:40:45 +00:00
dependabot[bot] d04aaf4ae1 build(deps): bump actions/checkout from v2.3.2 to v2.3.4 (#5698) 
Bumps [actions/checkout](https://github.com/actions/checkout) from v2.3.2 to v2.3.4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.3.2...5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-26 12:40:26 +00:00
dependabot[bot] 191b8c1a6d build(deps): bump codecov/codecov-action from v1.0.13 to v1.3.1 (#5701) 
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.0.13 to v1.3.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.0.13...fcebab03f26c7530a22baa63f06b3e0515f0c7cd)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-26 12:40:17 +00:00
Saugat Pachhai 8c90771388 Add dependabot to update github-actions and python requirements (#5697) 2021-03-26 18:20:10 +05:45
Ruslan Kuprieiev c0671a2d9b gha: don't upload pkg artifacts 2021-03-26 03:18:29 +02:00
Ruslan Kuprieiev f8498d6340 gha: don't build packages for PRs 2021-03-26 03:17:22 +02:00
Ruslan Kuprieiev 49cdc2bea3 build: add osx signing/notarization (#5695) 
* build: add osx signing/notarization

* gha: temporarily enable pkg builds
 2021-03-26 02:31:32 +02:00
Ruslan Kuprieiev 8bebed93dd build: osxpkg: build lxml (#5687) 
lxml wheels are built with an old SDK, which breaks osxpkg notarization:

```
	* error for path "dvc-2.0.7.pkg/dvc-2.0.7.pkg Contents/Payload/usr/local/lib/dvc/lxml/etree.cpython-37m-darwin.so": The binary uses an SDK older than the 10.9 SDK.
```
 2021-03-24 20:30:03 +02:00
Ruslan Kuprieiev f2ba67aece build: upgrade pyinstaller to 4.2 (#5682) 
* build: upgrade pyinstaller to 4.2

* pyinstaller: hooks: remove hook-googleapiclient.model.py

https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py

* gha: use python 3.8 for pyinstaller
 2021-03-24 09:43:05 +02:00
Ruslan Kuprieiev 19cdef1675 build: separate building and packaging (#5675) 
* build: separate pyinstaller and fpm/innosetup

* gha: upload deb/rpm/osxpkg/exe artifacts

* build: dynamically generate dvc version
 2021-03-23 19:55:44 +02:00
Saugat Pachhai b691b15007 Add a link to help debugging more information regarding the issue (#5601) 
* Add a link to help debugging more information regarding the issue

* Update .github/ISSUE_TEMPLATE/bug_report.md
 2021-03-11 16:02:53 +02:00
Saugat Pachhai 6c00fedb46 remove flaky-service 
Looks like the project is dead: https://github.com/GoogleCloudPlatform/flaky-service.

And, the site https://flaky.dev seems to be dead.
 2021-03-11 15:21:12 +05:45
Ruslan Kuprieiev 4a75067f4c github: template: use dvc doctor instead of dvc version 2021-03-05 16:08:46 +02:00
Ruslan Kuprieiev 98a1a92fa3 snap: debug failing tests 2021-03-03 23:11:19 +02:00
Batuhan Taskaya 9ae8bcdf52 gdrive: support service auth with json (#5485) 
* gdrive: support service auth with json

* retry logic
 2021-02-18 17:10:35 +02:00