Commit Graph
6443 Commits
Author SHA1 Message Date
dependabot[bot]andSaugat Pachhai 5ca813024f build(deps): bump moto from 1.3.16.dev122 to 2.0.2 (#5709)
* build(deps): bump moto from 1.3.16.dev122 to 2.0.3.dev12

Bumps [moto](https://github.com/spulec/moto) from 1.3.16.dev122 to 2.0.3.dev12.
- [Release notes](https://github.com/spulec/moto/releases)
- [Changelog](https://github.com/spulec/moto/blob/master/CHANGELOG.md)
- [Commits](https://github.com/spulec/moto/commits)

Signed-off-by: dependabot[bot] <[email protected]>

* Update test_requirements.txt

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai <[email protected]>
2021-03-26 20:05:32 +05:45
Batuhan Taskaya 042b1c7f7d pull: skip isdir() checks on downloading files (#5696)
* pull: use download_file

* tree: implement filesystem.download_file
2021-03-26 15:22:45 +02:00
dependabot[bot] 61e470b85e build(deps): bump jaraco-windows from 3.9.2 to 5.5.0 (#5700)
Bumps [jaraco-windows](https://github.com/jaraco/jaraco.windows) from 3.9.2 to 5.5.0.
- [Release notes](https://github.com/jaraco/jaraco.windows/releases)
- [Changelog](https://github.com/jaraco/jaraco.windows/blob/main/CHANGES.rst)
- [Commits](https://github.com/jaraco/jaraco.windows/compare/3.9.2...v5.5.0)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 18:41:44 +05:45
dependabot[bot] c3bf0e1cb5 build(deps): bump actions/setup-python from v2.1.4 to v2.2.1 (#5702)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from v2.1.4 to v2.2.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.1.4...3105fb18c05ddd93efea5f9e0bef7a03a6e9e7df)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 18:40:55 +05:45
dependabot[bot] cdb5d668da build(deps): bump actions/upload-release-asset from v1.0.1 to v1.0.2 (#5704)
Bumps [actions/upload-release-asset](https://github.com/actions/upload-release-asset) from v1.0.1 to v1.0.2.
- [Release notes](https://github.com/actions/upload-release-asset/releases)
- [Commits](https://github.com/actions/upload-release-asset/compare/v1.0.1...e8f9f06c4b078e705bd2ea027f0926603fc9b4d5)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:48:57 +00:00
dependabot[bot] 81d0d7aae7 build(deps): bump beautifulsoup4 from 4.4.0 to 4.9.3 (#5703)
Bumps [beautifulsoup4](http://www.crummy.com/software/BeautifulSoup/bs4/) from 4.4.0 to 4.9.3.

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:45:32 +00:00
dependabot[bot] f1375fd706 build(deps): bump actions/cache from v2 to v2.1.4 (#5699)
Bumps [actions/cache](https://github.com/actions/cache) from v2 to v2.1.4.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2...26968a09c0ea4f3e233fdddbafd1166051a095f6)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:40:45 +00:00
dependabot[bot] d04aaf4ae1 build(deps): bump actions/checkout from v2.3.2 to v2.3.4 (#5698)
Bumps [actions/checkout](https://github.com/actions/checkout) from v2.3.2 to v2.3.4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.3.2...5a4ac9002d0be2fb38bd78e4b4dbde5606d7042f)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:40:26 +00:00
dependabot[bot] 191b8c1a6d build(deps): bump codecov/codecov-action from v1.0.13 to v1.3.1 (#5701)
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from v1.0.13 to v1.3.1.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v1.0.13...fcebab03f26c7530a22baa63f06b3e0515f0c7cd)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-03-26 12:40:17 +00:00
Saugat Pachhai 8c90771388 Add dependabot to update github-actions and python requirements (#5697) 2021-03-26 18:20:10 +05:45
Saugat Pachhai 1931a6ec1e Move tests requirements to test_requirements.txt (#5674)
Plan is to add dependabot to update those packages required for testing
2021-03-26 14:13:43 +02:00
Ruslan Kuprieiev 4ffa0675b4 build: fix --notarize 2021-03-26 03:47:34 +02:00
Ruslan Kuprieiev c0671a2d9b gha: don't upload pkg artifacts 2.0.11 2021-03-26 03:18:29 +02:00
Ruslan Kuprieiev f8498d6340 gha: don't build packages for PRs 2021-03-26 03:17:22 +02:00
Ruslan Kuprieiev 27b589f1b9 dvc: bump to 2.0.11 2021-03-26 03:06:36 +02:00
Ruslan Kuprieiev 49cdc2bea3 build: add osx signing/notarization (#5695)
* build: add osx signing/notarization

* gha: temporarily enable pkg builds
2021-03-26 02:31:32 +02:00
Ruslan Kuprieiev a3a55339ab dvc: bump to 2.0.10 2.0.10 2021-03-25 02:21:52 +02:00
Ruslan Kuprieiev 8abffbf68b scripts: fpm: install zsh competion to install / "share"
Currently we create /usr/share/... which is not allowed by macos:

> the package is trying to install content to the system volume
2021-03-25 02:21:11 +02:00
Ruslan Kuprieiev 2e76dabbaa dvc: bump to 2.0.9 2.0.9 2021-03-24 20:30:48 +02:00
Ruslan Kuprieiev 8bebed93dd build: osxpkg: build lxml (#5687)
lxml wheels are built with an old SDK, which breaks osxpkg notarization:

```
	* error for path "dvc-2.0.7.pkg/dvc-2.0.7.pkg Contents/Payload/usr/local/lib/dvc/lxml/etree.cpython-37m-darwin.so": The binary uses an SDK older than the 10.9 SDK.
```
2021-03-24 20:30:03 +02:00
Ruslan Kuprieiev 58a5442d71 dvc: bump to 2.0.8 2.0.8 2021-03-24 19:20:04 +02:00
Saugat Pachhai 76ece8c64a plots: quote and resolve paths properly (#5670)
This commit fixes 3 things:
1. The path printed on the terminal is properly is quoted.  This was
problematic when the output is piped to `xdg-open` if the path contained
spaces or quotes or special characters.

2. The path is resolved. This is not an issue at all, but makes the
output nicer.

3. The path with which the browser is opened with uses relative path.
This is done to solve issue in WSL, that the absolute path is not
accessible outside the WSL for host programs as the linux filesystem
is available on `/wsl$` path.
2021-03-24 20:27:24 +05:45
Saugat Pachhai 8b6d655ebe use global UI object for printing (#5653) 2021-03-24 17:22:17 +05:45
Ruslan Kuprieiev f2ba67aece build: upgrade pyinstaller to 4.2 (#5682)
* build: upgrade pyinstaller to 4.2

* pyinstaller: hooks: remove hook-googleapiclient.model.py

https://github.com/pyinstaller/pyinstaller-hooks-contrib/blob/master/src/_pyinstaller_hooks_contrib/hooks/stdhooks/hook-googleapiclient.model.py

* gha: use python 3.8 for pyinstaller
2021-03-24 09:43:05 +02:00
Ruslan Kuprieiev 19cdef1675 build: separate building and packaging (#5675)
* build: separate pyinstaller and fpm/innosetup

* gha: upload deb/rpm/osxpkg/exe artifacts

* build: dynamically generate dvc version
2021-03-23 19:55:44 +02:00
Saugat Pachhai dc3fe6fbec Update pylint-pytest to 1.0.3 (#5673) 2021-03-23 15:28:33 +05:45
Ruslan Kuprieiev 619c55d399 dvc: bump to 2.0.7 2.0.7 2021-03-22 15:29:52 +02:00
Alex Ford b3bfa346c3 Using existing logging level if defined in addLoggingLevel (#5632)
* Use existing level if defined in addLoggingLevel

Fixes https://github.com/iterative/dvc/issues/4546

* Add addLoggingLevel test

* Format fix.
2021-03-22 15:15:16 +02:00
Saugat Pachhai 0a8927e3d8 diff: clear calculated renamed files from inverted hashmap (#5652)
* diff: clear calculated renamed files from inverted hashmap

We were trying to reuse the first encounter of file from inverted hashmap again and again, which was throwing ValueError

* refactor

* remove ordering

* add tests
2021-03-22 15:13:52 +02:00
Batuhan Taskaya 35597ed83d fsspec: skip instance cache on all filesystems (#5668) 2021-03-22 14:36:21 +02:00
Batuhan Taskaya 21fc06e947 fsspec: support raw buckets on path splitting (#5658) 2021-03-22 02:01:06 +02:00
Saugat Pachhai f2b216261e move Repo.diff related test to diff (#5654) 2021-03-19 16:58:21 +05:45
Batuhan Taskaya 6026573860 update: use unresolved path with --to-remote calls (#5631) 2021-03-17 16:48:00 +02:00
Ruslan Kuprieiev dcc55aa645 dvc: bump to 2.0.6 2.0.6 2021-03-17 00:01:10 +02:00
Batuhan Taskaya 862e18488c gs: process etag when it is available (#5634) 2021-03-16 14:08:59 +02:00
Saugat Pachhai 0b810fb256 Advertise Python 3.9 support in classifiers 2021-03-16 13:37:37 +05:45
Helio MachadoandPeter Rowlands 4ed55d0051 build: add pyinstaller hook for googleapiclient (#5619)
* build: add pyinstaller hook for googleapiclient

The Google API Discovery Service JSON files from
googleapiclient/discovery_cache/documents were being
excluded due to a bug and, particularly, drive.v2.json
is essential for our gdrive backend. Now we're explicitly
asking PyInstaller to embed these files on the generated packages.

Fixes #5618

* Suppress linter error for unrecognized imports

Co-authored-by: Peter Rowlands (변기호) <[email protected]>

* Apply black style after manual code review

Co-authored-by: Peter Rowlands (변기호) <[email protected]>
2021-03-14 12:56:49 +02:00
nik123 6ac7def6f1 setup: remove duplicate 'pytest-cov' dependency (#5617)
* setup: remove duplicate 'pytest-cov' dependency

* setup: returned pytest-cov version
2021-03-14 14:08:11 +05:45
Peter Rowlands (변기호) 8c1d46e502 checkpoints: completely remove checkpoint outs on exp run --reset (#5586)
* checkpoints: remove checkpoint outs on --reset

* exp run: no longer prune/reset lockfiles

* update tests

* make --queue imply --reset unless --rev is provided
2021-03-13 01:29:50 +09:00
Batuhan Taskaya 11b0581536 import-url: support importing empty directories on object storage services (#5536)
* import-url: support importing empty directories on object storage services

Empty directories are basically files with a trailing slash at the end of their names,
with empty contents on object service providers (such as azure, google storage). This
patch implements the `is_empty()` method for the `FSSpecWrapper` to determining whether
a path is empty (directory or a file) and also tweaks `.isdir()` in order to see empty
directories.

The base class is only used by azure and gcsfs for now, but if we go on the route of
migrating normal providers (like proper filesystems, not just object storages), we might
need to do these tweaks on a subclass (like `ObjectStorageFSSpecWrapper` etc), but I
don't think it is neccesary as of now.

The `is_empty` method is already there, and also used in the importing part to
warn users that they are importing something empty (https://github.com/iterative/dvc/blob/e385e59a098cd661ea1ea4257c9a5735c5c32bc6/dvc/output/base.py#L252-L254)
so this is not a new method. Just for the reference, as far as I can locate there
is no variant of this method in the fsspec.

* tests: revert custom mkdir for cloud storages

* tests: allign assingments

* add some comments
2021-03-12 18:11:08 +02:00
Batuhan Taskaya abcb5b2f6f gs: disable consistency checker (#5613) 2021-03-12 14:04:54 +02:00
Peter Rowlands (변기호) d365e6d5b4 exp run: fix issue where duplicate workspace runs would incorrectly conflict (#5611)
* exp run: gracefully handle duplicate workspace runs

* exp run: fix issue where errors could be logged twice

* move ref conflict check into its own function
2021-03-12 20:13:49 +09:00
Peter Rowlands (변기호) bd470e9586 stage add: unsuppress -c/--checkpoints help text (#5599)
* stage add: unsuppress -c/--checkpoints help text

* review fix
2021-03-12 14:47:12 +09:00
Peter Rowlands (변기호) fe4cb9c439 exp run: do nothing if result matches parent (HEAD) (#5600) 2021-03-12 14:09:03 +09:00
Ruslan Kuprieiev 2db55032c4 objects: file: remove unused load() method (#5610) 2021-03-12 05:35:48 +02:00
Peter Rowlands (변기호) 686898240c Revert "repro/exp run: use git status when reminding to git-track files (#5544)" (#5596)
This reverts commit ce2d7c65db.
2021-03-12 11:50:13 +09:00
Saugat Pachhai d245008987 plots: support opening plots file directly in the browser (#5584)
* plots: support opening plots file directly in the browser

* add tests

* fix path on windows tests
2021-03-12 04:23:07 +02:00
Ruslan Kuprieiev 29ad87ebe2 objects: split into separate files (#5609) 2021-03-12 04:12:46 +02:00
Ruslan Kuprieiev 4af3b3d20d dvc: refactor objects (#5607)
* dvc: get rid of DirInfo

DirInfo was created before we had proper objects, as a way to have a
structure that represents a staged directory. Right now we have a
dedicated Tree object that is specifically meant for that.

As a result, HashInfo no longer needs to have dir_info and we no longer
need `get_hash/get_dir_hash` helpers.

* objects: move Tree into tree.py
2021-03-12 01:10:34 +02:00
Saugat Pachhai b691b15007 Add a link to help debugging more information regarding the issue (#5601)
* Add a link to help debugging more information regarding the issue

* Update .github/ISSUE_TEMPLATE/bug_report.md
2021-03-11 16:02:53 +02:00