Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,932 Commits 7 Branches 579 Tags
8580e66d6044997b1e45ca891d246d3a11d44605
Commit Graph
6932 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Paweł Redzyński 8580e66d60 gha: move cancelaltion into other workflows (#6771) 2021-10-08 18:03:07 +03:00
Saugat Pachhai c24ae2b0c3 exp init: show defaults in help message (#6763) 2021-10-08 20:14:12 +05:45
Saugat Pachhai 4e17279440 exp init: add config support (#6761) 2021-10-08 20:14:01 +05:45
Batuhan Taskaya 50cdf4f213 fsspec: normalize checksum() to return string (#6767) 2021-10-08 16:57:37 +03:00
Saugat Pachhai f628f670c9 advertise Python 3.10 support (#6772) 
* advertise Python 3.10 support

Pygit2 does not have windows wheels yet and no pyarrow
for 3.10. But these platform-dependent things and
remote-dependent things should not stop us from
advertising support as we are pure-python based
package and we cannot claim support for all of our
dependencies.

* set pygit2 for Python 3.6 to <1.7
 2021-10-08 15:50:05 +03:00
Batuhan Taskaya d7489bb65f hdfs: enable mocked hdfs tests by default (#6766) 2021-10-08 12:42:41 +02:00
dependabot[bot]andPaweł Redzyński 5a4f4d3a8e build(deps): Bump filelock from 3.2.0 to 3.3.0 (#6737) 
* build(deps): Bump filelock from 3.2.0 to 3.3.0

Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.2.0...3.3.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* tests: remotes: ignore abstract class instantiation for filelock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Paweł Redzyński <[email protected]>
 2021-10-08 15:36:33 +05:45
dependabot[bot] 2c24b24562 build(deps-dev): Bump types-requests from 2.25.6 to 2.25.9 (#6770) 
Bumps [types-requests](https://github.com/python/typeshed) from 2.25.6 to 2.25.9.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-08 14:59:09 +05:45
dependabot[bot] df9f76c2c0 build(deps-dev): Bump types-paramiko from 0.1.9 to 2.7.1 (#6769) 
Bumps [types-paramiko](https://github.com/python/typeshed) from 0.1.9 to 2.7.1.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-paramiko
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-08 14:58:19 +05:45
dependabot[bot] 0db3ece060 build(deps-dev): Bump types-toml from 0.1.5 to 0.10.0 (#6768) 
Bumps [types-toml](https://github.com/python/typeshed) from 0.1.5 to 0.10.0.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-toml
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-08 14:58:08 +05:45
Saugat Pachhai 7c437cc349 make dependabot updates work (#6760) 
Dependabot was throwing Illformed requirements `"< '3.10'"` for the
hdfs. But I noticed that it did accept other requirements where we
do similar. So it seems the indentation fixes it.

Even then, dependabot did not update any dependencies. I removed
setup.py file, after which immediately it started creating PRs.
So, I recknoed that it must have been due to some mechanism
in place from dependabot to disallow execution, so I tried to
remove few blocks of code from setup.py. And, I was eventually
able to figure out that it was from the block where we import version.

So, I added a try-except block and it magically again worked. :)

Let's see how it goes.
 2021-10-08 14:46:47 +05:45
Saugat Pachhai 3ed5efceb7 move defaults to the class level (#6762) 2021-10-08 12:18:28 +05:45
dependabot[bot]andPaweł Redzyński d33df60e41 build(deps): Bump dvclive from 0.3.0 to 0.4.0 (#6753) 
* build(deps): Bump dvclive from 0.3.0 to 0.4.0

Bumps [dvclive](https://github.com/iterative/dvclive) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/iterative/dvclive/releases)
- [Commits](https://github.com/iterative/dvclive/compare/0.3.0...0.4.0)

---
updated-dependencies:
- dependency-name: dvclive
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* live: adjust tests to dvclive==0.4.0

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Paweł Redzyński <[email protected]>
 2021-10-08 07:02:54 +05:45
Saugat Pachhai c5415342d0 move requirements to setup.cfg (#6758) 
* move requirements to setup.cfg

* hash based on setup.cfg

* do not break webhdfs installation
 2021-10-08 07:00:14 +05:45
Paweł Redzyński dd86076ba6 gha: cancel previous workflows (#6756) 2021-10-07 17:59:44 +03:00
Ruslan Kuprieiev 7dc3859400 pyinstaller: hooks: remove webhdfs 2021-10-07 17:30:22 +03:00
Batuhan Taskaya 4f3b28e268 webhdfs: migrate to fsspec (#6662) 
* webhdfs: migrate to fsspec

* skip the tests that can't use hdfs
 2021-10-07 17:02:31 +03:00
Ruslan Kuprieiev 32f435c47f gha: benchmarks: fix pytest flags (#6749) 
Kudos @skshetry for catching.
 2021-10-06 17:07:41 +03:00
Batuhan Taskaya 054feae778 hdfs: migrate to fsspec (#6604) 
* hdfs: migrate to fsspec

* normalize windows paths when delegating back to fsspec
 2021-10-06 15:44:27 +03:00
Saugat Pachhai 2abe515391 Pin dvclive to 0.3.0 2021-10-06 14:28:36 +05:45
eggqq007andZhenyu Gao 1eb89a2dc0 command: Should not consume 'print' to output default remote info (#6650) 
1. should not consume 'print' to output default remote info,
   otherwise the --quiet would not work.

Co-authored-by: Zhenyu Gao <[email protected]>
 2021-10-06 09:27:47 +05:45
eggqq007andZhenyu Gao 9c300a0a1c command: output prettify json in tty (#6743) 
1. update diff.py,experiments.py,metrics.py,params.py,status.py to
print prettify json
2. rewrite write_json in __init__.py to accept default function
3. add new unit test for write_json in tests/unit/ui/test_console.py
4. Fix #6712

Co-authored-by: Zhenyu Gao <[email protected]>
 2021-10-05 20:19:56 +05:45
Ruslan Kuprieiev 7b68a6f0ca gha: run benchmarks on PRs (#6733) 2021-10-05 14:18:43 +03:00
Saugat Pachhai ee823a59b9 Run on Python3.10 (#6745) 2021-10-05 16:46:53 +05:45
David de la Iglesia CastroandSaugat Pachhai 16f404b16a Fix dvclive running subdir (#6740) 
* Fix dvclive running subdir

* Apply suggestions from code review

Co-authored-by: Saugat Pachhai <[email protected]>

* Applied suggestions

* Update tests/func/test_live.py

Co-authored-by: Saugat Pachhai <[email protected]>
 2021-10-05 13:16:33 +05:45
Saugat Pachhai 0eda4bd91c exp init: only ask for that are not provided in an interactive mode (#6739) 2021-10-04 14:33:08 +05:45
Saugat Pachhai a098a50c8d move exp init logic to dvc.repo.experiments.init (#6738) 2021-10-04 12:26:34 +05:45
dependabot[bot] f57a378c0c build(deps): Bump jaraco-windows from 5.6.0 to 5.7.0 (#6735) 
Bumps [jaraco-windows](https://github.com/jaraco/jaraco.windows) from 5.6.0 to 5.7.0.
- [Release notes](https://github.com/jaraco/jaraco.windows/releases)
- [Changelog](https://github.com/jaraco/jaraco.windows/blob/main/CHANGES.rst)
- [Commits](https://github.com/jaraco/jaraco.windows/compare/v5.6.0...v5.7.0)

---
updated-dependencies:
- dependency-name: jaraco-windows
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-04 10:12:47 +05:45
dependabot[bot] 3e0a50e52e build(deps): Bump pytest-cov from 2.12.1 to 3.0.0 (#6736) 
Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov) from 2.12.1 to 3.0.0.
- [Release notes](https://github.com/pytest-dev/pytest-cov/releases)
- [Changelog](https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest-cov/compare/v2.12.1...v3.0.0)

---
updated-dependencies:
- dependency-name: pytest-cov
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-04 09:31:27 +05:45
Guillaume COTER ec72866eb9 Bump typing_extensions to >=3.7.4 (#6731) (#6732) 2021-10-03 20:10:28 +05:45
Batuhan Taskaya e53d390ff9 http: explicitly ignore all http-related errors (from server side to client connection drops) (#6728) 2021-10-02 20:42:23 +03:00
Batuhan Taskaya dd330ac592 setup: bump fsspec/s3fs/gcsfs (#6727) 2021-10-02 18:54:09 +02:00
pwoolvett 4ccb712b6d build(setup): avoid leaking tests package into site-packages (#6585) 
Signed-off-by: Pablo Woolvett <[email protected]>
 2021-10-02 16:46:45 +03:00
Saugat Pachhai ecee2d3522 Move static metadata to setup.cfg (#6581) 2021-10-02 16:08:30 +03:00
Anas ChetouanCHETOUANpre-commit-ci[bot]CHETOUAN
 		20bb6ddf3c doctor: fixes unknown cache directory error on some windows mapped network shares (#6364) 
* doctor: fixes unknown cache director error on some windows mapped network shares

On Windows, resolve calls os.path._getfinalpathname, which calls GetFinalPathNameByHandle.
This starts from the final resolved path of the file in the NT namespace (e.g. \Device\Mup\server\share...)
and works backward as required to get the VOLUME_NAME_DOS. If the final path is a UNC share (i.e. \Device\Mup),
We fix this by calling resolve() also on the partition mountpoints
We also take the opportunity to make checks in case a PermissionError is returned to us

Fixes github.com/iterative/dvc/issues/6305

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* doctor: fixes unknown cache director error on some windows mapped network shares

On Windows, resolve calls os.path._getfinalpathname, which calls GetFinalPathNameByHandle.
This starts from the final resolved path of the file in the NT namespace (e.g. \Device\Mup\server\share...)
and works backward as required to get the VOLUME_NAME_DOS. If the final path is a UNC share (i.e. \Device\Mup),
We fix this by calling resolve() also on the partition mountpoints
We also take the opportunity to make checks in case a PermissionError is returned to us

Fixes #6305

Co-authored-by: CHETOUAN <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: CHETOUAN <[email protected]>
 2021-10-02 16:06:42 +03:00
Saugat Pachhai e58fcd864b [WIP] exp init: introduce --with={default,live} flag (#6716) 
* exp init: introduce `--with={default,live}` flag

This flag will select the different mode of outputs of stage.
With `--with=live`, it'll enable `live` setting in the stage.
On interactive mode, it'll always ask user if they want to
use live (unless `--with=live` is specified), and then it'll
skip metrics/plots question and moves on to ask for path to
dvclive path.

Also, if live setting is enabled in the stage, it'll set the
`models` to be a checkpoint output.

* s/with/type and don't ask for live interactively
 2021-10-02 16:12:44 +05:45
Saugat Pachhai 0514db3bfa deps: loose requirements for dataclasses and aiohttp-retry (#6724) 2021-10-01 17:00:03 +05:45
Saugat Pachhai 1e0b2d7af0 benedict: monkeypatch, performance improvements, bugfixes (#6706) 
Fixes #6476, fixes #6423 and closes #6484.
 2021-10-01 18:16:47 +09:00
Casper da Costa-Luis bfcc8f877c add licence year + company (#6717) 2021-10-01 09:54:55 +02:00
Saugat Pachhai 46ad8745d3 Use latest pygit2 from master for 3.10 (#6723) 
* Use latest pygit2 from master for 3.10

* ci: update brew
 2021-10-01 11:48:11 +05:45
dependabot[bot] eb13d3ca83 build(deps): Bump google-cloud-storage from 1.42.2 to 1.42.3 (#6722) 
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 1.42.2 to 1.42.3.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v1.42.2...v1.42.3)

---
updated-dependencies:
- dependency-name: google-cloud-storage
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 08:27:40 +05:45
Ruslan Kuprieiev f46beb4958 checkout: move to objects (#6721) 
Checkout operation is pretty much the opposite of staging, so it too belongs in objects.
 2021-09-30 23:06:03 +03:00
Saugat Pachhai 9a1d74048a cli: add short alias for --show-{csv,json,md} (#6711) 
As `--csv`/`--json`/`--md`
 2021-09-30 20:38:32 +05:45
dependabot[bot] cce6aed378 build(deps): Bump filelock from 3.1.0 to 3.2.0 (#6710) 
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.1.0...3.2.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-30 08:49:39 +05:45
Saugat Pachhai 612696cf4a deps: add contextvars as a dependency for Python3.6 (#6705) 2021-09-29 10:39:14 +05:45
Saugat Pachhai (सौगात) 3f5f983f5f ui: import sys on write_json 2021-09-29 09:59:30 +05:45
eggqq007andZhenyu Gao fed9f088ca command: Show dvc list output with pretty JSON format if output to tty (#6647) 
1. update ls __init__.py to output pretty JSON format
2. introduce write_json function into ui __init__.py

Co-authored-by: Zhenyu Gao <[email protected]>
 2021-09-29 08:44:33 +05:45
Saugat Pachhai 05eaeef012 utils.http: remove open url (#6700) 
This was used before on opening http urls as a stream,
but we no longer use those (as we use fsspec's open method).

One thing to note however is that, compared to previous
implementation, fsspec's http filesystem's open is not very
robust to failures.
 2021-09-28 19:19:52 +05:45
Saugat Pachhai 48c0527755 exp init: show heading based on the presence of dvc.yaml file (#6701) 2021-09-28 15:11:58 +05:45
Saugat Pachhai 4668a35f64 exp init: support interactive and explicit options (#6681) 2021-09-28 14:36:21 +05:45