Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
6,919 Commits 7 Branches 579 Tags
c5415342d02fd0b3e0b186534945c0f33d794757
Commit Graph
6919 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Saugat Pachhai c5415342d0 move requirements to setup.cfg (#6758) 
* move requirements to setup.cfg

* hash based on setup.cfg

* do not break webhdfs installation
 2021-10-08 07:00:14 +05:45
Paweł Redzyński dd86076ba6 gha: cancel previous workflows (#6756) 2021-10-07 17:59:44 +03:00
Ruslan Kuprieiev 7dc3859400 pyinstaller: hooks: remove webhdfs 2021-10-07 17:30:22 +03:00
Batuhan Taskaya 4f3b28e268 webhdfs: migrate to fsspec (#6662) 
* webhdfs: migrate to fsspec

* skip the tests that can't use hdfs
 2021-10-07 17:02:31 +03:00
Ruslan Kuprieiev 32f435c47f gha: benchmarks: fix pytest flags (#6749) 
Kudos @skshetry for catching.
 2021-10-06 17:07:41 +03:00
Batuhan Taskaya 054feae778 hdfs: migrate to fsspec (#6604) 
* hdfs: migrate to fsspec

* normalize windows paths when delegating back to fsspec
 2021-10-06 15:44:27 +03:00
Saugat Pachhai 2abe515391 Pin dvclive to 0.3.0 2021-10-06 14:28:36 +05:45
eggqq007andZhenyu Gao 1eb89a2dc0 command: Should not consume 'print' to output default remote info (#6650) 
1. should not consume 'print' to output default remote info,
   otherwise the --quiet would not work.

Co-authored-by: Zhenyu Gao <[email protected]>
 2021-10-06 09:27:47 +05:45
eggqq007andZhenyu Gao 9c300a0a1c command: output prettify json in tty (#6743) 
1. update diff.py,experiments.py,metrics.py,params.py,status.py to
print prettify json
2. rewrite write_json in __init__.py to accept default function
3. add new unit test for write_json in tests/unit/ui/test_console.py
4. Fix #6712

Co-authored-by: Zhenyu Gao <[email protected]>
 2021-10-05 20:19:56 +05:45
Ruslan Kuprieiev 7b68a6f0ca gha: run benchmarks on PRs (#6733) 2021-10-05 14:18:43 +03:00
Saugat Pachhai ee823a59b9 Run on Python3.10 (#6745) 2021-10-05 16:46:53 +05:45
David de la Iglesia CastroandSaugat Pachhai 16f404b16a Fix dvclive running subdir (#6740) 
* Fix dvclive running subdir

* Apply suggestions from code review

Co-authored-by: Saugat Pachhai <[email protected]>

* Applied suggestions

* Update tests/func/test_live.py

Co-authored-by: Saugat Pachhai <[email protected]>
 2021-10-05 13:16:33 +05:45
Saugat Pachhai 0eda4bd91c exp init: only ask for that are not provided in an interactive mode (#6739) 2021-10-04 14:33:08 +05:45
Saugat Pachhai a098a50c8d move exp init logic to dvc.repo.experiments.init (#6738) 2021-10-04 12:26:34 +05:45
dependabot[bot] f57a378c0c build(deps): Bump jaraco-windows from 5.6.0 to 5.7.0 (#6735) 
Bumps [jaraco-windows](https://github.com/jaraco/jaraco.windows) from 5.6.0 to 5.7.0.
- [Release notes](https://github.com/jaraco/jaraco.windows/releases)
- [Changelog](https://github.com/jaraco/jaraco.windows/blob/main/CHANGES.rst)
- [Commits](https://github.com/jaraco/jaraco.windows/compare/v5.6.0...v5.7.0)

---
updated-dependencies:
- dependency-name: jaraco-windows
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-04 10:12:47 +05:45
dependabot[bot] 3e0a50e52e build(deps): Bump pytest-cov from 2.12.1 to 3.0.0 (#6736) 
Bumps [pytest-cov](https://github.com/pytest-dev/pytest-cov) from 2.12.1 to 3.0.0.
- [Release notes](https://github.com/pytest-dev/pytest-cov/releases)
- [Changelog](https://github.com/pytest-dev/pytest-cov/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest-cov/compare/v2.12.1...v3.0.0)

---
updated-dependencies:
- dependency-name: pytest-cov
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-04 09:31:27 +05:45
Guillaume COTER ec72866eb9 Bump typing_extensions to >=3.7.4 (#6731) (#6732) 2021-10-03 20:10:28 +05:45
Batuhan Taskaya e53d390ff9 http: explicitly ignore all http-related errors (from server side to client connection drops) (#6728) 2021-10-02 20:42:23 +03:00
Batuhan Taskaya dd330ac592 setup: bump fsspec/s3fs/gcsfs (#6727) 2021-10-02 18:54:09 +02:00
pwoolvett 4ccb712b6d build(setup): avoid leaking tests package into site-packages (#6585) 
Signed-off-by: Pablo Woolvett <[email protected]>
 2021-10-02 16:46:45 +03:00
Saugat Pachhai ecee2d3522 Move static metadata to setup.cfg (#6581) 2021-10-02 16:08:30 +03:00
Anas ChetouanCHETOUANpre-commit-ci[bot]CHETOUAN
 		20bb6ddf3c doctor: fixes unknown cache directory error on some windows mapped network shares (#6364) 
* doctor: fixes unknown cache director error on some windows mapped network shares

On Windows, resolve calls os.path._getfinalpathname, which calls GetFinalPathNameByHandle.
This starts from the final resolved path of the file in the NT namespace (e.g. \Device\Mup\server\share...)
and works backward as required to get the VOLUME_NAME_DOS. If the final path is a UNC share (i.e. \Device\Mup),
We fix this by calling resolve() also on the partition mountpoints
We also take the opportunity to make checks in case a PermissionError is returned to us

Fixes github.com/iterative/dvc/issues/6305

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* doctor: fixes unknown cache director error on some windows mapped network shares

On Windows, resolve calls os.path._getfinalpathname, which calls GetFinalPathNameByHandle.
This starts from the final resolved path of the file in the NT namespace (e.g. \Device\Mup\server\share...)
and works backward as required to get the VOLUME_NAME_DOS. If the final path is a UNC share (i.e. \Device\Mup),
We fix this by calling resolve() also on the partition mountpoints
We also take the opportunity to make checks in case a PermissionError is returned to us

Fixes #6305

Co-authored-by: CHETOUAN <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: CHETOUAN <[email protected]>
 2021-10-02 16:06:42 +03:00
Saugat Pachhai e58fcd864b [WIP] exp init: introduce --with={default,live} flag (#6716) 
* exp init: introduce `--with={default,live}` flag

This flag will select the different mode of outputs of stage.
With `--with=live`, it'll enable `live` setting in the stage.
On interactive mode, it'll always ask user if they want to
use live (unless `--with=live` is specified), and then it'll
skip metrics/plots question and moves on to ask for path to
dvclive path.

Also, if live setting is enabled in the stage, it'll set the
`models` to be a checkpoint output.

* s/with/type and don't ask for live interactively
 2021-10-02 16:12:44 +05:45
Saugat Pachhai 0514db3bfa deps: loose requirements for dataclasses and aiohttp-retry (#6724) 2021-10-01 17:00:03 +05:45
Saugat Pachhai 1e0b2d7af0 benedict: monkeypatch, performance improvements, bugfixes (#6706) 
Fixes #6476, fixes #6423 and closes #6484.
 2021-10-01 18:16:47 +09:00
Casper da Costa-Luis bfcc8f877c add licence year + company (#6717) 2021-10-01 09:54:55 +02:00
Saugat Pachhai 46ad8745d3 Use latest pygit2 from master for 3.10 (#6723) 
* Use latest pygit2 from master for 3.10

* ci: update brew
 2021-10-01 11:48:11 +05:45
dependabot[bot] eb13d3ca83 build(deps): Bump google-cloud-storage from 1.42.2 to 1.42.3 (#6722) 
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 1.42.2 to 1.42.3.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v1.42.2...v1.42.3)

---
updated-dependencies:
- dependency-name: google-cloud-storage
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-01 08:27:40 +05:45
Ruslan Kuprieiev f46beb4958 checkout: move to objects (#6721) 
Checkout operation is pretty much the opposite of staging, so it too belongs in objects.
 2021-09-30 23:06:03 +03:00
Saugat Pachhai 9a1d74048a cli: add short alias for --show-{csv,json,md} (#6711) 
As `--csv`/`--json`/`--md`
 2021-09-30 20:38:32 +05:45
dependabot[bot] cce6aed378 build(deps): Bump filelock from 3.1.0 to 3.2.0 (#6710) 
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.1.0...3.2.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-30 08:49:39 +05:45
Saugat Pachhai 612696cf4a deps: add contextvars as a dependency for Python3.6 (#6705) 2021-09-29 10:39:14 +05:45
Saugat Pachhai (सौगात) 3f5f983f5f ui: import sys on write_json 2021-09-29 09:59:30 +05:45
eggqq007andZhenyu Gao fed9f088ca command: Show dvc list output with pretty JSON format if output to tty (#6647) 
1. update ls __init__.py to output pretty JSON format
2. introduce write_json function into ui __init__.py

Co-authored-by: Zhenyu Gao <[email protected]>
 2021-09-29 08:44:33 +05:45
Saugat Pachhai 05eaeef012 utils.http: remove open url (#6700) 
This was used before on opening http urls as a stream,
but we no longer use those (as we use fsspec's open method).

One thing to note however is that, compared to previous
implementation, fsspec's http filesystem's open is not very
robust to failures.
 2021-09-28 19:19:52 +05:45
Saugat Pachhai 48c0527755 exp init: show heading based on the presence of dvc.yaml file (#6701) 2021-09-28 15:11:58 +05:45
Saugat Pachhai 4668a35f64 exp init: support interactive and explicit options (#6681) 2021-09-28 14:36:21 +05:45
dependabot[bot] 8107ec849b build(deps): Bump filelock from 3.0.12 to 3.1.0 (#6698) 
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.0.12 to 3.1.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Commits](https://github.com/tox-dev/py-filelock/compare/v3.0.12...3.1.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-28 14:18:17 +05:45
dependabot[bot]andSaugat Pachhai 19284b3b4c build(deps): Bump urllib3 from 1.26.6 to 1.26.7 (#6697) 
* build(deps): Bump urllib3 from 1.26.6 to 1.26.7

Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.6 to 1.26.7.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/1.26.6...1.26.7)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Get rid of urllib3 as a test dependency

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Saugat Pachhai <[email protected]>
 2021-09-28 13:52:02 +05:45
dependabot[bot] 315d1cde99 build(deps): Bump pytest-xdist from 2.3.0 to 2.4.0 (#6696) 
Bumps [pytest-xdist](https://github.com/pytest-dev/pytest-xdist) from 2.3.0 to 2.4.0.
- [Release notes](https://github.com/pytest-dev/pytest-xdist/releases)
- [Changelog](https://github.com/pytest-dev/pytest-xdist/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest-xdist/compare/v2.3.0...v2.4.0)

---
updated-dependencies:
- dependency-name: pytest-xdist
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-28 07:50:59 +00:00
dependabot[bot] 1d415efd0b build(deps): Bump google-cloud-storage from 1.42.1 to 1.42.2 (#6635) 
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 1.42.1 to 1.42.2.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v1.42.1...v1.42.2)

---
updated-dependencies:
- dependency-name: google-cloud-storage
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-28 13:12:18 +05:45
Saugat Pachhai e7c20aa1d8 better exception messages for schema errors (#6679) 
* Generate prettier schema error messages

* add test for #6285
 2021-09-28 11:12:16 +05:45
Saugat Pachhai 058377f33e ui: support pager universally (#6692) 2021-09-27 18:11:20 +05:45
Saugat Pachhai c7b1b14905 ui: use ui.rich_text instead of rich.text.Text (#6690) 2021-09-27 12:04:19 +05:45
Amey Kulkarni fbd8f5c185 Issue 6651: Added alias ls for dvc list (#6683) 2021-09-25 21:14:11 +03:00
whysage 7fe370b729 remote: add support for Amazon S3 timeout configuration (#6634) 
Fixes #6241
 2021-09-25 14:06:43 +03:00
Batuhan Taskaya f9df6d2a16 http: allow reading proxies from the current environment; (#6688) 2021-09-25 11:32:44 +02:00
Ruslan Kuprieiev 2e4fb376da system: get rid of speedcopy (#6686) 
speedcopy only patches shutil.copyfile, which we don't even use
anywhere because it lacks callbacks/progress and we instead use
our own copyfile implementation, so this is totally useless for
us and can be removed.
 2021-09-25 04:42:45 +03:00
Ruslan Kuprieiev 6d6a3f17b1 system: remove legacy code (#6685) 2021-09-25 03:04:54 +03:00
Ruslan Kuprieiev bdaf4aced6 state: cleanup redundant or localfs-specific logic (#6684) 
Pre-requisite for making state work with other filesystems.
 2021-09-25 02:11:35 +03:00