Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-01 23:08:43 +01:00
Code Issues Packages Projects Releases Wiki Activity
7,053 Commits 7 Branches 579 Tags
e0fd0790fc77b19694aabef3c94b52e7cd0135f1
Commit Graph
7053 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Paweł Redzyński e0fd0790fc plots: special chars handling for vega plots paths 
Fixes: #6894
 2021-11-17 01:57:35 +02:00
dberenbaum f99d3cda57 scm: hint to autostage 2021-11-16 18:01:18 +02:00
pre-commit-ci[bot] 003461f9a2 [pre-commit.ci] pre-commit autoupdate 
updates:
- [github.com/timothycrosley/isort: 5.9.3 → 5.10.1](https://github.com/timothycrosley/isort/compare/5.9.3...5.10.1)
 2021-11-16 07:55:56 +05:45
Ruslan Kuprieiev 88660f6a66 dvc: use transfer in checkout/doctor/odb 
All three of those use more or less the same logic for testing different
link types and maybe then using them. So now instead of having 3
implementations, we'll have just one.

Filesystem `transfer` now will not try to catch particular errno's, but
instead will just ignore any OSError and move down to the next link type
to try out, and if the last one fails - it will raise the last exception,
so if it is a common problem - we'll be able to see it.

This patch also removes `odb.confirmed_cache_type`, as it was caching
link type that is not really a cache-level thing, but rather a checkout thing,
that should only be cached for particular objects we are linking, as multiple
objects could be linked to different filesystems from the same odb (e.g.
external local output on hdd and local data on your ssd).

Next step here will be to introduce metadata diff, so that we can get rid of
temporary Link wrapper and introduce Transfer and corresponding transfer
config, that will know what links to use and could have some other things
like callbacks, so we don't continue passing dozens of kwargs along the stack.

Continuing from #6963

Fixes #6980
 2021-11-15 22:50:00 +02:00
David de la Iglesia Castro 7df6090a9c params: Use ast.literal_eval for python params. 
Add unit tests for `parse_py`

Closes  #6953
 2021-11-15 17:52:49 +02:00
Dimitar Todorov fdba79c519 change method override to call super instead of repeating the calls 2021-11-15 17:50:14 +02:00
Димитър Тодоров (Dimitar Todorov)andSaugat Pachhai 2f6b6cc586 Update dvc/fs/http.py 
Co-authored-by: Saugat Pachhai <[email protected]>
 2021-11-15 17:50:14 +02:00
Dimitar Todorov 0fea622266 push: fix http storage ignoring method setting 
Add method overriding the default put_file in http.py
where the upload method is passed to the http implementation.

Fixes: #6694
 2021-11-15 17:50:14 +02:00
David de la Iglesia Castro 4a1985d7cd dvc render: Add RENDERERS registry. 
Related: #6840, #4455, #6943
 2021-11-15 16:37:24 +01:00
David de la Iglesia Castro c8903dc3a6 live: Add test for logging images with dvclive 
Closes iterative/dvclive#190
 2021-11-15 12:47:02 +01:00
David de la Iglesia Castro 28dd39a1a0 exp diff: Remove --old option. 
After #6814 the header of `diff_table` is `a_rev/`b_rev` instead of `Old`/`New` so the name of the option was counterintuitive.

In addition, make `a_rev` show by default, for consistency with output format of other `diff_table` uses (`dvc metrics diff` / `dvc params diff`).
 2021-11-12 12:58:12 +02:00
dependabot[bot] ef9d8580cc build: bump types-requests from 2.25.12 to 2.26.0 
Bumps [types-requests](https://github.com/python/typeshed) from 2.25.12 to 2.26.0.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-12 12:40:38 +02:00
dependabot[bot] 1af23f32b7 build: bump pyinstaller from 4.6 to 4.7 
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 4.6 to 4.7.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/v4.7/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v4.6...v4.7)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-12 12:40:27 +02:00
Saugat Pachhai (सौगात) 7b488887b4 exp init: suppress plots if live is selected as a stage type 2021-11-12 11:25:39 +05:45
Saugat Pachhai (सौगात) 8262dbd37c exp init: hide metrics/plots from worktree on --type=live 2021-11-12 11:21:58 +05:45
Ruslan Kuprieiev 264d78b637 ostage: don't forget to convert https etags 
Fixes #6966
 2021-11-12 00:54:44 +02:00
Ruslan Kuprieiev 1b2400842d checkout: catch ENOSYS 2021-11-11 22:38:29 +02:00
Saugat Pachhai (सौगात) 81db879c51 hardlink: fallback to other link methods if 38 error code is received 
Fixes #6962, where GDdrive mounted on Colab was failing when trying to hardlink
with the message that the 'Function not implemented'.
 2021-11-11 22:38:29 +02:00
Saugat Pachhai (सौगात) d668cbeb45 exp init: warn for code/data if they don't exist 2021-11-11 08:09:37 +05:45
Saugat Pachhai (सौगात) a466241552 exp init: validate params as soon as it's entered in prompt 
It'll give a error message to the user and reprompt them.
This mechanism can be extended to other prompts, and we
can show a warning message and/or reprompt them based on
validations.

Closes #6865. Part of #6446.
 2021-11-11 08:09:37 +05:45
Ruslan Kuprieiev 7e3e257c7f utils: fs: remove unused fs_copy 2021-11-11 01:26:37 +02:00
Paweł Redzyński ab56e4bb9b gha: limit python version to 3.9.7 
Related: https://bugs.python.org/issue45235
Closes: #6956
 2021-11-11 01:14:03 +02:00
Paweł Redzyński 36ed182f26 argparse: dont override cmd 
Related: #6956, https://bugs.python.org/issue45235
 2021-11-11 01:14:03 +02:00
Ruslan Kuprieiev 045d1f5e3e localfs: remove LocalPath 
fs.path is meant to be lightweight, and we should try to avoid using os.path.

Leftover from #6946
 2021-11-10 20:10:48 +02:00
Ruslan Kuprieiev 5214aec1bb fspath: introduce relparts 2021-11-10 20:10:48 +02:00
David de la Iglesia Castro 72066f233b exp show: Add --only-changed option. 
Use TabularData `drop_duplicates` to only show metrics/params with values varying across the selected experiments.

Fixes #5966
 2021-11-10 16:52:18 +02:00
David de la Iglesia Castro cb6f625dd9 TabularData: Add drop_duplicates method. 
This method mimics pandas `drop_duplicates` when axis=rows.

For axis=cols this method will drop columns with 0-variance (only
1 unique value). NaNs (`_fill_value`) won't be counted as unique values.

So the following column (`-` would be the `_fill_value`) will be dropped:

foo
-
foo
foo
 2021-11-10 16:52:18 +02:00
dependabot[bot] 87fadd3225 build: bump dvclive from 0.4.2 to 0.4.3 
Bumps [dvclive](https://github.com/iterative/dvclive) from 0.4.2 to 0.4.3.
- [Release notes](https://github.com/iterative/dvclive/releases)
- [Commits](https://github.com/iterative/dvclive/compare/0.4.2...0.4.3)

---
updated-dependencies:
- dependency-name: dvclive
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-10 11:46:41 +01:00
dependabot[bot] 8f028028b9 build: bump types-requests from 2.25.11 to 2.25.12 
Bumps [types-requests](https://github.com/python/typeshed) from 2.25.11 to 2.25.12.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-10 11:45:42 +01:00
Batuhan Taskaya 9aec8f1926 dvc: get rid of path_info 
Intermediate PR in removing path_info from our code base.

It is completely removed from the main codebase and is still
being used in tests, which will be migrated in the followups.

For fs-specific simple path manipulation, we've introduced
`fs.path` helper (similar to `os.path`).

Fixes #6774
 2021-11-10 00:52:48 +02:00
dependabot[bot] 4175f9ffa4 build: bump s3/gcsfs from 2021.10.1 to 2021.11.0 
Bumps [gcsfs](https://github.com/fsspec/gcsfs) from 2021.10.1 to 2021.11.0.
- [Release notes](https://github.com/fsspec/gcsfs/releases)
Bumps [s3fs](https://github.com/fsspec/s3fs) from 2021.10.1 to 2021.11.0.
- [Release notes](https://github.com/fsspec/s3fs/releases)
- [Changelog](https://github.com/fsspec/s3fs/blob/main/release-procedure.md)

---
updated-dependencies:
- dependency-name: s3fs
  dependency-type: direct:development
  update-type: version-update:semver-minor
- dependency-name: gcsfs
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2.8.3 		2021-11-08 11:13:52 +08:00
Guðmundur HeimissonandRuslan Kuprieiev c5b1a73dc5 webhdfs: expose kerberos and https options (#6936) 
* Add fsspec options for webhdfs

* Add requests-kerberos dependency

* Update dvc/fs/webhdfs.py

Simplify config.pop.

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/fs/webhdfs.py

* Simplify kerb_kwargs config setting.

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2021-11-07 20:42:10 +02:00
Saugat Pachhai (सौगात) 044e79000d deps: bump rich to >=10.13.0 
It comes with the support for no. of parameters that matches with json.dumps.
See: https://github.com/willmcgugan/rich/pull/1547 and
https://github.com/willmcgugan/rich/pull/1644.

Also added support for printing json to stderr. Before we used to use json.dumps
and rich.json.JSON based on isatty. Now we use rich.print_json on both
tty and non-tty conditions, which may avoid different behaviours in the
future.
 2021-11-07 19:38:06 +02:00
karajan1001 be6128470f machine status: include two new field in the output 
1. include instance_type and instance_gpu in the output.
2. rerange the field by importance.
 2021-11-04 15:10:45 +02:00
Saugat Pachhai (सौगात) dd0d4f5a2c exp init: don't try tracking params if they are skipped 2021-11-03 19:21:10 +05:45
Saugat Pachhai (सौगात) 6e0dfb94e4 exp init: hide message about workspace tree on --explicit 2021-11-03 18:59:25 +05:45
Saugat Pachhai (सौगात) ce6c8a5678 exp init: fix --explicit failing with KeyError 
Fixes #6917
 2021-11-03 16:42:45 +05:45
dependabot[bot] b927337a30 build(deps): bump actions/checkout from 2.3.5 to 2.4.0 
Bumps [actions/checkout](https://github.com/actions/checkout) from 2.3.5 to 2.4.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.3.5...v2.4.0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-03 12:47:22 +02:00
Saugat Pachhai (सौगात) 03b372a073 exp init: ask for command before paths 2021-11-03 05:23:40 +05:45
Saugat Pachhai (सौगात) 915a755e2f build: remove numpy restriction 2021-11-03 05:23:17 +05:45
Paweł Redzyński 22cc2a5cbc plots: fix resolve template path bug 
Fix: #6854
 2021-11-02 16:49:56 +02:00
Saugat Pachhai (सौगात) a67d8a44b3 exp init: make confirmation prompt at the end similar to others 2021-11-02 17:24:09 +05:45
Saugat Pachhai (सौगात) 80a52e65ad exp init: minor refactor on way it returns prompted values 2021-11-02 14:57:40 +05:45
Saugat Pachhai (सौगात) 208ccb2430 exp init: autostage .gitignores, dvc.yaml and params file 2021-11-02 12:00:06 +05:45
dependabot[bot] 97e97c7f24 build(deps-dev): bump filelock from 3.3.1 to 3.3.2 
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.3.1 to 3.3.2.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Changelog](https://github.com/tox-dev/py-filelock/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.3.1...3.3.2)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-02 08:16:37 +05:45
dependabot[bot] 1d1451ddd3 build(deps): bump pyinstaller from 4.5.1 to 4.6 
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 4.5.1 to 4.6.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/v4.6/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v4.5.1...v4.6)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-02 08:16:18 +05:45
pre-commit-ci[bot] 0cc7c4c02b [pre-commit.ci] pre-commit autoupdate 
updates:
- [github.com/ambv/black: 21.9b0 → 21.10b0](https://github.com/ambv/black/compare/21.9b0...21.10b0)
 2021-11-02 06:30:13 +05:45
Saugat Pachhai d3d459caa0 Revert "deps: require chardet as a dependency" 
This reverts commit a4dc8ddc74.
 2021-11-02 06:17:31 +05:45
Saugat Pachhai a4dc8ddc74 deps: require chardet as a dependency 
Looks like azure-core is dependent on a dependency `chardet` that aiohttp brought it with, which has been replaced with `charset_normalizer`.
But azure-core depends on internal of aiohttp and looks like it tries to set it's own encoding parser, `chardet` which is now not available.
 2021-11-01 18:25:39 +02:00
Saugat Pachhai 3eceeea3e8 Remove python 3.10 restriction for pyarrow 
Pyarrow 6 is now available for Python 3.10. Fixes https://github.com/iterative/dvc/issues/6878.
 2021-11-01 20:00:21 +05:45