Commit Graph
7082 Commits
Author SHA1 Message Date
pre-commit-ci[bot] 644e9ede92 [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/ambv/black: 21.10b0 → 21.11b1](https://github.com/ambv/black/compare/21.10b0...21.11b1)
2021-11-22 21:36:19 +02:00
Paweł Redzyński a9512e8585 plots: collect: return callable instead of data
Related to: #6923, iterative/viewer#2657
2021-11-22 21:31:20 +02:00
Saugat Pachhai (सौगात) 00e53c5c32 tests: fix black 2021-11-22 17:33:12 +05:45
Saugat Pachhai (सौगात) 0aaf96bce4 exp init: show 'n to omit' on --explicit mode 2021-11-22 17:20:16 +05:45
Saugat Pachhai (सौगात) e07b99ed33 tests: increase timeout to 2 second for prompts test 2021-11-22 15:10:55 +05:45
Saugat Pachhai (सौगात) 7bc906011f tests: timeout only on functions, excluding tests fixtures
Sometimes, `tmp_dir` fixtures take longer than a second. Using `func_only`
makes it only consider duration of test function run, excluding fixtures
duration.
2021-11-22 15:10:55 +05:45
Saugat Pachhai (सौगात) b84227db89 scm: remove ignore list 2021-11-22 07:16:54 +02:00
David de la Iglesia Castro 318b269ad1 live: Call webbrowser.open.
Use `funcy.once_per_args` decorator to prevent opening the html multiple times if user closes first time.

Closes iterative/dvclive#180
2021-11-22 03:35:23 +02:00
Ruslan Kuprieiev 1176b6d875 Update setup.cfg 2021-11-22 03:31:23 +02:00
dependabot[bot] ee6539292b build(deps): Bump adlfs from 2021.9.1 to 2021.10.0
Bumps [adlfs](https://github.com/dask/adlfs) from 2021.9.1 to 2021.10.0.
- [Release notes](https://github.com/dask/adlfs/releases)
- [Changelog](https://github.com/dask/adlfs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/dask/adlfs/compare/2021.09.1...2021.10.0)

---
updated-dependencies:
- dependency-name: adlfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2021-11-22 03:31:23 +02:00
Saugat Pachhai (सौगात) bf313d92aa exp init: timeout prompts' test
The `exp init` tests may just keep waiting on the prompts blocking tests.
The blocking of prompts is likely a failure in tests setup or a bug
in the implementation. It's hard to find out these failures or avoid it.
If they take longer than 1 second, they most likely have failed as
most of test takes 0.1 second at max.
2021-11-22 03:27:16 +02:00
Saugat Pachhai (सौगात) 46289182d2 tests: run on Windows with Python 3.10
pygit2 wheels should be now available.
Had to skip pytest-docker for Windows + Python 3.10 due to
a dependency issue with it's dependency, see:
https://github.com/docker/docker-py/issues/2902.
2021-11-20 18:52:31 +02:00
Saugat Pachhai (सौगात) cb941a836d tests: cover exp init 2021-11-19 18:58:23 +05:45
Gustavo José de Sousa a70b3087b4 build: bump sshfs from 2021.8.1 to 2021.11.2
This version uses bumps asyncssh to a version that supports ProxyJump.
2021-11-18 23:58:57 +02:00
Ruslan Kuprieiev c9dd137426 fs: BaseFileSystem -> FileSystem
Following fsspec conventions and new object conventions.
2021-11-18 19:58:23 +02:00
Saugat Pachhai (सौगात) 93e6526907 scm: split tracking of to-be git-ignored/git-staged into SCMContext 2021-11-18 15:34:45 +02:00
dependabot[bot] 230eb7087d build: bump google-cloud-storage from 1.42.3 to 1.43.0
Bumps [google-cloud-storage](https://github.com/googleapis/python-storage) from 1.42.3 to 1.43.0.
- [Release notes](https://github.com/googleapis/python-storage/releases)
- [Changelog](https://github.com/googleapis/python-storage/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-storage/compare/v1.42.3...v1.43.0)

---
updated-dependencies:
- dependency-name: google-cloud-storage
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2021-11-18 16:36:00 +05:45
Saugat Pachhai (सौगात) c3b91e5f85 exp init: interactive mode refactor and simplification 2021-11-18 16:06:52 +05:45
Saugat Pachhai (सौगात) eac030598c ui.prompt: move prompts from exp init to ui.prompt
Some fixes along the way to make it more generic, but it still has
lots of features tied with exp init.
2021-11-18 13:22:45 +05:45
karajan1001 068b8371aa Machine: table output for machine status(#6892)
Change the output format of `dvc machine status` command to tables for a
better readability.
2021-11-18 11:22:13 +08:00
karajan1001 888b6392bb Machine:remove field name in config(#6892)
Because our machine name now is the key value of the machine config
dict. And the machine config value will been overwritten by it when the
config is loaded. We can remove the field `name` from the config.
2021-11-18 11:22:13 +08:00
karajan1001 b24a31b834 Machine: use table for list outputs(#6892)
fix: #6892

Current output of machine list is quite urgly, it is originaly used to
show configurations. In this PR, we use a new table format for it.
2021-11-18 11:22:13 +08:00
karajan1001 45553c5b30 TabularData: add a new how arg to dropna(#6892)
Borrowed this idea from pandas api. Current dropna methods will drop a
column or row if any data is missing. But it is also quite useful to
drop those columns or rows only if all of its elements are missing. This
is mostly used for pruning a table.
2021-11-18 11:22:13 +08:00
Peter Rowlands 28f57933db dulwich: flush stdin channel on asyncssh vendor write 2021-11-17 12:48:34 +02:00
Peter Rowlands 25dca0b665 dulwich: workaround github SSH server in asyncssh vendor
- allow mismatched RSA signature algo names in initial
  SSH_MSG_USERAUTH_PK_OK response
- unit test overridden asyncssh publickey auth compatibility with
  github ssh server
2021-11-17 12:48:34 +02:00
Peter Rowlands 45c4e1998d dulwich: fallback to default client config in asyncssh vendor
- vendor should use ssh-agent or default IdentityFile(s) when no SSH key
  has been provided by the user
- encrypted (fallback) keys will always be passed to ssh-agent
2021-11-17 12:48:34 +02:00
karajan1001 eea67a6396 machine:remove help text for irrelevant commands (#6940)
fix: #6940

`--global/system/project/local` help message are in some irrelevant
command `status`, `ssh`, `create`, `destroy`. These commands are more
related to running instances rather than `machine config`, level
options didn't work for them.

1. Remove `--global/system/project/local` options in `status`, `ssh`,
   `create`, `destroy`.
2. Add a test for it.
3. Correct a typo error.
2021-11-17 16:25:48 +08:00
karajan1001andDave Berenbaum f43c04f935 Make machine status name optional
fix: #6838
1. make machine status name optional.
2. show all machine status if no name provide.
3. UI changes for this cmd.
4. add a new test for it.

Co-authored-by: Dave Berenbaum <[email protected]>
2021-11-17 15:44:36 +08:00
dependabot[bot] 3486435a7f build: bump filelock from 3.3.2 to 3.4.0
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.3.2 to 3.4.0.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Changelog](https://github.com/tox-dev/py-filelock/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.3.2...3.4.0)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2021-11-17 14:04:17 +08:00
Paweł Redzyński e0fd0790fc plots: special chars handling for vega plots paths
Fixes: #6894
2021-11-17 01:57:35 +02:00
dberenbaum f99d3cda57 scm: hint to autostage 2021-11-16 18:01:18 +02:00
pre-commit-ci[bot] 003461f9a2 [pre-commit.ci] pre-commit autoupdate
updates:
- [github.com/timothycrosley/isort: 5.9.3 → 5.10.1](https://github.com/timothycrosley/isort/compare/5.9.3...5.10.1)
2021-11-16 07:55:56 +05:45
Ruslan Kuprieiev 88660f6a66 dvc: use transfer in checkout/doctor/odb
All three of those use more or less the same logic for testing different
link types and maybe then using them. So now instead of having 3
implementations, we'll have just one.

Filesystem `transfer` now will not try to catch particular errno's, but
instead will just ignore any OSError and move down to the next link type
to try out, and if the last one fails - it will raise the last exception,
so if it is a common problem - we'll be able to see it.

This patch also removes `odb.confirmed_cache_type`, as it was caching
link type that is not really a cache-level thing, but rather a checkout thing,
that should only be cached for particular objects we are linking, as multiple
objects could be linked to different filesystems from the same odb (e.g.
external local output on hdd and local data on your ssd).

Next step here will be to introduce metadata diff, so that we can get rid of
temporary Link wrapper and introduce Transfer and corresponding transfer
config, that will know what links to use and could have some other things
like callbacks, so we don't continue passing dozens of kwargs along the stack.

Continuing from #6963

Fixes #6980
2021-11-15 22:50:00 +02:00
David de la Iglesia Castro 7df6090a9c params: Use ast.literal_eval for python params.
Add unit tests for `parse_py`

Closes  #6953
2021-11-15 17:52:49 +02:00
Dimitar Todorov fdba79c519 change method override to call super instead of repeating the calls 2021-11-15 17:50:14 +02:00
Димитър Тодоров (Dimitar Todorov)andSaugat Pachhai 2f6b6cc586 Update dvc/fs/http.py
Co-authored-by: Saugat Pachhai <[email protected]>
2021-11-15 17:50:14 +02:00
Dimitar Todorov 0fea622266 push: fix http storage ignoring method setting
Add method overriding the default put_file in http.py
where the upload method is passed to the http implementation.

Fixes: #6694
2021-11-15 17:50:14 +02:00
David de la Iglesia Castro 4a1985d7cd dvc render: Add RENDERERS registry.
Related: #6840, #4455, #6943
2021-11-15 16:37:24 +01:00
David de la Iglesia Castro c8903dc3a6 live: Add test for logging images with dvclive
Closes iterative/dvclive#190
2021-11-15 12:47:02 +01:00
David de la Iglesia Castro 28dd39a1a0 exp diff: Remove --old option.
After #6814 the header of `diff_table` is `a_rev/`b_rev` instead of `Old`/`New` so the name of the option was counterintuitive.

In addition, make `a_rev` show by default, for consistency with output format of other `diff_table` uses (`dvc metrics diff` / `dvc params diff`).
2021-11-12 12:58:12 +02:00
dependabot[bot] ef9d8580cc build: bump types-requests from 2.25.12 to 2.26.0
Bumps [types-requests](https://github.com/python/typeshed) from 2.25.12 to 2.26.0.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2021-11-12 12:40:38 +02:00
dependabot[bot] 1af23f32b7 build: bump pyinstaller from 4.6 to 4.7
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 4.6 to 4.7.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/v4.7/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v4.6...v4.7)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2021-11-12 12:40:27 +02:00
Saugat Pachhai (सौगात) 7b488887b4 exp init: suppress plots if live is selected as a stage type 2021-11-12 11:25:39 +05:45
Saugat Pachhai (सौगात) 8262dbd37c exp init: hide metrics/plots from worktree on --type=live 2021-11-12 11:21:58 +05:45
Ruslan Kuprieiev 264d78b637 ostage: don't forget to convert https etags
Fixes #6966
2021-11-12 00:54:44 +02:00
Ruslan Kuprieiev 1b2400842d checkout: catch ENOSYS 2021-11-11 22:38:29 +02:00
Saugat Pachhai (सौगात) 81db879c51 hardlink: fallback to other link methods if 38 error code is received
Fixes #6962, where GDdrive mounted on Colab was failing when trying to hardlink
with the message that the 'Function not implemented'.
2021-11-11 22:38:29 +02:00
Saugat Pachhai (सौगात) d668cbeb45 exp init: warn for code/data if they don't exist 2021-11-11 08:09:37 +05:45
Saugat Pachhai (सौगात) a466241552 exp init: validate params as soon as it's entered in prompt
It'll give a error message to the user and reprompt them.
This mechanism can be extended to other prompts, and we
can show a warning message and/or reprompt them based on
validations.

Closes #6865. Part of #6446.
2021-11-11 08:09:37 +05:45
Ruslan Kuprieiev 7e3e257c7f utils: fs: remove unused fs_copy 2021-11-11 01:26:37 +02:00