Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
7,042 Commits 7 Branches 579 Tags
ef9d8580cc6716712c73a8efc805419ab4a3d154
Commit Graph
7042 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
dependabot[bot] ef9d8580cc build: bump types-requests from 2.25.12 to 2.26.0 
Bumps [types-requests](https://github.com/python/typeshed) from 2.25.12 to 2.26.0.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-12 12:40:38 +02:00
dependabot[bot] 1af23f32b7 build: bump pyinstaller from 4.6 to 4.7 
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 4.6 to 4.7.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/v4.7/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v4.6...v4.7)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-12 12:40:27 +02:00
Saugat Pachhai (सौगात) 7b488887b4 exp init: suppress plots if live is selected as a stage type 2021-11-12 11:25:39 +05:45
Saugat Pachhai (सौगात) 8262dbd37c exp init: hide metrics/plots from worktree on --type=live 2021-11-12 11:21:58 +05:45
Ruslan Kuprieiev 264d78b637 ostage: don't forget to convert https etags 
Fixes #6966
 2021-11-12 00:54:44 +02:00
Ruslan Kuprieiev 1b2400842d checkout: catch ENOSYS 2021-11-11 22:38:29 +02:00
Saugat Pachhai (सौगात) 81db879c51 hardlink: fallback to other link methods if 38 error code is received 
Fixes #6962, where GDdrive mounted on Colab was failing when trying to hardlink
with the message that the 'Function not implemented'.
 2021-11-11 22:38:29 +02:00
Saugat Pachhai (सौगात) d668cbeb45 exp init: warn for code/data if they don't exist 2021-11-11 08:09:37 +05:45
Saugat Pachhai (सौगात) a466241552 exp init: validate params as soon as it's entered in prompt 
It'll give a error message to the user and reprompt them.
This mechanism can be extended to other prompts, and we
can show a warning message and/or reprompt them based on
validations.

Closes #6865. Part of #6446.
 2021-11-11 08:09:37 +05:45
Ruslan Kuprieiev 7e3e257c7f utils: fs: remove unused fs_copy 2021-11-11 01:26:37 +02:00
Paweł Redzyński ab56e4bb9b gha: limit python version to 3.9.7 
Related: https://bugs.python.org/issue45235
Closes: #6956
 2021-11-11 01:14:03 +02:00
Paweł Redzyński 36ed182f26 argparse: dont override cmd 
Related: #6956, https://bugs.python.org/issue45235
 2021-11-11 01:14:03 +02:00
Ruslan Kuprieiev 045d1f5e3e localfs: remove LocalPath 
fs.path is meant to be lightweight, and we should try to avoid using os.path.

Leftover from #6946
 2021-11-10 20:10:48 +02:00
Ruslan Kuprieiev 5214aec1bb fspath: introduce relparts 2021-11-10 20:10:48 +02:00
David de la Iglesia Castro 72066f233b exp show: Add --only-changed option. 
Use TabularData `drop_duplicates` to only show metrics/params with values varying across the selected experiments.

Fixes #5966
 2021-11-10 16:52:18 +02:00
David de la Iglesia Castro cb6f625dd9 TabularData: Add drop_duplicates method. 
This method mimics pandas `drop_duplicates` when axis=rows.

For axis=cols this method will drop columns with 0-variance (only
1 unique value). NaNs (`_fill_value`) won't be counted as unique values.

So the following column (`-` would be the `_fill_value`) will be dropped:

foo
-
foo
foo
 2021-11-10 16:52:18 +02:00
dependabot[bot] 87fadd3225 build: bump dvclive from 0.4.2 to 0.4.3 
Bumps [dvclive](https://github.com/iterative/dvclive) from 0.4.2 to 0.4.3.
- [Release notes](https://github.com/iterative/dvclive/releases)
- [Commits](https://github.com/iterative/dvclive/compare/0.4.2...0.4.3)

---
updated-dependencies:
- dependency-name: dvclive
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-10 11:46:41 +01:00
dependabot[bot] 8f028028b9 build: bump types-requests from 2.25.11 to 2.25.12 
Bumps [types-requests](https://github.com/python/typeshed) from 2.25.11 to 2.25.12.
- [Release notes](https://github.com/python/typeshed/releases)
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-requests
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-10 11:45:42 +01:00
Batuhan Taskaya 9aec8f1926 dvc: get rid of path_info 
Intermediate PR in removing path_info from our code base.

It is completely removed from the main codebase and is still
being used in tests, which will be migrated in the followups.

For fs-specific simple path manipulation, we've introduced
`fs.path` helper (similar to `os.path`).

Fixes #6774
 2021-11-10 00:52:48 +02:00
dependabot[bot] 4175f9ffa4 build: bump s3/gcsfs from 2021.10.1 to 2021.11.0 
Bumps [gcsfs](https://github.com/fsspec/gcsfs) from 2021.10.1 to 2021.11.0.
- [Release notes](https://github.com/fsspec/gcsfs/releases)
Bumps [s3fs](https://github.com/fsspec/s3fs) from 2021.10.1 to 2021.11.0.
- [Release notes](https://github.com/fsspec/s3fs/releases)
- [Changelog](https://github.com/fsspec/s3fs/blob/main/release-procedure.md)

---
updated-dependencies:
- dependency-name: s3fs
  dependency-type: direct:development
  update-type: version-update:semver-minor
- dependency-name: gcsfs
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
2.8.3 		2021-11-08 11:13:52 +08:00
Guðmundur HeimissonandRuslan Kuprieiev c5b1a73dc5 webhdfs: expose kerberos and https options (#6936) 
* Add fsspec options for webhdfs

* Add requests-kerberos dependency

* Update dvc/fs/webhdfs.py

Simplify config.pop.

Co-authored-by: Ruslan Kuprieiev <[email protected]>

* Update dvc/fs/webhdfs.py

* Simplify kerb_kwargs config setting.

Co-authored-by: Ruslan Kuprieiev <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
 2021-11-07 20:42:10 +02:00
Saugat Pachhai (सौगात) 044e79000d deps: bump rich to >=10.13.0 
It comes with the support for no. of parameters that matches with json.dumps.
See: https://github.com/willmcgugan/rich/pull/1547 and
https://github.com/willmcgugan/rich/pull/1644.

Also added support for printing json to stderr. Before we used to use json.dumps
and rich.json.JSON based on isatty. Now we use rich.print_json on both
tty and non-tty conditions, which may avoid different behaviours in the
future.
 2021-11-07 19:38:06 +02:00
karajan1001 be6128470f machine status: include two new field in the output 
1. include instance_type and instance_gpu in the output.
2. rerange the field by importance.
 2021-11-04 15:10:45 +02:00
Saugat Pachhai (सौगात) dd0d4f5a2c exp init: don't try tracking params if they are skipped 2021-11-03 19:21:10 +05:45
Saugat Pachhai (सौगात) 6e0dfb94e4 exp init: hide message about workspace tree on --explicit 2021-11-03 18:59:25 +05:45
Saugat Pachhai (सौगात) ce6c8a5678 exp init: fix --explicit failing with KeyError 
Fixes #6917
 2021-11-03 16:42:45 +05:45
dependabot[bot] b927337a30 build(deps): bump actions/checkout from 2.3.5 to 2.4.0 
Bumps [actions/checkout](https://github.com/actions/checkout) from 2.3.5 to 2.4.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2.3.5...v2.4.0)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-03 12:47:22 +02:00
Saugat Pachhai (सौगात) 03b372a073 exp init: ask for command before paths 2021-11-03 05:23:40 +05:45
Saugat Pachhai (सौगात) 915a755e2f build: remove numpy restriction 2021-11-03 05:23:17 +05:45
Paweł Redzyński 22cc2a5cbc plots: fix resolve template path bug 
Fix: #6854
 2021-11-02 16:49:56 +02:00
Saugat Pachhai (सौगात) a67d8a44b3 exp init: make confirmation prompt at the end similar to others 2021-11-02 17:24:09 +05:45
Saugat Pachhai (सौगात) 80a52e65ad exp init: minor refactor on way it returns prompted values 2021-11-02 14:57:40 +05:45
Saugat Pachhai (सौगात) 208ccb2430 exp init: autostage .gitignores, dvc.yaml and params file 2021-11-02 12:00:06 +05:45
dependabot[bot] 97e97c7f24 build(deps-dev): bump filelock from 3.3.1 to 3.3.2 
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.3.1 to 3.3.2.
- [Release notes](https://github.com/tox-dev/py-filelock/releases)
- [Changelog](https://github.com/tox-dev/py-filelock/blob/main/docs/changelog.rst)
- [Commits](https://github.com/tox-dev/py-filelock/compare/3.3.1...3.3.2)

---
updated-dependencies:
- dependency-name: filelock
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-02 08:16:37 +05:45
dependabot[bot] 1d1451ddd3 build(deps): bump pyinstaller from 4.5.1 to 4.6 
Bumps [pyinstaller](https://github.com/pyinstaller/pyinstaller) from 4.5.1 to 4.6.
- [Release notes](https://github.com/pyinstaller/pyinstaller/releases)
- [Changelog](https://github.com/pyinstaller/pyinstaller/blob/v4.6/doc/CHANGES.rst)
- [Commits](https://github.com/pyinstaller/pyinstaller/compare/v4.5.1...v4.6)

---
updated-dependencies:
- dependency-name: pyinstaller
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
 2021-11-02 08:16:18 +05:45
pre-commit-ci[bot] 0cc7c4c02b [pre-commit.ci] pre-commit autoupdate 
updates:
- [github.com/ambv/black: 21.9b0 → 21.10b0](https://github.com/ambv/black/compare/21.9b0...21.10b0)
 2021-11-02 06:30:13 +05:45
Saugat Pachhai d3d459caa0 Revert "deps: require chardet as a dependency" 
This reverts commit a4dc8ddc74.
 2021-11-02 06:17:31 +05:45
Saugat Pachhai a4dc8ddc74 deps: require chardet as a dependency 
Looks like azure-core is dependent on a dependency `chardet` that aiohttp brought it with, which has been replaced with `charset_normalizer`.
But azure-core depends on internal of aiohttp and looks like it tries to set it's own encoding parser, `chardet` which is now not available.
 2021-11-01 18:25:39 +02:00
Saugat Pachhai 3eceeea3e8 Remove python 3.10 restriction for pyarrow 
Pyarrow 6 is now available for Python 3.10. Fixes https://github.com/iterative/dvc/issues/6878.
 2021-11-01 20:00:21 +05:45
Saugat Pachhai (सौगात) 8fc959060a tests: skip based on conditional import on pyarrow 
From now on, pyarrow dependency is optional for the tests.
 2021-11-01 19:15:28 +05:45
David de la Iglesia Castro 0638d99fe3 LiveTask: write output path at start/end 
Writing at the end of each step (create_summary) can mess up with some ML Framework progress bars, I moved the message to the monitor task, as the path won't change during training.

Fixes iterative/dvclive#177
 2021-10-29 14:41:03 +02:00
David de la Iglesia Castro 24abc6f328 TabularData: Add dropna method. 
This method mimics pandas `dropna`.
 2021-10-29 15:28:36 +03:00
David de la Iglesia Castro 4dd544121f plots: Use URI for --open. 
We were previously using relpath in *all* cases, mainly to support WSL but relpath doesn't work
on macOS.

Added a check for using relpath *only* for WSL.

Fixes #6092
 2021-10-29 13:27:12 +02:00
Ruslan Kuprieiev 869ea442ef output: save: remove redundant changed() call 
No longer needed since object introduction, as we will compare staged
object during commit/checkout anyway.
 2021-10-29 01:45:10 +03:00
Ruslan Kuprieiev 922eb20e44 staging: don't use move 2021-10-27 20:19:17 +03:00
Ruslan Kuprieiev a2f91ed5fd checkout: always confirm link type 
Currently whether or not link confirmation will run depends on
particular filesystem and existing workspace link type, which introduces
inconsistency in behaviour, where we sometimes catch unsupported links
before touching user data and sometimes we don't.
 2021-10-27 20:19:17 +03:00
Ruslan Kuprieiev 017762fc2c checkout: don't forget to protect() back objects 
Depending on a filesyste (e.g. on NTFS), `_remove` might reset
read-only permissions in order to delete a hardlink to protected object,
which will also reset it for the object itself, making it unprotected,
so we need to protect it back.
 2021-10-27 20:19:17 +03:00
Ruslan Kuprieiev 874aceccf8 system: use linkat instead of link 
`os.link` is broken across platforms when dealing with symlinks.

See https://bugs.python.org/issue41355 for more info.
 2021-10-27 20:19:17 +03:00
Ruslan Kuprieiev 41aee9a3b7 localfs: use mtime/size/inode for checksum 
Currently it depends on the whole `info()` output, which has too much
information that doesn't really matter. To check if file contents didn't
change mtime, size, inode combo is enough.
 2021-10-27 20:19:17 +03:00
Peter Rowlands (변기호) 0d895e2661 exp: add initial ssh executor (#6833) 
* executor: move local specific git initialization out of base

* machine: return kwargs for executor init instead of executor object

* executor: add initial ssh executor

* ssh-executor: disable strict host checking for dvc-machine instances

* initial tests for ssh executor

* drop GIT_SSH_COMMAND usage

* push: accept destination odb as a parameter

* separate cache initialization into executor.init_cache

* add test for ssh executor cache init

* remove dead code

* machine: fixture-ize default test scenarios

* test SSHExecutor.from_machine

* tpi: return default SSH port for constructing executors
 2021-10-26 16:08:17 +09:00