* Add Studio to the related technologies list
* Add CML to DVC Studio description
* fix k8s names a bit
Co-authored-by: Ivan Shcheklein <[email protected]>
* add: prevent crash while getting relpath
Since python 3.8 os.path.realpath has resolved windows network shares
to UNC paths. This breaks the logic for relpath as the UNC and mapped
drive paths don't have an obvious common prefix. As it is not easy to go
back to the mapped drive path from UNC path, this is resolved by always
using the UNC paths for calculating relative paths in windows.
Fixes#5832
* relpath: Update changes following review
Two changes:
- use pytest.mark.skipif rather than if statement inside test to make
it clearer that test is windows specific
- Use tail rather than upper_path to denote rest of path after drive
name taken off (to match python documentation)
Pull request: #5837
* tests, linter: update to add test case and apply linter though pre-commit
Co-authored-by: Tom Rutter <[email protected]>
* objects: make HashInfo & HashFile hashable
* stage/output: collect objects (HashFiles) instead of NamedCache
* remote/data_cloud: use objs instead of NamedCache
* repo: use objects instead of NamedCache
* rerun black after rebase
* remove NamedCache usage from run-cache
* drop NamedCache
* update HashInfo.__hash__
* update unit tests
* revert remote_name change
* start cleaning up test failures
* fix tree obj expansion
* update missing cache logging
* fix external dict handling
* update naive odb.gc
* update state save behavior for pull
* update test_data_cloud
* update test_remote
* fix gc with multiple repos
* objects.tree: support filtering w/preserved hash_info
* fix stage cache restore
* drop fs.path_info from HashFile.__hash__
* objects: add optional name attribute
* revert missing cache logging name/suffix changes
* revert name changes in remote tests
* review fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update dvc/remote/base.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* config, remote: Made S3 CA bundle customizable
botocore allows a path to a custom CA bundle either by passing a path
to the CA bundle file into the verify argument of
boto3.session.Session.client or passing None (the default) which will
fall back to the AWS config. Previously, the DVC config only accepted a
boolean into the ssl_verify option in the remote S3 config. This changes
the DVC config to accept both string and None in addition to boolean and
defaults to None. I also changed the default for ssl_verfiy to None in
BaseS3FileSystem. Thus, if ssl_verify is not provided, botocore will
fall back to the AWS config.
Testing
Unit tests to cover the changes to the config schema and addition
ssl_verify types that will be passed into S3FileSystem. Also, ran
dvc push -r object-store data/cifar-10-python.tar.gz
in my work environment that has a private S3 endpoint that requires a
custom CA bundle, both with and without ssl_verify specified in the
config. This was successful, showing that communication could be
established. And I ran
dvc remote modify object-store ssl_verify "$HOME/.aws/cabundle.pem"
and confirmed that the custom CA bundle path was added to the config.
Fixes#6012
* Removed default None on ssl_verify
Responding to PR comment, removed the Optional, default None on
ssl_verify since the config keys are optional by default. Rather
than a missing ssl_verify producing a None that eventually gets
filtered, it doesn't appear in the parsed config in the first place.
Co-authored-by: Robert Van Wesep <[email protected]>
astroid==2.5.7 has improved inference and was throwing
errors that were false negatives before. Fixing these
and ignoring some.
Also added `funcy.decorator` to the `signature-mutators`,
as it changes function signature (i.e. provides `Call` as
a first argument which is not required to pass).
Example errors that we were getting because of this:
```
tests/remotes/gdrive.py:40:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
************* Module dvc.stage.serialize
dvc/stage/serialize.py:38:1: E1120: No value for argument 'func' in
function call (no-value-for-parameter)
************* Module dvc.fs.http
dvc/fs/http.py:19:1: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
************* Module dvc.fs.gdrive
dvc/fs/gdrive.py:66:11: E1120: No value for argument 'tries' in function call (no-value-for-parameter)
dvc/fs/gdrive.py:459:5: E1120: No value for argument 'ctx' in function call (no-value-for-parameter)
```
Pre-requisite for removing fs.path_info and switching to regular strings
as paths in filesystems.
With new filesystems we no longer need the zoo of different dependency
and output types, as most of them are now just handled by the underlying
filesystem. Though deps/outs are still due for a refactor in other
aspects.
Batuhan Taskaya
Saugat Pachhai
dependabot[bot]
Ruslan Kuprieiev
pre-commit-ci[bot]
Dmitry PetrovandIvan Shcheklein
tomrutterandTom Rutter
rgvanwesepandRobert Van Wesep