Because our machine name now is the key value of the machine config
dict. And the machine config value will been overwritten by it when the
config is loaded. We can remove the field `name` from the config.
Borrowed this idea from pandas api. Current dropna methods will drop a
column or row if any data is missing. But it is also quite useful to
drop those columns or rows only if all of its elements are missing. This
is mostly used for pruning a table.
- allow mismatched RSA signature algo names in initial
SSH_MSG_USERAUTH_PK_OK response
- unit test overridden asyncssh publickey auth compatibility with
github ssh server
fix: #6940
`--global/system/project/local` help message are in some irrelevant
command `status`, `ssh`, `create`, `destroy`. These commands are more
related to running instances rather than `machine config`, level
options didn't work for them.
1. Remove `--global/system/project/local` options in `status`, `ssh`,
`create`, `destroy`.
2. Add a test for it.
3. Correct a typo error.
fix: #6838
1. make machine status name optional.
2. show all machine status if no name provide.
3. UI changes for this cmd.
4. add a new test for it.
Co-authored-by: Dave Berenbaum <[email protected]>
All three of those use more or less the same logic for testing different
link types and maybe then using them. So now instead of having 3
implementations, we'll have just one.
Filesystem `transfer` now will not try to catch particular errno's, but
instead will just ignore any OSError and move down to the next link type
to try out, and if the last one fails - it will raise the last exception,
so if it is a common problem - we'll be able to see it.
This patch also removes `odb.confirmed_cache_type`, as it was caching
link type that is not really a cache-level thing, but rather a checkout thing,
that should only be cached for particular objects we are linking, as multiple
objects could be linked to different filesystems from the same odb (e.g.
external local output on hdd and local data on your ssd).
Next step here will be to introduce metadata diff, so that we can get rid of
temporary Link wrapper and introduce Transfer and corresponding transfer
config, that will know what links to use and could have some other things
like callbacks, so we don't continue passing dozens of kwargs along the stack.
Continuing from #6963Fixes#6980
After #6814 the header of `diff_table` is `a_rev/`b_rev` instead of `Old`/`New` so the name of the option was counterintuitive.
In addition, make `a_rev` show by default, for consistency with output format of other `diff_table` uses (`dvc metrics diff` / `dvc params diff`).
This method mimics pandas `drop_duplicates` when axis=rows.
For axis=cols this method will drop columns with 0-variance (only
1 unique value). NaNs (`_fill_value`) won't be counted as unique values.
So the following column (`-` would be the `_fill_value`) will be dropped:
foo
-
foo
foo
Intermediate PR in removing path_info from our code base.
It is completely removed from the main codebase and is still
being used in tests, which will be migrated in the followups.
For fs-specific simple path manipulation, we've introduced
`fs.path` helper (similar to `os.path`).
Fixes#6774
We were previously using relpath in *all* cases, mainly to support WSL but relpath doesn't work
on macOS.
Added a check for using relpath *only* for WSL.
Fixes#6092
* executor: move local specific git initialization out of base
* machine: return kwargs for executor init instead of executor object
* executor: add initial ssh executor
* ssh-executor: disable strict host checking for dvc-machine instances
* initial tests for ssh executor
* drop GIT_SSH_COMMAND usage
* push: accept destination odb as a parameter
* separate cache initialization into executor.init_cache
* add test for ssh executor cache init
* remove dead code
* machine: fixture-ize default test scenarios
* test SSHExecutor.from_machine
* tpi: return default SSH port for constructing executors
* Add experiment name check.
1. Add experiment name check (https://git-scm.com/docs/git-check-ref-format)
2. Add duplicate exp name check.
3. Add some unit test for it.
* Ban slash / in dvc exp names
* Use dulwich backend for ref name checking
* Some bug fix
* Update dvc/repo/experiments/__init__.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Update dvc/repo/experiments/__init__.py
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
* Some review changes
* Make some funtion more reusable.
Co-authored-by: Peter Rowlands (변기호) <[email protected]>
If supported, reflink, will provide the fastest transfer possible.
That being said, right now we use `move` in most of the places,
which still takes presidence.
Pre-requisite for #6387
scm.install does 3 different things, and all of those are
very much tied to DVC. So, as we are splitting scm out,
it should now belong to `dvc.install`.
* git: support client kwargs for auth in remote methods
* tests: use docker for git/ssh server
* move dulwich backend into subdir
* add base sync<->async support
* dulwich: add initial asyncssh SSH vendor
* dulwich: use asyncssh as the default SSH vendor
* add test for git over ssh
* test read (fetch) over ssh
* ci: re-enable ssh tests
temporary workaround for #6788, #6789
* move async stubs into dvc.scm.asyn
* tests: use sshfs and not dvc.fs.ssh
* allow explicit 0-length read in asyncssh vendor
* `dvc machine`: add a new command `rename` to it
fix#6481
1. add new command `dvc machine rename`
2. add a unit test for the call.
3. add several functional tests for it.t push myself fix6481
* Move rename logic to tpi
* use tpi.state_mv instead of rename
* Rebase to a newer version
* Update tpi version
* Remove unused assert and try catch
* throw error if new parameters are added by dvc run exp -S
Throws an error if dvc run exp --set-param references a parameter key path
that is not in the target config file. This is done to prevent silent errors
in experimentation when new params are accidentally added instead of modifying
current ones due to typos.
This implementation is based on initial suggestions in issue #5477.
* fix mypy confusion from overwriting a var
* tests: remove modify param test that was adding params
One of the test in test_modify_params was still adding a parameter
* tests: remove param modification during exp run in test_untracked
Spurious usage of modifying a parameter when the params.yaml is not yet existing in test_untracked.
* remove benedict dependency for verifying params
Instead, the keypath util module from benedict is adapted into the dvc codebased and used
natively to verify that a keypath exists within a dict.
* exp: list all invalid param modifications in error message
* exp-run: more specific exception for new params via --set-param
Uses established MissingPramsError instead of DvcException. Modified message to be more similar to other uses of this exception.
* Revert "remove benedict dependency for verifying params"
This reverts commit 6170f16dcc4033667fbacb0d0bc13c60898f58c4.
* exp: use internal benedict import for validating params
* exp: combine new param validation into merge_params
Moves new param validation into existing merge_params function as an optional check
* exp: minor ammendments to error message
* Revert "tests: remove param modification during exp run in test_untracked"
This reverts commit c9eb1a62af58586509ded362cc79e75ddc4b119c.
* exp: fix test_untracked to not fail due to new params
Fixed by creating the params file with the new parameter already present.
* machine: Add a new command `dvc machine status`
fix#6482
1. add a new command `dvc machien status`
2. add a unit test for the cli call.
3. add a unit test for the shown result.
* Update dvc/command/machine.py
Co-authored-by: Saugat Pachhai <[email protected]>
* Update dvc/command/machine.py
Co-authored-by: Saugat Pachhai <[email protected]>
* Update machine.py
* Remove sensitive fields
* Update dvc/machine/__init__.py
Co-authored-by: Casper da Costa-Luis <[email protected]>
* Update tests/func/machine/__init__.py
Co-authored-by: Casper da Costa-Luis <[email protected]>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use white list to limit shown fields
Co-authored-by: Saugat Pachhai <[email protected]>
Co-authored-by: Casper da Costa-Luis <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Saugat Pachhai (सौगात)
Ruslan Kuprieiev
karajan1001
Peter Rowlands
Paweł Redzyński
David de la Iglesia Castro
Batuhan Taskaya
Guðmundur HeimissonandRuslan Kuprieiev
Matthew Beckers
eggqq007andZhenyu Gao
Michael Lingelbach