Unlike PyInstaller, pyenv doesn't leave backups of original env vars
when it modifies them. If we look into the shim, pyenv and pyenv-exec,
we can figure out that the PATH is modified like this:
PATH=$PYENV_BIN_PATH:${bin_path}:${plugin_bin}:$PATH
where
PYENV_BIN_PATH - starts with $PYENV_ROOT, see pyenv-exec source code.
bin_path - might not start with $PYENV_ROOT as it runs realpath on
it, see pyenv source code.
plugin_bin - might contain more than 1 entry, which start with
$PYENV_ROOT, see pyenv source code.
So having this, we can make a rightful assumption about what parts of the PATH
we need to remove in order to get the original PATH.
Fixes#2506
This is an extension of our callback stages (no deps, so always
considered as changed) for stages with dependencies, so that these
stages could be used properly in the middle of the DAG. See attached
issue for more info.
Related to #2378
* http: add exception when trying to traverse remote
Fix#2380
* http: exception raise on `list_cache_paths` instead of `all`
`NotImplementedError` -> `RemoteActionNotImplemented`
* http: raise exception on `__init__`
* set default no_traverse to True on remote base
* tests: explicitly use traverse on data cloud and no traverse with HTTP
* remote: define a new RemoteConfigError
* remote: reuse ConfigError instead of RemoteConfigError
* tests: set no_traverse to False with SSHMocked
* deepsource: fix minor issue with spacing
* remove blankspace
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
Current test is testing for signal handler restoration in an obscure
way. As a result, the test wasn't actually testing anything, because
KeyboardInterrupt is raised by default_int_handler, and so raising it
in a mock won't be caught by signal handlers, because they are dealing
with real singnals.
Fix#2406
Signed-off-by: Ruslan Kuprieiev <[email protected]>
This employs generating temporary signed public url and then streaming
from it. The bulk of the code might be reused for other remotes.
Url streaming does resume on connection error using "Range" header and
regenerates public url too.
If remote class doesn't provide .open() then this falls back to pulling
a file and opening it locally.
Related to #1629
Speeds up HDFS tests significantly. E.g. ExternalHDFS test goes from 600
sec to 190sec. test_open_external[HDFS] from 160sec to 3sec.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* remote: drop not working http resume
Closes#2136
* remote: drop @transfer_context()
It is not needed for ssh anymore and GS/S3 are switched to client
instances in cached properties.
* remote: drop upload/download chunking
* state: use ignore file handler when accessing metadata
* fs: don't add dir size to overall size for dir
* ignore: use posix-like path on windows
* ignore: remove tree from dvc_walk arguments
* ignore: handle DvcIgnoreFilter creation outside of DVC repo
* ignore: throw on dvcignore file under output dir, but not dependency dir
* ignore: move ignore initialization to Repo init
* ignore: work in progress
* ignore: work in progress 2
* ignore: throw on .dvcignore upon collecting dir
* ignore: refactor
* ignore: loading upper level ignores in stages()
* Update dvc/repo/__init__.py
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
* dvcignore review refactor
* erepo: dvc: implement `update` for imports from ext repos
Fixes#1774Fixes#2139Fixes#2201
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* deps: repo: make private
Signed-off-by: Ruslan Kuprieiev <[email protected]>
* update: only support repo imports
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Mr. Outis
Ruslan Kuprieiev
Ruslan Kuprieiev
pawel
joshua postel
Casper da Costa-Luis
Alexander Schepanovski
Ivan Shcheklein