We used to have this line previously, thinking that it was only a py2
issue, but it turned out that it happens on py3 as well. So bring this
line back for now.
We should skip running post-checkout hook on rebases and merges to not
accidentally break them in the middle of the process. Git doesn't
provide any native mechanisms for hooks to know whether or not this is a
rebase/merge that we are going through, but we can use
`.git/rebase-merge` existance as a marker.
Previous approach has only accounted for branches and was causing issues
when checking-out tags.
Fixes#3241
It is important that we ensure that rwlocked contents are flushed to
disk, because otherwise other rwlock readers might get outdated state.
Kudos @shcheklein for catching this one.
Otherwise this will raise an exception from analytics worker:
```
dvc config cache.type sometypo
```
But running that same command with `--system` or `--global` flag outside
of repo won't raise that.
Also need to better handle ConfigObjError, but that is a separate issue.
Current implementation is triggering some non-obvious gc bug, that
causes memory leaks. Using win32file simplifies the code and gets rid of
this bug.
To reproduce it, run:
```
from dvc.system import System
while True:
System.inode(__file__)
```
and see your Windows machine run out of memory pretty quickly.
Imagine the scenario of shared cache. To set it up, user would do:
```
dvc config cache.dir /path
dvc config cache.shared group
```
Now, when he runs any dvc operation that involves the Repo instance,
RemoteLOCAL will also be initialized and it will currently auto-create
the directory with a regular `os.mkdir`, which is not using group
permissions. To solve that, we can't simply use `self.makdirs` instead,
as it will cause same issues if you run any dvc command before
`dvc config cache.shared group`. So what we should do is create
directories on demand, instead of creating it in advance, same as we do
in other remotes.
The change itself is pretty trivial and involved removing that `mkdir`
line, adjusting tests to not rely on cache dir existence and also
adjusting `dvc version` to not rely on cache directory existance.
Fixes#2563
Config files like `~/.zshenv`, `~/.bashenv` etc, are loaded on each
shell invocation, which might overwrite the environment breaking the
commands we are trying to wrap. To fix that, we need to provide
appropriate flags to the shell we are using to avoid loading those
configs.
Fixes#2506
Unlike PyInstaller, pyenv doesn't leave backups of original env vars
when it modifies them. If we look into the shim, pyenv and pyenv-exec,
we can figure out that the PATH is modified like this:
PATH=$PYENV_BIN_PATH:${bin_path}:${plugin_bin}:$PATH
where
PYENV_BIN_PATH - starts with $PYENV_ROOT, see pyenv-exec source code.
bin_path - might not start with $PYENV_ROOT as it runs realpath on
it, see pyenv source code.
plugin_bin - might contain more than 1 entry, which start with
$PYENV_ROOT, see pyenv source code.
So having this, we can make a rightful assumption about what parts of the PATH
we need to remove in order to get the original PATH.
Fixes#2506
This way dvc will actually launch the same script that it is running
from and not installed dvc module, that might not match.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
For the longest time we've had `repo.lock`, but were only using it from
CLI when running specific command, instead of simply properly taking it
on API operations. This patch takes care of that.
Leftover code was causing problems when working with symlinks, because
it tried to use `realpath` on symlinks to further use it to determine
parent directory.
Fixes#2476
This is an extension of our callback stages (no deps, so always
considered as changed) for stages with dependencies, so that these
stages could be used properly in the middle of the DAG. See attached
issue for more info.
Related to #2378
Current test is testing for signal handler restoration in an obscure
way. As a result, the test wasn't actually testing anything, because
KeyboardInterrupt is raised by default_int_handler, and so raising it
in a mock won't be caught by signal handlers, because they are dealing
with real singnals.
Fix#2406
Signed-off-by: Ruslan Kuprieiev <[email protected]>
In this mode, dvc will set appropriate file/dir permissions to support
sharing cache between different users within the same group.
Fixes#2287
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Related to #1629
Speeds up HDFS tests significantly. E.g. ExternalHDFS test goes from 600
sec to 190sec. test_open_external[HDFS] from 160sec to 3sec.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Fixes#2155
No longer need to explicitly ignore paramiko warnings.
Big thanks to @cowlinator for the heads-up!
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Fixes#2039
Required by #2012 to be able to fetch data through the API without
interuption from updater.
Signed-off-by: Ruslan Kuprieiev <[email protected]>
Ruslan Kuprieiev