Stone_Red/dvc
1
0
Fork 0
mirror of https://github.com/Stone-Red-Code/dvc.git synced 2026-11-02 07:18:25 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,950 Commits 7 Branches 579 Tags
c25715dc729f49e5885564c28d0bdb01b0e1ff02
Commit Graph
3950 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruslan Kuprieiev fdf0a8e99a Merge pull request #2562 from efiop/pyenv 
run: improve pyenv PATH modification criteria
 2019-10-02 10:16:52 +03:00
Ruslan Kuprieiev 132de572af run: improve pyenv PATH modification criteria 
https://discordapp.com/channels/485586884165107732/485596304961962003/628766062472724510

Related to #2506
 2019-10-02 06:33:01 +03:00
Ruslan Kuprieiev 8fb3eaaa08 dvc: bump to 0.61.0 0.61.0 2019-10-01 10:56:00 +03:00
Ruslan Kuprieiev 39104a1dee Merge pull request #2557 from efiop/2506 
run: make shell not load config files
 2019-10-01 07:59:07 +03:00
Ruslan Kuprieiev ee976a1c1a run: make shell not load config files 
Config files like `~/.zshenv`, `~/.bashenv` etc, are loaded on each
shell invocation, which might overwrite the environment breaking the
commands we are trying to wrap. To fix that, we need to provide
appropriate flags to the shell we are using to avoid loading those
configs.

Fixes #2506
 2019-09-30 19:34:55 +03:00
Mr. Outis 9496f34f45 tests: Windows specific filename for gitignore 2019-09-30 11:28:38 -05:00
Mr. Outis 80cb1c919a git: escape question mark character 2019-09-30 11:27:53 -05:00
Mr. Outis e692be2dc9 sftp: add a more descriptive error on mkdirs (#2554) 
* sftp: add a more descriptive error on mkdirs

Fix #2535

* sftp: raise exception instead of logging one
 2019-09-30 19:24:31 +03:00
Mr. Outis a9201f8237 git: add a note referencing gitignore pattern 2019-09-30 11:21:39 -05:00
Paweł RedzyńskiandRuslan Kuprieiev 119548e2a1 utils: fix_env: dont delete LD_LIBRARY_PATH from env (#2541) 
* utils: fix_env: dont delete LD_LIBRARY_PATH from env

* utils: add related issue to comment on why there is empty LD_LIBRARY_PATH

* Update dvc/utils/__init__.py

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* scm: git: move LD_LIBRARY_PATH fix to Git.clone method
 2019-09-30 15:28:36 +03:00
Ruslan Kuprieiev f0de362e0d Merge pull request #2548 from iterative/py27 
fix py27 compat
 2019-09-30 08:35:29 +03:00
Mr. Outis c9c36c2049 git: escape filename before ignoring it 
Fix #2043
 2019-09-29 23:16:46 -05:00
Casper da Costa-Luis 1f46ca2463 silly import fix 2019-09-29 17:56:52 +01:00
Casper da Costa-Luis 5f65e8096e back to initial approach 2019-09-29 17:54:04 +01:00
Casper da Costa-Luis 83f55de7a9 move compatibility helper 2019-09-29 16:43:37 +01:00
Ruslan Kuprieiev beea7e56f2 Merge pull request #2543 from casperdcl/checksum 
remove unneeded TqdmThreadPoolExecutor
 2019-09-29 09:15:01 +03:00
Ruslan Kuprieiev e3450257c6 Merge pull request #2544 from yfarjoun/patch-1 
Added a space in error message
 2019-09-29 09:13:10 +03:00
Casper da Costa-Luis a0d99e2486 remove unneeded TqdmThreadPoolExecutor 
- fixes #2539
 2019-09-28 23:54:14 +01:00
Casper da Costa-Luis 4c01a86302 fix py27 compat 2019-09-28 23:12:55 +01:00
Yossi Farjoun 6df4188bd2 Added a space in error message 
als made consistent the placement of inter-word space when there's a newline in the code
 2019-09-28 11:42:03 -04:00
Ruslan Kuprieiev 5c76b46b46 Merge pull request #2538 from mroutis/fix-2524 
py2: display friendly error when raising a UnicodeError
 2019-09-28 03:35:14 +03:00
Mr. Outis 39dc79b418 main: fix UnicodeError misplacement 2019-09-27 18:20:00 -05:00
Mr. Outis 0bdad65f04 main: make sure to raise UinocedeError for py3 2019-09-27 14:18:08 -05:00
Ruslan Kuprieiev 22afc48d9b Merge pull request #2540 from Suor/repo-imports 
perf: import dvc.repo dynamically
 2019-09-27 12:02:21 +03:00
Alexander Schepanovski 0203fd4017 perf: import dvc.repo dynamically 2019-09-27 15:08:42 +07:00
Mr. Outis 1763607a88 py2: display friendly error when raising a UnicodeError 2019-09-26 19:50:15 -05:00
Ruslan Kuprieiev 4061d4aff1 dvc: bump to 0.60.1 0.60.1 2019-09-26 18:34:06 +03:00
Ruslan Kuprieiev 3eb23870d2 build: posix: don't forget to install optional psutil 
Fixes https://github.com/iterative/dvc/issues/2537
 2019-09-26 18:33:36 +03:00
Ruslan Kuprieiev 381601f4ee dvc: cache graph/stages/pipelines (#2529) 
* dvc: cache graph/stages/pipelines

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* repo: check_dag: don't reset the cache

https://github.com/iterative/dvc/pull/2529#discussion_r327711928

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* repo: rename check_dag(stages) -> check_modified_graph(new_stages)

https://github.com/iterative/dvc/pull/2529#discussion_r327708439

Signed-off-by: Ruslan Kuprieiev <[email protected]>

* repo: refactor "active" logic and remove it from Repo class

* repo: use tree.setter to automate cache reset

As noted by @Suor's , this is much less prone to errors.

* repo: move graph-related staticmethods into graph.py

As per @Suor's request.

* dvc: remove unused from_directory

https://github.com/iterative/dvc/pull/2529#discussion_r328079258

* repo: use _tree instead of __tree

https://github.com/iterative/dvc/pull/2529#discussion_r328071115

* repo: add a note about cache invalidation on repo.lock release

https://github.com/iterative/dvc/pull/2529#discussion_r328071635

* repo: rename reset() -> _reset()

https://github.com/iterative/dvc/pull/2529#discussion_r328079899

* pipeline: use graph helpers

https://github.com/iterative/dvc/pull/2529#discussion_r327502293

* repo: use list comprehension for collect()

https://github.com/iterative/dvc/pull/2529#discussion_r328076651

* repo: graph: only use first cycle

https://github.com/iterative/dvc/pull/2529#discussion_r328081346

* dvc: don't use G.nodes()

https://github.com/iterative/dvc/pull/2529#discussion_r328089671
https://github.com/iterative/dvc/pull/2529#discussion_r328089878
 2019-09-26 07:42:29 +03:00
Ruslan Kuprieiev fea05b3924 Merge pull request #2536 from mroutis/contributing 
move .github/CONTRIBUTING.md -> CONTRIBUTING.md
 2019-09-26 06:27:30 +03:00
Mr. Outis 5c5dfed68f .github/CONTRIBUTING.md -> CONTRIBUTING.md 2019-09-25 21:33:46 -05:00
Ruslan Kuprieiev 987cf883c5 dvc: bump to 0.60.0 0.60.0 2019-09-25 11:12:20 +03:00
Ruslan Kuprieiev dcce70f4b9 output: base: checkout: make returns consistent 
As reported by deep source: Either all return statements in a function should return an expression, or none of them should.
 2019-09-24 16:33:15 +03:00
Paweł Redzyński be012e70cf checkout: raise on non existing dir cache (#2514) 
* checkout: raise on non existing dir cache

* checkout: don't throw on failed load_dir_cache

* checkout: raise on failed checkouts, only after all checkouts are perfromed

* remote: base: raise on failed dir cache loading

* rename exception for failed checkout
 2019-09-24 16:31:36 +03:00
Ruslan Kuprieiev 7409fadb84 Merge pull request #2527 from Suor/delay-cache 
perf: delay cache creation
 2019-09-24 08:11:45 +03:00
Alexander Schepanovski e2eb4ae0c3 perf: delay cache creation 
The older code instantiates non-local cache remotes on Repo creation if
any of those are configured. And remote instantiation loads their API
libs.
 2019-09-24 09:03:03 +07:00
Ruslan Kuprieiev 33f1cdfab8 Merge pull request #2525 from efiop/2506 
dvc: remove PATH prefixes appended by pyenv
 2019-09-23 19:35:46 +03:00
Ruslan Kuprieiev 313fdce5fa Merge pull request #2528 from Suor/datacloud-dead-code 
dvc: remove dead code in DataCloud
 2019-09-23 16:26:44 +03:00
Alexander Schepanovski 4006c6434e dvc: remove dead code in DataCloud 2019-09-23 20:00:19 +07:00
Ruslan Kuprieiev bd2b5900a1 Merge pull request #2526 from efiop/2495 
dvc: remote: optimize imports
 2019-09-23 12:44:09 +03:00
Ruslan Kuprieiev fd14ca3269 dvc: remote: optimize imports 
Decreases `time dvc --version` from ~1.4 sec to ~0.3 sec.

Related #2495
Related #2445

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-09-23 11:30:45 +03:00
Ruslan Kuprieiev 43975c8618 dvc: remove PATH prefixes appended by pyenv 
Unlike PyInstaller, pyenv doesn't leave backups of original env vars
when it modifies them. If we look into the shim, pyenv and pyenv-exec,
we can figure out that the PATH is modified like this:

    PATH=$PYENV_BIN_PATH:${bin_path}:${plugin_bin}:$PATH

where

    PYENV_BIN_PATH - starts with $PYENV_ROOT, see pyenv-exec source code.
    bin_path - might not start with $PYENV_ROOT as it runs realpath on
        it, see pyenv source code.
    plugin_bin - might contain more than 1 entry, which start with
        $PYENV_ROOT, see pyenv source code.

So having this, we can make a rightful assumption about what parts of the PATH
we need to remove in order to get the original PATH.

Fixes #2506
 2019-09-23 04:25:42 +03:00
Ruslan Kuprieiev 70efa6c52a Merge pull request #2523 from efiop/daemon 
daemon: use `sys.argv[0]` instead of `-m dvc`
 2019-09-22 05:42:56 +03:00
Ruslan Kuprieiev 0347cfb891 gitignore: don't forget to add /tmp 2019-09-22 05:22:55 +03:00
Ruslan Kuprieiev 31b40816b5 daemon: use sys.argv[0] instead of -m dvc 
This way dvc will actually launch the same script that it is running
from and not installed dvc module, that might not match.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
 2019-09-22 05:18:51 +03:00
Mr. OutisandRuslan Kuprieiev ae90904fb3 pull: download data from imported stages (#2499) 
* wip - pull is working

* use fetch instead of reproduce

* test pull and fetch

* style changes

* fetch: catch config error on fetch instead of get_remote

* output: remove unused guard clause

* fetch: define a more specific error instead of ConfigError

* fetch: display number of errors correctly

* exceptions: remove "Failed" prefix to download/upload

* fetch: catch other exceptions that might halt the download

* fetch: log catched exceptions

* fetch: reformat exception

Co-Authored-By: Ruslan Kuprieiev <[email protected]>

* fix formatting
 2019-09-22 01:28:12 +03:00
Ruslan Kuprieiev 23fc2662d0 Merge pull request #2519 from efiop/755 
dvc: move repo lock from CLI to API
 2019-09-21 19:12:34 +03:00
Ruslan Kuprieiev 9d6ca1ef6c dvc: move repo lock from CLI to API 
For the longest time we've had `repo.lock`, but were only using it from
CLI when running specific command, instead of simply properly taking it
on API operations. This patch takes care of that.
 2019-09-21 05:10:06 +03:00
Ruslan Kuprieiev 64c3973522 Merge pull request #2517 from Suor/later-imports 
perf: import some things later
 2019-09-20 18:00:01 +03:00
Ruslan Kuprieiev f1d3f4344e Merge pull request #2494 from Suor/lazy 
perf: do not use pkg_resources
 2019-09-20 13:47:47 +03:00