Commit Graph
9 Commits
Author SHA1 Message Date
Saugat PachhaiandRuslan Kuprieiev 2fecc66cb9 addressing: allow stage name addressing without ":" (#3842)
* addressing: allow stage name addressing without colon

* repo: reorganize imports

* refactor/reorganize collect_granular

* add tests for stage addressing

* fix ds and cc issues

* fix unpack error in test

* tests: stage string representations

* exception: add error message

Also remove `cause` when throwing NoOutputOrStage, as
CheckoutErrorSuggestGit will start chaining all the messages
in cause of the exception.

* rename NoOutputOrStage NoOutputOrStageError

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-05-26 03:51:51 +03:00
Casper da Costa-Luis 2bb7257a06 lint: isort pre-commit & misc improvements (#3749)
* test: clean flake8 config root clutter

* test: pre-commit: add isort

Compatible with black (see https://github.com/timothycrosley/isort/issues/694)

* lint: reorder imports

* lint: more complex imports

* lint: fix deepsource

* config: isort: add first party dvc

* tests: restyled: add isort

* temp commit to revert before merge

* Revert "temp commit to revert before merge"

This reverts commit 11db0fd8e95fe891597cbb9bf804480909b19e24.
2020-05-06 23:56:07 +03:00
Ruslan Kuprieiev 93d7b9173b tests: use --single-stage 2020-05-04 20:23:49 +03:00
7275843c08 remote: use .dir checksum existence to infer file contents existence (#3632)
* repo: separate dir cache and file cache in memory

- `used_cache()`/`get_used_cache()` in repo/stage/output now return
  tuples of (dir_cache, file_cache) instead of one flat/merged cache

* update tests for new get_used_cache behavior

* remote: if .dir checksum exists on remote, assume contents also exists

- affects all commands which use `cache_exists()` (remote status)

* push: only upload .dir file after full file contents has been uploaded

* gc: always remove .dir checksums first

* functional tests for push/gc

* repo: support nesting dir caches in NamedCache

* remote: NamedCache updates

* Fix tests

* Fix deepsource warnings

* tests: use pytest mocker fixture

* Update dvc/cache.py

Co-Authored-By: Saugat Pachhai <[email protected]>

Co-authored-by: Ruslan Kuprieiev <[email protected]>
Co-authored-by: Saugat Pachhai <[email protected]>
2020-04-14 23:07:58 +03:00
Alexander SchepanovskiandRuslan Kuprieiev a4094f0a7f dvc: add Repo(skip_checks) flag (#3490)
* dvc: add Repo._skip_graph_checks flag

Will skip modified graph checks on add, run and import. The aim is to
avoid collecting stages and building graph on these commands.

The skip_checks flag is aimed to be private, but let people use dvc on
big repos without monkey patching or writing yaml files themselves.

* dvc: explain skip_graphs_checks flag in a comment

* test: that skip_graph_checks works

* repo: improve hack description and test

Co-authored-by: Ruslan Kuprieiev <[email protected]>
2020-03-24 01:39:54 +02:00
Alexander Schepanovski e69b44d1ed dvc: optimize all target specific commands to not build graph (#3459) 2020-03-15 21:53:38 +02:00
Paweł Redzyński 9b2ef002ea Stage: create: reset repo after new stage creation (#3349) 2020-02-19 20:25:04 +02:00
Mr. Outis 753aef4910 black: bump version 19.3b0 -> 19.10b0 (#3307) 2020-02-12 02:41:24 +02:00
Ruslan Kuprieiev 2361bceb70 repro: fix bug in _get_active_graph (#3301)
We are currently removing all of the deps stages from the DAG even if
those deps are referenced by some parallel branches of execution.
The new approach is to remove edges and then gradually remove nodes if
there are no more `in` edges left.

Discord context:
https://discordapp.com/channels/485586884165107732/485596304961962003/676464313942409265
2020-02-11 01:45:43 +02:00