Commit Graph
34 Commits
Author SHA1 Message Date
Alexander Schepanovski 6ccae711fe dvc: refactor config (#3298)
* dvc: refactor config

Advantages:
- read like a dict
- modify as a dict for testing or manipulation purposes
- wrap modifications into context manager to save them:

    with config.edit("local") as conf:
        conf["core"]["remote"] = "new-default"

Design decisions:
- dropped string constants
- remote sections are parsed into a simple subdict upon load
- remote config schema is now defined by url scheme
- relative paths are handled transparently upon load/save
- user config and remote manipulations are moved to commands
- same dev manipulations are done by simply modifying a config dict

* test: do not set cache.type for remotes that doesn't support it

* dvc: fix windows absolute paths validation in config

* test: fix windows separate drive test

* dvc: stylistic fixes in config
2020-02-13 19:35:37 +02:00
Ruslan Kuprieiev 242b5b18f6 Merge pull request #3193 from mroutis/fix-3192
config: verify before saving a modification
2020-01-27 18:08:53 +02:00
Paweł Redzyński 6557fed55d remote: verify cache exists on dir checksum 2020-01-24 13:37:33 +01:00
Jorge Orpinel d06bd6abd8 tests: match recent commits and
other misc. language improvements
2020-01-23 00:35:15 -06:00
Mr. Outis 566cc6c862 tests: move tests from remote to config 2020-01-19 02:29:07 -06:00
Mr. Outis efcddb6f5b remote: fix merging two different config levels 2020-01-19 01:56:24 -06:00
Mr. Outis 263e57e5e6 config: verify before saving a modification
Close #3192
2020-01-18 21:36:24 -06:00
Ruslan Kuprieiev 58cc31cbe4 Merge pull request #3134 from skshetry/refactor-local
test: merge local related function in Local helper class
2020-01-14 10:56:01 +02:00
Saugat Pachhai a27910d71d test: merge local related function in Local helper class 2020-01-14 14:04:02 +05:45
Mr. Outis ff094b2560 remote: error out when modifying a non-existing remote
Close #3131
2020-01-13 18:29:40 -06:00
Paweł Redzyński 04912f2c86 remote: test: migrate to dir helpers 2020-01-08 12:21:33 +01:00
Mr. Outis 086acf6404 py3: dvc/utils/compat.py -> dvc/compat.py 2019-12-31 23:42:07 -06:00
Paweł Redzyński 53163366b6 remote: s3: revert jobs adjustment, pass OSError to main 2019-12-17 11:41:10 +01:00
pawel 6ba4aba419 remote: base: [down/up]load: raise on too many open files 2019-12-17 11:41:03 +01:00
Saugat Pachhai 42991d3b75 Move more stuff from data_cloud to tests/remotes.py 2019-11-30 10:04:45 +05:45
Kaio Giurizatto Utsch 82a535b97e Change logging to print
Python logging recommends using print instead of logging for ordinary
output of command line program.

Also changed the test to reflect this.
2019-11-26 15:02:03 -03:00
Kaio Giurizatto Utsch 95924a34e6 Undo last commit
Turns out that the argument was needed after all
2019-11-26 14:19:09 -03:00
Kaio Giurizatto Utsch 9611107896 Remove unused argument 2019-11-25 17:51:49 -03:00
Kaio Giurizatto Utsch c2def64dc7 remote: add test case to show default 2019-11-25 17:35:45 -03:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
n3hrox 89502bee88 fix tests 2019-11-01 18:05:27 +01:00
Casper da Costa-Luis 60566b380e minor test update 2019-08-16 22:33:00 +01:00
Casper da Costa-Luis d0315a3509 minor black, test fixes 2019-08-11 00:09:18 +01:00
Jorge Orpinel c7ef6592fe output: remove a couple more "(s)" left
Per https://github.com/iterative/dvc/pull/2281#issuecomment-513053199
2019-07-20 13:44:36 -06:00
Alexander Schepanovski db75693fe1 dvc: return non zero error code on push/pull/fetch error
Closes #2038
2019-06-14 23:25:39 +07:00
Alexander Schepanovski d80293c44d dvc: path info refactoring
Replaced Path* dataclasses with PathInfo/URLInfo/CloudURLInfo, which
know how parse, serialize, stringify and join themselves. They also
provide useful accessors like part of an url or pathlib.Path like
.parent or .name.

Along the way two families of bugs where fixed and should never occur in
the future:
- paths not matched as strings anymore, e.g. using str.startswith()
- Output.path is eliminated as ambiguous

Both families might had caused undefined surprising behavior when local
and remote paths clash.
2019-06-11 19:19:17 +07:00
Ruslan Kuprieiev 9e9e078de3 config: move logic from CLI to API
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-06-06 14:41:02 +03:00
pawel 01c2f6c174 tests: rename dvc repo fixture 2019-05-21 15:04:43 -07:00
Ruslan Kuprieiev b4143b19dd remote: unify class names
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-12 17:29:38 +03:00
Ruslan Kuprieiev a593a9ce2e dvc: refactor PathInfos
Following same conventions from remotes/outsputs/dependencies.

Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-05-12 17:18:46 +03:00
pared 3ff10005fa path_info: remove dict path_info, introduce PathInfo class 2019-05-10 22:25:50 -07:00
pared 28f7ccfc6c remote: base: dump dir_info json with sorted keys 2019-05-01 23:34:52 -07:00
Alexander Schepanovski a6823e0113 dvc: add progress bar when hashing a large dir
Added an easier way to show progress bar over for loops.
2019-04-30 20:26:02 +07:00
Ruslan Kuprieiev 6f11f1f51f test: move func tests to tests/func
Signed-off-by: Ruslan Kuprieiev <[email protected]>
2019-04-16 20:19:22 +03:00