Commit Graph
57 Commits
Author SHA1 Message Date
Alexander Schepanovski 6ccae711fe dvc: refactor config (#3298)
* dvc: refactor config

Advantages:
- read like a dict
- modify as a dict for testing or manipulation purposes
- wrap modifications into context manager to save them:

    with config.edit("local") as conf:
        conf["core"]["remote"] = "new-default"

Design decisions:
- dropped string constants
- remote sections are parsed into a simple subdict upon load
- remote config schema is now defined by url scheme
- relative paths are handled transparently upon load/save
- user config and remote manipulations are moved to commands
- same dev manipulations are done by simply modifying a config dict

* test: do not set cache.type for remotes that doesn't support it

* dvc: fix windows absolute paths validation in config

* test: fix windows separate drive test

* dvc: stylistic fixes in config
2020-02-13 19:35:37 +02:00
Alexander Schepanovski 0eb4e7e15f api: move summon out
Will be added to dvcx instead.
2020-01-31 00:02:28 +07:00
Dmitry Petrov e6d0d291e2 Code review feedback and change schema to dict of dvc-objects 2020-01-28 18:44:36 -08:00
Dmitry Petrov d5a3b6f4f6 Summon dataframe support 2020-01-26 02:50:42 -08:00
Dmitry Petrov 8f799210d3 Extract Summon class 2020-01-24 16:35:17 -08:00
Saugat Pachhai 003b5c392a api: fallback to external_repo on error 2020-01-23 14:44:38 +05:45
Jorge Orpinel ec0c39a28e term: review capitalization and backquotes around "dvc"
rel: https://github.com/iterative/dvc/pull/3156#discussion_r369897073
2020-01-22 20:27:01 -06:00
Alexander Schepanovski 72812b519e api: fix get_url() non-dvc repo error message 2020-01-19 23:01:53 +07:00
Alexander Schepanovski ee022eada0 api: adjust get_url requiring dvc repo to the new reality 2020-01-19 21:38:37 +07:00
Saugat Pachhai 50206d44dc test: fix test related to path on windows
On windows, `str(exc_info)` is returning escaped path, eg:
'<ExceptionInfo UrlNotDvcRepoError("URL \'C:\\\\Users\\\\travis\\\\AppData\\\\Local\\\\Temp\\\\pytest-of-travis\\\\pytest-0\\\\popen-gw2\\\\test_get_url_git_only_repo_thr0\' is not a dvc repository.") tblen=2>'

This commit just replaces the double slashes with single one. Bit of a hack, I know.
2020-01-16 21:06:52 +05:45
Saugat Pachhai aa7119a741 get: test get_url raises exception if not a dvc repo 2020-01-16 19:28:14 +05:45
Ruslan Kuprieiev 8041d70552 Merge pull request #3044 from pared/2896_api
tests: api: convert to dir helpers
2020-01-06 16:21:31 +02:00
Alexander Schepanovski 5dd2d7b192 test: do not use repo template in erepo_dir 2020-01-04 10:47:13 +07:00
Alexander Schepanovski b194bfa1ff test: do not bootstrap versions in erepo_dir 2020-01-04 10:47:13 +07:00
pawel af425318aa tests: api: roll back erepo usage 2020-01-04 03:10:43 +01:00
pawel 020c0496a6 tests: api: convert to dir helpers 2020-01-04 03:08:51 +01:00
pawel d9eaabe9bf some fixmes 2020-01-04 03:08:51 +01:00
pawel 39f010e50a api: open: fix open scm controlled test 2020-01-04 03:08:51 +01:00
Alexander Schepanovski 1e5fb102a2 test: add @TmpDir.chdir() 2020-01-03 16:58:32 +07:00
Mr. Outis 086acf6404 py3: dvc/utils/compat.py -> dvc/compat.py 2019-12-31 23:42:07 -06:00
Mr. Outis 49264b55ac py2: fix problem with ruamel.yaml and unicode 2019-12-25 13:07:40 -06:00
Mr. Outis 4a25612282 💅 refactor 2019-12-25 01:04:52 -06:00
Mr. Outis 75ad14aaa2 💅 black 2019-12-25 00:43:33 -06:00
Mr. Outis 0658622ca2 summon: pair-programming adjustments 2019-12-25 00:42:55 -06:00
Mr. Outis 1e65c76ef1 summon: raise SummonError appropiately 2019-12-24 20:20:03 -06:00
Mr. Outis fbb7aebd31 tests/summon: use different args for non-canon.yml 2019-12-24 19:50:43 -06:00
Mr. Outis 1c22cd5af4 summon: adjust comments 2019-12-24 02:20:08 -06:00
Mr. Outis addc56c232 summon: allow different YAML file names 2019-12-18 23:38:52 -06:00
Mr. Outis c852d3316d 💅 black 2019-12-18 23:34:43 -06:00
Mr. Outis 4273ff49b3 summon: params -> args; use summon key in yaml 2019-12-18 23:32:28 -06:00
Mr. Outis 29339d29a2 summon: file & method -> call 2019-12-18 23:25:45 -06:00
Mr. Outis 1b3c771df5 summon: artifacs.yaml -> dvcsummon.yaml; artifact -> object 2019-12-18 22:42:55 -06:00
Mr. Outis 32f35ea686 summon: remove scm.add, *_gen already handles it 2019-12-18 22:29:44 -06:00
Mr. Outis 5d66a572d4 summon: support overriding parameters 2019-12-18 17:20:07 -06:00
Mr. Outis d0344ce975 summon: grimoire/spells -> artifacts 2019-12-18 17:00:39 -06:00
Mr. Outis 789db9169a summon: support checking out dependencies 2019-12-18 16:41:03 -06:00
Mr. Outis b921730f37 summon: safely get a module; change naming 2019-12-18 13:32:23 -06:00
Mr. Outis 3d133186d4 summon: ugly hack 2019-12-18 00:48:10 -06:00
c27988a042 test: minor fixture refactor fixes from @pared and @efiop
Co-Authored-By: Ruslan Kuprieiev <[email protected]>
Co-Authored-By: Paweł Redzyński <[email protected]>
2019-12-02 23:33:44 +01:00
Alexander Schepanovski 0385810662 test: fix python 2, 3.5 and mac issues with new fixtures 2019-12-02 23:32:11 +01:00
Alexander Schepanovski fcebad821e test: refactor tmp dir helper fixtures
- `tmp_dir` descends from Path
- convenient template methods to generate files and dirs, add them to
dvc and git and commit to git
- independent of old basic_env structures
- can go in any order
- modular: start with empty `tmp_dir`, initialize git with `scm`, dvc
repo with `dvc` add "repo template" with `repo_template`, ...
2019-12-02 23:31:54 +01:00
Saugat Pachhai 22c0a644e5 address @efiop's suggestions 2019-12-01 16:08:53 +05:45
Saugat Pachhai f4f836e17c Merge branch 'master' into gs-external-deps
* master:
  travis: windows: use python 3.7.5
  test: skip non supported remotes fast in api tests
2019-12-01 09:01:47 +05:45
Saugat Pachhai 63cf117ee9 Move to tests/remotes.py 2019-11-30 08:58:43 +05:45
Saugat Pachhai 4714929b5b Merge branch 'master' into gs-external-deps
* master:
  address @efiop's suggestions
  s3: add support for top level empty directories
  remote: http: calculate length basing on response and not head call
  test: remote: http: clear test file name
  HTTPError: reformat error message
  Update dvc/remote/http.py
  remote: http: rename http error, refactor test
  remote: http: raise exception when download response with error status code
  Support GDrive as remote
  setup: don't forget contextlib2
  remote: refactor ssh ask password code
  remote: protect all remote client/session creation code with locks
  test: refactor & remove redundant test fixtures
  NoRemoteInExternalRepoError: dont pass cause of exception
  perf: optimize cache listing for local, ssh and hdfs
  remote: small .save_info()/.get_checksum() cleanup
2019-11-30 08:39:56 +05:45
Saugat Pachhai caa67c725e dvc: refactor tests 2019-11-30 08:22:39 +05:45
Alexander Schepanovski 865ed6f8ea test: skip non supported remotes fast in api tests 2019-11-29 23:42:26 +01:00
Dani Hodovic 78c505f6cb test: refactor & remove redundant test fixtures
https://github.com/iterative/dvc/pull/2837#discussion_r351318487
2019-11-28 19:04:56 +01:00
Ruslan Kuprieiev 95fe1578bb dvc: automatically reorder imports 2019-11-05 02:34:40 +02:00
Paweł Redzyński d2f8cb0520 api: open scm controlled files (#2566)
* tree: make open method use mode and encoding

* api: try to open using self.tree if output not found

* repo: rename OutputMissingException

* test: api: test adding not cached output

* api: open: exists test instead of try

* repo: open: try opening from tree first

* repo: open: handle missing and not cached files

* repo: open: separate find out logic from opening
2019-10-08 15:23:42 +03:00