Commit Graph
2 Commits
Author SHA1 Message Date
skshetry 8768b50aed experimental: support connection strings for databases (#10125)
* experimental: support connection strings for databases using connectorx

This adds an experimental support for connector strings using connectorx.

User has to provide config for the connection strings with:

Example:
```console
dvc config --local db_connection.pg postgres://user:pass@localhost:5432/database
```

And then, they can import using:
```console
dvc import-db --sql 'query' --conn pg
```

Check supported databases and how to configure in https://sfu-db.github.io/connector-x/databases.html.
This `db_connection` config is subject to change.

connectorx still has the same problem with dbt as before: no support for streaming,
but it'll be a bit more simpler to configure with the connection strings.

* add pandas to deps

* sqlalchemy

* dbt streaming sql

* fix test

* remove connectorx

* remove check in set

* use mode='wb' for serialization
2023-12-03 19:23:09 +05:45
skshetry 3d7bbb8492 tests: add functional test for db (#10101)
* tests: add functional test for db

* fix test
2023-11-22 16:15:07 +05:45