REVIEWED-INTERNAL ONLY: Does Collibra DQ support Databricks as a JDBC datasource for a DQ Connector?

This is what Brian told us on April 13:

We’ve seen numerous customers request to access their Databricks Delta Lake via JDBC. Based on the recurring feedback, we decided to prioritize the development. The expected release date is 2022.05 for a default-packaged Databricks JDBC connector.

There are 3 primary ways customers desire to use their Databricks infrastructure. Below is how customers can run DQ with Databricks infrastructure.

JDBC

https://docs.databricks.com/integrations/bi/jdbc-odbc-bi.html#jdbc-driver

- We intend to add the driver packaging to our containers starting 2022.05 release.

NOTEBOOK

https://docs.databricks.com/notebooks/index.html

- We will continue to support this. This has been used for several years.

SPARK SUBMIT / JOB API

https://docs.databricks.com/dev-tools/api/latest/examples.html#create-a-spark-submit-job

https://docs.databricks.com/dev-tools/api/latest/jobs.html

- At this time, we are not able to prioritize work for building custom spark submissions.

Q: How can I access my Databricks Delta Lake cluster via JDBC?
Answer from Brian Mearns:
Based on the recurring feedback, we decided to prioritize the development. The expected release date is 2022.05 for a default-packaged Databricks JDBC connector.

There are 3 primary ways customers desire to use their Databricks infrastructure. Below is how customers can run DQ with Databricks infrastructure.

1. JDBC

https://docs.databricks.com/integrations/bi/jdbc-odbc-bi.html#jdbc-driver

- We intend to add the driver packaging to our containers starting 2022.05 release.

2. NOTEBOOK

https://docs.databricks.com/notebooks/index.html

- We will continue to support this. This has been used for several years.

3. SPARK SUBMIT / JOB API

https://docs.databricks.com/dev-tools/api/latest/examples.html#create-a-spark-submit-job

https://docs.databricks.com/dev-tools/api/latest/jobs.html

- At this time, we are not able to prioritize work for building custom spark submissions.

From the Review Team: “Internal only. This is a complex topic that is still being defined. We will not move this to documentation or Help Center content.”

Laurent, tagging the review team: @evan.nowlin @julianna.guzman.collibra.com @eliana.valderrama FYI