Database Views are brought in with Asset Type Table in Spark connection

We have enabled database views in our spark connection for Databricks. However, the views are being imported with the asset type “Table” rather than “View” or “Database View.” Is there a way to change the asset type for the connection?

ashleymctavish

Posted 4 years ago · Edited 1 year ago·Last reply 3 years ago

5 comments

ashleymctavish

OP4 years ago · Edited

Thank you Noor for all your help!

ashleymctavish

OP4 years ago · Edited

Hi Noor,
I checked the Table Type attribute and that also shows “Table”. And yes, we are using the Simba Spark JDBC driver for Databricks.

noor

·4 years ago · Edited

Please use “Apache Spark” driver from marketplace for databricks. We initially used the simba driver. It did not fetch table level descriptions from databricks ((I recall I was using Jobserver at that time)).

So, then we used “Apache Spark” driver. It is provided by CDATA to Collibra. Again, this driver did not fetch views from databricks, although it was able to fetch table level descriptions.

So, finally, after few weeks, CDATA fixed the issue, and the driver could fetch views also from databricks, against the asset type “Database View”

I think the updated driver will soon be available in Marketplace. You can talk to your CSM for it. Please refer the ticket I raised during talks with your CSM. It is 85248.

By the way, I use now EDGE for metadata ingestion.

sumagirish

·3 years ago · Edited

Hi Noor,
I am in a similar situation and would like to ingest Views from our Azure Databricks. On the edge site it only shows me tables in a Schema and no Views. Please explain how you are able to get views using Edge

noor

·4 years ago · Edited

Please check the attribute “Table Type”. Is that VIEW, at least?

Are you using the Simba driver for databricks available on Marketplace?