Does anyone have experience using Data Bricks and how it can be integrated with Collibra?

Hi all Did anyone make use of the JDBC Driver for Apache Spark from Collibra Marketplace? I plan to use it for Databricks but the documentation does not explicitly mention the details of technical user to extract metadata. It says “Username” & “Password” are needed unlike the “Personal Access To

Hi. We are in the process of implementing Collibra DQ for the first time (we are in Australia) and have had a number of issues trying to use Collibra DQ with our Databricks environment in AWS. The first few times we were not able to run DQ jobs at all on Databricks and after re-installing different

Has anyone registered databricks JDBC connection via EDGE in DGC? Since there is no schemas concept in databricks, does that mean I should register as many JDBC connections as the number of databases in databricks? If so, so far so good. But the metadata ingestion capability seems to be having one-t

Hey fellow Collibra users. We’re looking to implement data profiling, data sampling, and metadata ingestion of an AWS-hosted Databricks env using Collibra Edge and Databricks’ JDBC Driver(https://www.databricks.com/spark/jdbc-drivers-download) into our Collibra Env. Our Solution Architecture team wo

Hello Folks, I have seen a brief youtube video on integration between CollibraDQ and Databricks through Spark Notebook. Is it possible to get the sample code for further understanding. Additioally, it would be great, if someone can post the link of development documentation as well. Thank you so muc

Hi, I’ve successfully run the integration in Q4 2022 with version 1.0.7 by building project with maven, setting Collibra and Databricks accounts, tokens and permissions and running the springboot integration. I’ve noticed that there was a new 2.0.9 available, so I wanted to test it out to learn abou

Hello Fellow Data Citizens, I hope you are well! I have read that Spark-Submit is an available option to execute CollibraDQ jobs. However, It has some limitations. Can I please request more clarification/information. Thank you. Spark-Submit path is not supported in Production by Collibra support te

We have enabled database views in our spark connection for Databricks. However, the views are being imported with the asset type “Table” rather than “View” or “Database View.” Is there a way to change the asset type for the connection?

Hello all, Wanted to check on how others are dealing with generating lineage in Collibra from databricks pipelines that are written using Spark Languages like Scala, Python. I see the lineage harvester has an out of the box connectivity for Spark SQL. Converting the pipeline code written in Python o

Hello, all I tried to connect Azure Databricks using Collibra JDBC Driver. I’ve set up the connection via the Catalog successfully with the Driver downloaded from https://marketplace.collibra.com/listings/jdbc-driver-for-databricks/ But I got an error. I guess it might because of the single quote

Hello friends, We are trying to migrate our data quality rules and assets from Informatica to Collibra.  Can somebody please help us to how effectively it can be done. We have large number of rules, so that manually recreating them will not be working. Thanks in advance Haz

Hello Data Citizens, I hope you are well Can someone provide an update on the release plan of Databricks Pushdown for CollibraDQ. I have heard that it will be available as part of May release. Thank you. Cheers, Muhammad

Hi all, Does anyone know where the location is for the owl python libraries? Testing out calling a DQ job from a python notebook and can’t find the libs Thanks Shane

Hi all Wanted to check if anyone has made an attempt to exploit the Hive Metastore of Databricks for lineage? Today, I loaded metadata of 2 databricks databases using the Collibra Marketplace provided Databricks driver. Here is the scenario - Database 1 > Table_A Database 2 > View_A based on T

Dear all, The Spark JDBC driver (SparkJDBC42.jar) is unable to capture certain information from the below table structure: table level comment the TBLPROPERTIES key-value pair information PARTITION BY information However, it captures the column level comment (eg. the comment against employee

We are synchronizing schemas with Databricks via Edge, we have noted that Database views are getting converted into Table after the synchronization. In Databricks its database view but in Collibra it showing as table. Does anyone had faced similar issue?

Hi All, I am wondering if anyone can help me out with a Databricks Domain Mapping functionality. Currently, all metadata (schemas) ingested from Databricks UC is very messy and all lie inside a root community. I have tried moving each individual Physical Data Dictionary to the target domain and afte

Hi all is it possible to display the domain of an asset as a box in a diagram? Currently, I think the diagram boxed view for a column is arranged in this way : System > Database > Table > Column I want to see the domain name appearing as an extra box above Table. ((This requirement is for b

Question: Does Collibra DQ support Databricks as a JDBC datasource for a DQ Connector? Answer from Vadim Vaks: No. There is no JDBC Spark Databricks driver packaged with Collibra DQ because we don’t support it. So regarding this JDBC-Spark template in the Collibra DQ > Admin Console > Connect