2 Messages
Anyone with experience using Collibra DQ on Databricks?
Hi. We are in the process of implementing Collibra DQ for the first time (we are in Australia) and have had a number of issues trying to use Collibra DQ with our Databricks environment in AWS. The first few times we were not able to run DQ jobs at all on Databricks and after re-installing different versions of Collibra DQ and Databricks drivers we are finally able to run DQ jobs but are experiencing some weird performance issues (e.g when you first click on a Data Source connection it in the UI, it takes 10 minutes for it to return).
If anyone else has implemented Collibra DQ with Databricks we would be very interested to hear about your experiences, whether you encountered any technical issues and how you overcame these.
Thanks
Rani
Community_Alex
675 Messages
•
13.2K Points
2 years ago
@rani.gerszonovicz Have you done a search for Databricks across all our forums (or even just with this ‘Data Quality’ one)? You can do that via the search icon on the top right of this page
. I did that and got these posts that were tagged with ‘databricks’ if any of those can help.
Alternatively, you should reach out to your CSM (I believe it’s @bhumika.mehta.collibra.com) to see if they can might also be able to track down some internal assistance here at Collibra.
0
0
ranigerszonovicz
2 Messages
2 years ago
Thanks Kristen. I think most of these articles are Collibra DGC related, not DQ but we will definitely take a look. We have already raised a support ticket for our specific Databricks connectivity/performance issue and Bhumika is across this but interested to hear about other customers’ experiences with DQ and Databricks in case there is something unique/different about our environment which is causing the issues.
0
0
muhammadsalahuddin
28 Messages
2 years ago
Thank you @rani.gerszonovicz. It would be great to hear from other clients about their experience with Collibra DQ and Databricks. Additionally, If we can get the performance metrics in terms of data volume and required compute power, it will certainly be helpful after go live. Thank you.
0
0