(July 2022) Collibra Data Quality Product Roadmap Webinar

On July 12, 2022, we hosted a Product Roadmap Webinar: Collibra Data Quality & Observability. Exclusively for Collibra Data Quality customers, attendees were able to meet the product team, ask questions, and provide feedback.

During this webinar, Brian Mearns, Director, Product Management of Data Quality shared:

• New product capabilities and enhancements we’ve implemented over the last 12 months

• Visibility into major product roadmap highlights and our vision for the future

Here’s the replay:

Additional resources referenced in the presentation:

• Collibra Data Quality User Guide

• DQ Connector: No-code DQ integration to build connectivity without IT involvement

• Personal-based DQ workflows: Centralize all DQ requests within Collibra Data Intelligence Cloud for faster remediation

• Data discovery with auto-validation rules: Sensitive data discovery and data quality enforcement

Below you’ll find answers to questions that we were not able to respond to during the webinar:

Q: Since the data is not stored in a table, how can you query? Is there any documentation?
A: We support file based DQ Checks: https://dq-docs.collibra.com/apis-1/owlcheck-spark/job-files#owlchecks-with-file
Including remote files: AWS S3, HDFS, GCS, Azure DLS, Azure BLOB, and NFS.

Q: When is Snowflake support available in production?
A: Snowflake Pushdown is currently in private beta testing. After final internal Collibra DQ testing and approval a GA date will be announced. The DQ Snowflake Connector is already GA.

Q: Will Snowflake pushdown be available to existing on-prem implementations? Or only available if we move to SaaS?
A: The DQ Snowflake Pushdown capability is installation agnostic. The functionality can be delivered on either.

Q: In SSIS, do your products offer connectors for backend data deduping, ex. Kingswaysoft for nocode?
A: Assuming SSIS is SQL Server Integration Services, then no, we don’t integrate with Kingswaysoft. We can perform Dupe checks on SQL Server directly using the JDBC Connector. If you share a specific use case there may be a programmatic integration we can help you to achieve.

Q: What is the roadmap during the coming months, especially for the user experience, with improvements of the interface?
A: We are swapping the underlying front-end architecture from Jquery for React. In this process the Explorer and Rules pages will get a redesign, while others will get minor enhancements. Overall the UI will have more consistent styling, tooltips, and UX as a by-product of converting the components used across the application. This is ongoing work, targeted to be completed late Q4/early Q1 2023. You’ll start to see pages and sections converted throughout the remainder of the year.

Q: Can you elaborate on authentication token timeouts on API connections with PowerBI, for example. We have experienced both 15 min time outs and have recently been instructed on how to extend this to 60 minutes? Not sure this is sustainable.
A: The Collibra DQ REST API token timeout is based on the DQ Web UI timeout.
This can be edited here: <INSTALL_PATH>/owl/config/owl-env.sh

Edit the value of the variable TIMEOUT which is the value in seconds before the timeout kicks in (eg. 3600 is one hour).

Then you will have to restart the Collibra DQ webapp: /<install_path>/bin/owlmanage.sh start=owlweb

Q: Any DQ connector with Salesforce in the roadmap?
A: We have a Generic JDBC Connector implementation. We could attempt a PoC using the Salesforce native JDBC driver: https://github.com/forcedotcom/Salesforce-CDP-jdbc/releases
This has not yet been tested. No Salesforce Connector is on the road map currently.

Q: When do you expect integration of Collibra DQ with Collibra (DIC)? Will it be as one tool?
A: Yes, it will be one tool, eventually. This is the “Holy Grail” of our integration road map, however knowing that the DQ software was acquired by Collibra, there are many integration points and hurdles to overcome. There is not a date set for the final integration to be complete, however it remains one of the top priorities at Collibra to finalize this integration work. We prioritize Lineage, Governance and Catalog integration in 2023 and beyond.