Integrations best practices
Updated
Applies to: integrating with Collibra.
Recommendation
Collibra offers a wide range of ways to connect to external data sources. Follow the recommendations and key component overview in this article to optimize your Collibra Data Intelligence Platform metadata connector or integration approach.
Impact
Reduce time and resource costs for integration delivery, installation and support.
Leverage Collibra out-of-the-box features.
Background - integration types overview
Native/Direct integrations are available in Collibra as out-of-the-box features, such as S3 and Tableau.
Metadata connectors are mostly available in
Collibra Marketplace
or on the technology vendor website:
JDBC connectors certified by Collibra.
Vendor supported JDBC connectors non-certified by Collibra.
Spring Boot applications installed as a standalone integration app from Collibra Marketplace.
Data Lineage
is available as a standalone application:
Technical Lineage Harvester as a standalone JAVA application. We support three types:
Data Source as JDBC.
Data Source as ETL.
BI Harvesters such as PowerBI, Tableau and MicroStrategy.
Custom integration usually developed on any ESB (Enterprise Service Bus) or using object-oriented language, for example Java Spring Boot service or Python service:
Custom Integrations using Java or Python can fully utilize REST IMPORT APIs to implement non-supported integrations.
We can extend use cases and use Collibra workflows in these integrations which can be called by custom integrations using REST API for bidirectional integrations and workflow task assignments to specific roles inside Collibra.
Insights Data Access which provide APIs can be used to extract data from Collibra. This helps in giving an overview of Collibra Maturity Assessments.
Manual data import as Excel files and .csv files from Collibra UI.
Collibra Data Quality & Observability Connectors used to extract the DQ Rules from Collibra DQ into Collibra Data Intelligence Platform. This is currently only supported in Edge.
Best practice recommendations
Integration type selection and key decision factors
This section aims to help you chose the best integration method to minimize your resource time and cost, and leverage the out-of-the-box feature set for your unique use case.
We recommend that you evaluate the different integration options in the following order of priority. Stop at the first one that provides the functionality you require:
Priority | Integration approach |
1 | Native/Direct integration. |
2 | JDBC connectors certified by Collibra. |
3 | Vendor supported JDBC connectors non-certified by Collibra. |
4 | Spring Boot applications installed as a standalone integration app from the Collibra Marketplace. |
5 | Custom integration developed by any ESB or custom integration service. |
Integration selection key factors
Below is a table of integration type and decision-making factors you should consider when choosing a new integration method:
| Collibra Supported | Time to install and configure | Jobserver or Edge required | Upgrade process | License vendor | Collibra features enabled |
Collibra native integration | Yes | Low | S3 integration: Edge or Jobserver required. | Automatic, during DGC releases timeline. | Collibra |
|
Jobserver metadata connectors | Supporting party has to be checked on Collibra Marketplace. | Low | Jobserver | Manual update once new version released. | Metadata connector provider (might not be supported when Free of charge) | Profiling, Data Sampling, Classification |
Edge capability templates | Yes | Low | Edge | Automatic, during Collibra releases timeline. | Collibra | Profiling, Data Sampling , Classification |
Collibra Data Quality & Observability connector | Yes | Low | Edge | Automatic, during Edge release. | Collibra |
|
Spring Boot applications | Supporting party has to be checked on Collibra Marketplace. | Medium | Not needed | Manual | Metadata connector provider |
|
Data Lineage | Yes | Medium | Not needed | Manual | Collibra | Technical Lineage, Business Lineage |
Reporting Insights | Yes | Medium | Not needed | Manual | Collibra | Reporting insights |
Custom integration as any EBS or custom integration service | No | High | Not needed | Depending on ESB Platform or Custom Integration Technology | ESB Platform Vendor or Technology Vendor | Custom integration as any EBS or custom integration service |
Validation Criteria
One of the important validation criteria is “Time to Install and Configure”, where values are:
Low is < 1 day
Medium is 1 - 5 days (usually 1-2 days)
High is 5 - 20 or more days (usually 10+ days).
Please refer to the above section for details about time needed.
Additional Information
For more information see the following resources:
See the Collibra documentation Center for more information on any of the above elements.
Many integrations and connectors are available on the Collibra Marketplace.