1 Message
Metadata ingestion from DBT to Collibra
Has anyone successfully ingested metadata from DBT to Collibra? We’re encountering difficulties and would greatly appreciate any assistance.
We’re attempting to ingest metadata using the custom Spring Boot application available on the Collibra Marketplace (https://marketplace.collibra.com/listings/dbt-lineage-to-collibra-integration/). However, although the API response indicates success (200), no metadata is being ingested.
API: http://<ip_of_the_server>:8080/providedProjects/
Steps taken:
- Downloaded the Collibra Integration Library from https://marketplace.collibra.com/listings/spring-boot-integration-library/
- Installed the Integration Library via Maven from the terminal.
- Configured the application.properties file in the src/main/resources folder.
- Executed the application and verified using the Swagger URL.
- Made an API call from Postman for Provided Projects, receiving a response of 200.
P.S. - We are running the application on an EC2 instance.
Thank you in advance for your assistance!
mehjabeensultana
15 Messages
•
150 Points
11 months ago
Edge provides technical lineage harvester with dbt Cloud and dbt Core. Have you explored those options?
0
0
brianlloyd
34 Messages
•
2K Points
7 months ago
Hello - we implemented the dbt Core Collibra connector/capability and it works for us. We did have to set up a Google Storage Bucket for the development team to copy the /target files into, then we wrote a Linux bash shell script to copy the files from the Google Bucket to a local file on the Linux Edge server using gsutil, then run the edgecli program to copy the files into the Collibra Storage Container for the Lineage Harvesting to work.
It sounds like a pain in the tucas but it is pretty straight forward and runs in minutes from start to finish.
0
0