Fivetran to Collibra Integration (Python)

Hi Kristen, yes. We can help with that. @ralph.rimorin.collibra.com has built the integration. He should be able to help. Let me reach out and get back to you.

Hi Ralph,

First of all thank you for your message. I am quite new to Collibra so would need some further clarifications.

What does it mean to “catalog first the target or destination database” and then link it to a System asset? And with Edge, do you mean the web browser?

From what I see, we have created mock assets like you described: System -> Database -> Schema -> Table -> Column. What when I run the code I get the following error:

Traceback (most recent call last):
  File "/Users/anda.zhang/Downloads/pyi-template-fivetran-2.0.0/fivetran_collibra_ingestion_job.py", line 44, in <module>
    fivetran_to_collibra_facade.extract_fivetran_assets_then_import_to_collibra_by_destination()
  File "/Users/anda.zhang/Downloads/pyi-template-fivetran-2.0.0/tasks/fivetran_collibra_facade.py", line 565, in extract_fivetran_assets_then_import_to_collibra_by_destination
    self.build_destination_asset_import_command(importRequest, destination)
  File "/Users/anda.zhang/Downloads/pyi-template-fivetran-2.0.0/tasks/fivetran_collibra_facade.py", line 266, in build_destination_asset_import_command
    destination_system_assets = self.query_system_assets(system_uuid=self.destination_system_uuid, system_location=destination['config']['host'], db_name=destination['config']['database'], schema_name='FOR_QUERY_SCHEMA_EXPECT_TO_RETURN_EMPTY')
KeyError: 'host'

It seems like, it can not find the sys_uuid.

Hi Anda,

About the Mock assets, you cannot just mock the data unless you are familiar with the asset identifiers and relations ingested with the Catalog feature of Collibra. All of these follows some naming convention of the identifiers for example for the Column, it will have a full name format something like (SystemName > SchemaName > TableName > ColumnName). Please reach out to the data team (or Data Governance team within your company) that is working with Collibra or the CSM (Customer Success Manager) assigned to your EQT Partners to get some help about Edge or Collibra Catalog.

I am seeing this error, looks like something related to the parsing of data being returned by the Fivetran API Destination endpoint. I will run a test on Monday and see if there are some incompatibility issue between the recent Fivetran API and this code.

KeyError: ‘host’

Thanks!

Hi Anda,

I believe it should still work… Would you be open to a call to review your configurations and mock data?

Ralph

Hi Ralph,

Yes, that would be great. Are you able to join on a call 9:00 - 10:00 tomorrow CET?

/Anda

9am EST works perfectly. Let me invite you

/Anda