1 Message
Importing custom metadata columns
We would like to extend our Collibra catalog with custom asset attributes, such as a secondary description column. We would like to ingest the additional metadata attribute from Redshift, by using tags inside the comment field to delimit the custom attributes. Is this possible, and if so, can someone point me in the direction of what I’d need to learn to do this in Collibra?
noor
262 Messages
3 years ago
If there are descriptions/comments for tables & columns in Redshift, and if you are using the Collibra marketplace Redshift JDBC driver, you may see those descriptions/comments appearing under “Description from source system” attribute. The other “Description” attribute stays as-,is.
But, if you want to extract more metadata from Redshift through the driver-connector than what is “typically” possible - like table names, column names, data types, descriptions - it is not possible. In my current project, there is valuable metadata in the definitions of Databricks tables. But, I cannot extract that due to limitations in driver-connector.
I think Collibra driver-connector can improve here - in the connection form, there can be options to extract desired metadata (of course, with some limit) and map that against attributes, relationships, assets…
This form should also facilitate writing SQL queries so metadata can be selectively extracted. Metadata at times can be inside a table as content…
0
0
noor
262 Messages
3 years ago
By the way, you can do this via Import APIS or Core APIs…
0
0