Custom Data Lineage name attribute

Hello Data Citizens,

I got the following question: when creating a custom lineage json file, which name is needed for the “name” attribute, the display name or the full name? Unfortunately the docs don’t mention it or I didn’t see it.

Sincerely,
Stefan

stefan_busch

Posted 3 years ago · Edited 2 years ago·Last reply 3 years ago

3 comments

alvinuseree

·3 years ago · Edited

Hey both, I’m quite sure it’s the displayName (but mapping against the path)- I’ve attached what I think is the missing piece of the puzzle.

https://productresources.collibra.com/docs/collibra/latest/Content/CollibraDataLineage/CustomTechnicalLineage/ref_custom-lineage-json-file.htm

You can see name and catalog_fullname referenced which probably means that name == displayName.

This example also seems to heavily imply displayName.

You could always give it a test yourself though with fullName and displayName and see what gets stitched to the catalog.

(I wonder why catalog_fullname and not just fullName) …

When you think about it logically that does make sense as the difference between displayName and fullname for most things physical is that the fullname references the path/tree for that asset (i.e. for a column - Schema > Table > Column). In the tech lineage file, you have to define the path/tree; if you were to use the fullname, in a way that would make defining the path/tree redundant.

Also, the nodes in the diagram just look better with displayNames!

stefan_busch

OP3 years ago · Edited

Thank you Alvin for your valuable input!

tomkuppens1

·3 years ago · Edited

Hi Stefan,
excellent question that I never explored, because my setup works, and now I’m wondering why it’s working …

In my custom lineage definition experiments I used the exact name the schema has in the database. I also set up sql_directory_source harvesting where you need to specify the schema name in the config file.
The stitching in both cases with the data dictionary works great, but the thing is - I see now - our display name and full name is a concatenation of the instanceName and schemaName and I use only schemaName in my references.

I have no clue which method is used to stitch the techlineage to the data dict. Sounds like I need to do some extra experiments.

Best,
Tom