11 Messages
•
200 Points
Please Vote: Ideation to fix a technical lineage issue
Fellow Data Citizens, we are looking for help in moving forward an ideation. We have noticed an issue with technical lineage where relations are being deleted by the lineage harvesting process when the relationship type is the same as used by the lineage harvester.
The Issue: When ingesting lineage via the lineage harvester, any relation created via the UI, import file or via API is deleted by the lineage harvester if it is the same relationship type as one harvested by the lineage harvester, By example, your team imports source/target relations to document an ETL process from a lineage harvested database table to some target. The next time that the lineage harvester runs, that imported ETL lineage is deleted. This means that your team needs to replace that lineage whenever the harvester is run which is an onerous and possibly costly process. The product team offered a workaround by which the lineage is loaded via custom technical lineage. This is complex given the lack of an API as well as inflexible for those that may want to use the UI or upload file capability.
The Solution: Our team has proposed a solution to an ideation (https://productresources.collibra.com/ideation-platform/?id=DCC-I-196) to solve the issue. Our proposed solution would be an enhancement to limit the modification or deletion of relationships by technical lineage to the relations created by technical lineage.
There is a similar ideation out there that was recently posted as well: https://ideas.collibra.com/ideas/CDL-I-8 Given the existence of another ideation and comments on our ideation, we would anticipate that many customers are facing the same issue.
Please help us move this issue up the priority list for the Edge team by voting for both ideations. In that way, we can show the impact of this issue and need for its resolution.
Thank you!
Dave Wright
No Responses!