Lineage from file to table
We have managed to run the Edge lineage capability to harvest column to column lineage within Snowflake. Our next step is go back to the source and try to model lineage between the original source file and the target Snowflake table. In general our source files are held in an ADLS2 data lake and then get loaded in via snowpipe.
We are planning to use the ADLS2 connector to harvest file metadata into the catalog. Once we have done that, is there a recommended way to show the technical lineage from file to table? For example, should we create custom lineage and submit to Edge? Or can we directly create relations between a file and the table it was loaded to?
Andrew Southcombe
Posted 2 years ago · Edited 1 year ago
0 comments
