9 Messages
•
550 Points
dbt incremental models with the Snowflake technical lineage connector
We have recently deployed the snowflake technical lineage capability via Edge. We're using a JDBC connection in SQL-API mode. We have a number of teams using dbt for transformation and we've noticed that technical lineage doesn't always work for incremental models. I'm not sure but this mainly seems to affect dbt versions > 1.1. I can't see any obvious differences in the SQL generated so I wondered if anyone else had come across this issue?
I'm aware of the specific dbt connector but this will be really onerous for us to deploy as we have quite a decentralised org structure so I'm hoping to get the snowflake lineage connector working in as many use cases as possible.
andrewsouthcombe
9 Messages
•
550 Points
2 months ago
In case anyone else has this issue I tried a couple of things which seemed to make an improvement:
{
"analyzeTemporaryTables": true
}
Snowflake SQL-API lineage missing due to frequent table replacement – Support Portal (collibra.com)
The other way to achieve this seems to be to use the SqlDirectory option and submit relevant sql statements directly.
0
0