Incremental Data from Collibra Insights

We dump the whole lot every day, and load up to the staging area.

We then Upsert any changes to a presentation schema any deletes / updates / inserts.

To understand what's changed, we use a hash on the columns and compare that

I've put this reply in more detail on other posts but pattern 

1. extract insights to staging schema
2. apply SCD and insert into core schema (truncate staging)
3. transform core into reporting schema, incl. hot & cold facts
4. put tableau across reporting

Also look at Output Module