Disney wants to compare source to target results on tables as large as 130 columns and 120 million rows (streaming data). They need to validate that the data is consistent after moving it in their environment. It is a labor intensive job they are currently using Veridata to solve. They are moving away from Veridata as an organization, and are looking at alternatives. What do you tell customers that want to run jobs of this size? Is there any answer beyond suggesting that they build a K3s environ...

How to run large source to target jobs in DQ? | Data Quality & Observability

How to run large source to target jobs in DQ? | Data Quality & Observability | Collibra Community