DQ Check for same value in 2 different Datasets.

I have 2 datasets that I want to make sure are in synch, meaning the same value is in each dataset/column. Dataset1 has a column (Integer type) that has 10 digits, and Dataset2 has the same column but its length is 12-13 digits. The difference in length is because in Dataset2, there are trailing zeros.

Here is an example of what I am referring to:

In DS1/col1 the value is 1234567899
In DS2/col1 the value is 123456789900

Regex or SQL?

Sample code would be welcome.

stevenrizzardi

Posted 5 years ago · Edited 1 year ago·Last reply 5 years ago

1 comment

leonkim

·5 years ago · Edited

So it looks like DS2/col1 is a double (or some sort of data type that has precision that is causing trailing zeros). Source activity with Validate Values option enabled (-vs -validatevalues -vskey joinKeyBetweenDS1AndDS2) should detect that column schemas are different. The individual cell value mismatch should also be detectable.