DQO Classic - Fastlink to Break Records
Summary: Grant Rollerson describes a recurring issue with a Data Quality (DQ) job that contains a custom rule. This rule consistently flags certain records as issues, even though they are known false positives, leading to daily notifications that waste the DQ analyst's time. Grant Rollerson seeks a more efficient solution compared to the current workaround of manually excluding these records, suggesting improvements such as rule learning capabilities or enhanced email notifications. A context from Ideation suggests there may be future improvements, though no specific changes are confirmed yet.
Does anyone have the same issue
a DQ Job
with a Custom Rule
which breaks every day, because of the same record(s)
... which is a known false positive
the DQ Analyst daily receives an email generated by an Alert, goes into DQO dataset (DQ Job) and confirm the break is the one they see daily and not a real break
... so there's no real issue but it's a pain in the neck and a waste of his/her time
So
Get the rule changed to explicitly exclude that record from the test, which is pretty hacky
OR is there any other way to solve this issue in a more programmatic way?
either have the Custom Rule "learn" to accept that record
or include the actual Break Records in the email, or post a fast link to same
Nothing definitive in Ideation but I see;
Possibility to Add Customizable Columns for DQ | Ideation platform
Brad Fuchs
31 Mar, 2025
We are working to make the break record experience more cohesive. Please keep an eye on upcoming releases as changes will start to roll out.
Ashish Sharma
·1 year agoHi Grant - As you stated, while Custom Rule are fully driven from user requirements perspective, training/invalidating them similar to AR or Patterns do not apply to Custom Rule. I have seen some users have built a DQ Error-DataWarehouse/Database which essentially logs the only new errors. In this case, for a given job and rule name combo, if a linkid is already present, it gets ignored/updated whereas net new entry are Inserted and used to notify users.