L

Tuesday, September 21st, 2021 6:12 PM

DQ Audit Trail for Rule Changes

A prospective client asked a fine question regarding “auditing the change of the custom rules.”
After some research and testing this, indeed we do have auditing of rule changes at this level of granularity:

Audit columns:

  1. dataset = Any rule is associated with a Dataset. So this is the name of the dataset with which the rule is associated.
  2. ruleNm = This is the name of the rule itself. All rules must have a unique name.
  3. ruleValue = This is the expression associated with the rule. If there is any modification the current ruleValue is shown (see the example below which I created).
  4. modType = insert | update | delete
  5. updtTs = Time and date of this change to the rule (eg. timestamp).
  6. isActive = enbabled | disabled (eg. active == enabled)

You can deactivate a rule using the “Status” of the rule on the far right:
28%20PM

  1. userName = Which user made the modification to this rule.

In the above example, what I have done is created a rule to check our own DQ Metastore Agent table, for any case where the master_default column has the value ‘yarn’ which worked fine.

Example is from the DQ Metastore Postgres “agent” table:

Then I came back and changed it to master_default = ‘yarm’ and that also worked fine, but it recorded the audit trail of these changes. I am the admin user on this instance. You can see I inserted the rule, and then I updated the rule shortly after.

The customer read my write up and responded:
"I liked your excellent example and explanation except I think the correct change is:

master_default = ‘yawn’

For my afternoon nap.

5 Messages

3 years ago

I have been researching this as well and found that the Rule History (rule_history) table also has the information.

It also has an API under Controller Rule in Swagger and it is called Get Rule History Audit. It does require the dataset name and the rule name if you want to get specific information.

Loading...