General Collibra DQ questions

Question

Do you have the ability to specify the time of batched notifications (i.e., daily at 8am)? If so, can you direct me to any material on how? Can you customize or introduce entirely new DQ dimensions (on Collibra DQ)? If you mask certain columns, is the underlying data residing on the postgres database also masked?

ahmadalqinneh · Answer

Thanks @laurent.weichberger.collibra.com - few follow-ups. I was aware that you can schedule your jobs (owlchecks) but the specific question from the prospect is can you schedule your notifications. I.e., jobs run throughout the course of the day but notifications always go out at 8AM. Similarly I’m aware that you can write custom rules, but can you assign those rules to custom dimensions? The 6 dimensions that collibra dq ships with are: Completeness, Consistency, Uniqueness, Validity, Accuracy, Timeliness. Lets say they want to add a 7th dimension “Integrity”, would they be able to do that? Thank you that link is very helpful!

BigDataBear · Answer

Hi Ahmad: https://dq-docs.collibra.com/scheduler/schedule-owlchecks You can write custom rules with SQL code, or Apache Spark SQL code. There is a difference between masking and encrption. Masking doesn’t encrypt the data. To encrypt we set the OWL_ENC=JAVA flag. See also: https://datacitizens.collibra.com/forum/t/reviewed-final-how-do-i-encrypt-sensitive-values-in-the-dq-metastore-tables-data-preview-and-rule-breaks/1261

1	Grant Rollerson SUNCORP CORPORATE SERVICES PTY LTD
2	Tom Friesen Algonquin College
3	SB S Benson Plus3 IT
4	S stefan_busch
5	Brian Lloyd USA TODAY Co.
6	D daverichard