Hi all, I’m struggling with the Data Quality rule set-up in Collibra. From what I understand, a Data Quality Rule has to be linked to a Data Quality Metric and a Data Asset (such as a column), by using the OOTB relationships below : [Data Quality Rule] Executed by / Executes [Data Quality Metric] [Data Quality Rule] Classified by / Classifies [Data Quality Dimension] Of course, when I do that and set up in the “Settings” page the Data Quality Rule to be applied on the column (as shown below), th...

Setting up a Data Quality Rule and linking it to a Data Asset | Data Quality & Observability

daverichard

·3 years ago · Edited

I seem to have run into the same issue where I can get the score, but I cannot get the Data Quality Dimensions to display. I have tried multiple setups including those mentioned above and implemented the PDF setup as well. No issues with getting a combined score for multiple rules, but I cannot get the dimensions to display with the custom relationship as described in the PDF or using the OOTB relationship.

Edited:

Figured it out, I was setting up the custom relationship on the Data Quality Rule and not the Data Quality Metric. Once applied to the metric the dimensions appeared.

christopherthullen1

·4 years ago · Edited

I would like to add that the use of the term “Data Quality Rule” to designate two completely different concepts is confusing. In the attached diagram, you can see where to configure the “Data Quality Rule” asset type, and where to configure this other “Data Quality Rule” thing. Is there a way a data quality tool can present concepts without using the same name for two different things?

ludovicjanssens

·4 years ago · Edited

Thanks for that pdf, Jennifer, great documentation! I did found out what I did wrong and could make it work a few days ago, but it took me a while… I used the wrong relations (at the client they changed the default one, hence I was in a tunnel vision). With this document it would’ve been a lot easier to understand; I’ll definitely store it safely somewhere

brigidhamm

·2 years ago

@ludovicjanssens Do you have any steps on using the correct relations to see the DQ rules?

ludovicjanssens

·4 years ago · Edited

Thanks for that pdf, Jennifer, great documentation! I did found out what I did wrong and could make it work a few days ago, but it took me a while… I used the wrong relations (at the client they changed the default one, hence I was in a tunnel vision). With this document it would’ve been a lot easier to understand; I’ll definitely store it safely somewhere

arthurburkhardt

·4 years ago · Edited

Yes, collibra has some “hidden magic”, which is not documented anywhere. The only way to find out about this magic is to look directly in the source code (java or javascript) to figure it out.

Priority values of issues
breadcrumb relations
page definitions
Reviewer, Stakeholder, Requester and Issue manager roles on issue types
the list goes on…

Alexandra Jorgenson

Admin4 years ago · Edited

I’ve moved this thread to the ‘Data Quality’ questions to make sure the DQ team sees your comments. @matthew.tsui.collibra.com , can you think of any documentation that might help here?

daverichard

·3 years ago · Edited

What an amazing resource! I was slowly working through the documentation, but this PDF laid out the process to implement this perfectly!

muhammadsalahuddin

·3 years ago · Edited

Thank you @jennifer_bauer_temple.baxter.com for sharing the document. It will save lot of time. Just another quick question. Did you get a chance to collate the future document containing additional information about Data Quality
Rules and Data Quality Metrics? if you can share it as well, it would be greatly appreciated. Thank you again and happy DQ

Pascal Hess

·1 year ago

@jennifertemple can you make this document available again?

Alexandra Jorgenson

Admin4 years ago · Edited

I’ve moved this thread to the ‘Data Quality’ questions to make sure the DQ team sees your comments. @matthew.tsui.collibra.com , can you think of any documentation that might help here?

ludovicjanssens

·4 years ago · Edited

It would be very interesting to get some more documentation and practical guidelines on this. I am really struggling to get it working although I followed all the above guidelines
Some examples and a step-by-step guide would be wonderful.

jennifertemple

·4 years ago · Edited

You need to create a custom relation for this to work as stated above. I chose to create the relationship Rule grouped by/groups Data Quality Dimension. Attached is a document I created on the steps I took to get the data quality working for me.CollibraDataQualityConfiguration.pdf (830.9 KB) Hope it helps

alvinuseree

·4 years ago · Edited

I think I see the confusion, my answer was dictated by the aggregation path you shared.

The categorization shown on the dashboard is dictated by the Asset Type at the END of the aggregation path and the relation defined in the categorization field. Therefore, in your case metric > is classified by data quality dimension.

In the DQ world there are 3 concepts to consider:

Metric: These are DQ Scores and are held as attributes on a given asset.
Aggregation Path: This is the breadcrumb of relations to work back from and dictates the Asset Type that holds the Metrics.
Categorization: These are logical groupings for the Asset Type at the end of the aggregation path (generally Dimension)

What this means is that you can define the relations you need to configure to generate the dashboard of your choice.

Take the following example:

In this case:

The Aggregation path goes from Asset (Looks like a Column) > Data Entity > Data Quality Rule
As Data Quality Rule is the Asset at the end of the aggregation path, to group by DQ dimension, a relation between the rule and dimension is needed (defined in the categorization field)
Data Quality Rule holds the relevant metrics

maximiliencote

OP4 years ago · Edited

Hi Arthur,

That was my understanding as well : as you mention, the Data Quality Dimension should be linked to the Data Quality Rule, and not to the Data Quality Metric. However, the breadcumb does not work on the Data Quality Dashboards with this configuration.

I’ve raised the issue to Collibra support more than a month ago, but the ticket is still not resolved.

maximiliencote

OP4 years ago · Edited

Hi Arthur,

That was my understanding as well : as you mention, the Data Quality Dimension should be linked to the Data Quality Rule, and not to the Data Quality Metric. However, the breadcumb does not work on the Data Quality Dashboards with this configuration.

I’ve raised the issue to Collibra support more than a month ago, but the ticket is still not resolved.

arthurburkhardt

·4 years ago · Edited

Which breadcrumb on which data quality dashboards? I’m not sure I’m following.
Are you talking about the “data quality rule” definitions in the assignments? If this is the case, those are meant to be custom. Some examples: About data quality rules (collibra.com)

maximiliencote

OP4 years ago · Edited

Hi Alvin,

Thanks for your quick reply, you got me unstuck from this bog

It works this way, by linking the Data Quality Dimension to the Data Quality Metrics. The strange thing is, this relationship doesn’t seem to be OOTB and needs to be created manually.

I think Collibra could get a bit better on documenting this part which I personally find hard to master! It means that the Quality Dimension can be linked to the Data Quality Rule and has to be linked to the Data Quality metric, which I find a bit awkward

maximiliencote

OP4 years ago · Edited

Hi Alvin,

Thanks for your quick reply, you got me unstuck from this bog

It works this way, by linking the Data Quality Dimension to the Data Quality Metrics. The strange thing is, this relationship doesn’t seem to be OOTB and needs to be created manually.

I think Collibra could get a bit better on documenting this part which I personally find hard to master! It means that the Quality Dimension can be linked to the Data Quality Rule and has to be linked to the Data Quality metric, which I find a bit awkward

arthurburkhardt

·4 years ago · Edited

I’ll provide contradictory information, so between the three of us, it definitely look like it might be good to have more prescriptive docs.

My understanding is the following diagram

AFAIK, the data quality rule defines the logical rule to be applied, but the data quality metric defines what is actually, technically implemented. But I don’t think there is a link between data quality metric and columns OOTB.

In any ways, yes, it does not seem like the data quality model is fully usable in its default configuration.

alvinuseree

·4 years ago · Edited

Hey Max,

Long time, hope you’re well!

I’ve attached an example diagram highlighting the asset types that need to be related. You’ve got the rule to the dimension and not the metric.