G

Thursday, July 14th, 2022 7:55 AM

Collibra Data Quality Integration - Upsert issues

When synchronising the Metadata Quality Extraction - the integration overwrites Data Quality Rule attributes - which is not ideal, given that users are expected to curate/manage these assets in the DQ Rulebook.

In addition, each time the synchronisation occurs this resets the DQ Dimension back to the default dimension specified in the DQ Connector - also problematic as you can only specify one Dimension and this is mandatory!

Please fix this! I can only see the alternative is to use different / custom attributes so these values are not lost…?

For example:

  1. DQ Rule is harvested
  2. User changes Status to Accepted
  3. User provides a description of the DQ Rule in DGC
  4. User provides the correct DQ Dimension for the Rule
  5. Synchronisation occurs
  6. Status Changed back to Candidate
  7. Description changed back to blank
  8. DQ Dimension changed back to default Dimension (e.g. Completeness)

35 Messages

2 years ago

Hello @graham.pearman,

Its an existing functionality as we dont allow changes to metadata coming from source system. We already have an item in backlog to bring description defined in CDQ while creating the rules into collibra

I will suggest to use Business Rules asset type here to define all business definitions and other characteristics and link to the data quality rules .
So business will define the business rules and this will be taken by DQ Developer to implement in CDQ and then once integration is complete they can link it to Business Rule and this can change the status of business rule to accepted. You can implement this process in our Collibra Workflows

39 Messages

Hey Bineesh

Thank you for this. I will feed it back to Prudential.

Re: DQ Dimensions. Is there also anything in the roadmap to not relate to a singular DQ Dimension (e.g. Completeness) in the DQ Connector - and take instead the value specified in Collibra DQ?

35 Messages

@graham.pearman You can use other dimensions as well. You need to link the rule to other dimensions which is available in details section of creating DQ Rule in CDQ . Once you use them in CDQ. This will automatically come in DGC as relationship to dimensions

Loading...