Critical Data
Hi i’m looking for any advice on identifying Critical Data Elements. Has anyone developed any sort of tool to apply some sort of score to data elements to determine if they are critical or not?
Some of the characteristics that we feel critical data possess are the following:
Is it used in KPI’s
Does the data connect multiple systems?
Is the data imperative to an operational function?
davidcorum
Posted 4 years ago · Edited 1 year ago·Last reply 3 years ago
6 comments
davidcorum
OP3 years ago · EditedHello everyone the feedback that has been provided has been great. You have let me know I am on the correct path. @mudassar.abdul.teck.com I have begun to collect reports that various sectors of the business identify as critical, and have begun to identify items that they may deem as "critical’
@simla.sivanandan.collibra.com I love the approach of classifying the critical data elements as “enterprise”, or “business”. Early in my work, and through conversation I have seen the need for two. What I am trying to do is develop an enterprise governance infrastructure that can cascade down to governance for the individual business units.
With all of the feedback that has been provided I am in the process of developing a scoring matrix that can help guide and facilitate the conversation around CDE. One of the challenges that I am having is that individual business units are stating they have over 500 CDE’s. So what I’m trying to do is build a matrix that scores, and weighs a data element across an assortment of variables. I will happily share when I make the matrix operational.
The biggest challenge right now is establishing an inventory for reports. We are using Collibra but are in the very early stages. So i’m literally using Excel and collecting reports across a multitude of BI tools (Tableau, MicroStrategy etc.)
davidcorum
OP3 years ago · EditedHello everyone the feedback that has been provided has been great. You have let me know I am on the correct path. @mudassar.abdul.teck.com I have begun to collect reports that various sectors of the business identify as critical, and have begun to identify items that they may deem as "critical’
@simla.sivanandan.collibra.com I love the approach of classifying the critical data elements as “enterprise”, or “business”. Early in my work, and through conversation I have seen the need for two. What I am trying to do is develop an enterprise governance infrastructure that can cascade down to governance for the individual business units.
With all of the feedback that has been provided I am in the process of developing a scoring matrix that can help guide and facilitate the conversation around CDE. One of the challenges that I am having is that individual business units are stating they have over 500 CDE’s. So what I’m trying to do is build a matrix that scores, and weighs a data element across an assortment of variables. I will happily share when I make the matrix operational.
The biggest challenge right now is establishing an inventory for reports. We are using Collibra but are in the very early stages. So i’m literally using Excel and collecting reports across a multitude of BI tools (Tableau, MicroStrategy etc.)
jaymilitscher
·3 years ago · EditedThat’s really great to hear @david.corum, and it’s exciting to see the community rallying around this question, thanks all!
a couple additional thoughts:
re: your BI report inventory - for your scoring variables you might want to consider the usage data from those BI tools (tableau, etc) to see which reports are most actively accessed across which audience groups.
and another low-tech way of scoring CDE’s: what KPIs get presented at a company level (e.g. All Hands meetings) every single month or quarter? Those are great candidates to start with in your project.
Looking fwd to seeing this evolve!
ruslanahmetzyanov
·3 years ago · EditedHi!
Just to add, for identifiyng critical data elements (though we don’t call them so
) - we additionally extract information from source systems:
per specific period of time.
Simla Sivanandan
·3 years ago · EditedHello @david.corum ,
Those are great starting points and what @mudassar.abdul.teck.com shared is spot on too.
I would also recommend that you approach this in a tiered fashion, not all critical data elements are “equally” critical. One of the approaches that I have seen is Enterprise Critical Data Element and Business Critical Data Element (ECDE vs BCDE). For e.g. - Annual Revenue is critical at an enterprise level, where as employee PII data is critical for HR. This approach has helped in driving adoption amongst business units and highlighting how seemingly non-critical data point for a business unit becomes critical to another one down the line.
Another approach is to mark any data points that are shared externally (such as regulatory reports) as Tier 1 critical, any data that drives business decisions as Tier 2 critical and so on.
By cataloging the reports and data sources in Collibra, most of the information is readily accessible. By also capturing the business processes and their RACI and linking them to the applications and reports, Collibra DIC can provide an enriched insight into what is critical and why it is so.
mudassara
·4 years ago · EditedHi @david.corum
In general, Collibra has provided some best practices in the below link
6 best practices for a data governance strategy
Learn data governance best practices to kickstart your program. See how Collibra expands data intelligence to connect data governance to business value.
Est. reading time: 10 minutes
Refer to step 3 in the below link
3. Identify critical data elements within the data domains
After defining the data domains, the next data governance best practice is to identify the critical data elements. In the lineage diagram above, evidently, data domains touch 10s, 100s, and 1000s of systems and applications containing key reports, critical data elements, business processes and more. In the early stages of your data governance program, there is no need to boil the ocean by focusing on all the data artifacts at once. The data governance best practice truly is to identify only what’s critical to the business.
A recent example of this data governance best practice in the Collibra Community comes from a technology company. The company needed data governance to validate customer reports and related source systems. During the first stage of implementation, the company identified just the ten most important reports and documented information about the systems of origin. Later, it scaled the initiative, applying certification requirements and related source system information for all reports. Simply put, a report is not certified if the owners cannot show its traceability all the way down the system of origin.
There are some specialized tools for CDE as per Solidatus and I have not used them as they are not free
https://marketplace.collibra.com/listings/solidatus-collibra-integration/ .You can explore
Based on my experience, I would just go by some 5-10 reports and collect CDE from them. You can try your Powerbi\Tableau which is the Presentation layer as a reference to start with . When you drop Tableau or Powerbi in Collibra it creates auto data lineage which might be helpful for you to trace CDE at Source.
The goal is to decompose a KPI of a report into data elements( columns) at the source by first checking with the business that use this KPI
For knowledge on CDE , Collibra offers Collibra university content, you can watch them learn more.
The below image might be helpful for a business person to understand it