Storing anonymized profiling results
I was to decide on whether to set the “Anonymize Data” property to true or false in the DGC Service Configuration settings. I noticed the below statement from the documentation:
“Collibra detects the data type of a column during profiling and only anonymizes the data if the data type attribute is Text or Geo.”
If set to true, the functionality is really going to help? How can all text data type columns be assumed as sensitive data, and hashed/masked therefore by the profiling module?
Or, am I missing anything that can guide me to leverage this functionality…
Br,
Noor.
noor
Posted 5 years ago · Edited 1 year ago·Last reply 5 years ago
5 comments
Former User
·5 years ago · EditedFew questions here to unpack.
With regards to your first one, the
Anonymize Dataconfiguration is a blanket configuration for all your ingestions and yes, will anonymise the everything with the type Text/ GEO.In terms of your second question, if you have concerns with Min, Max values showing real data and are questioning the value of sampling and profiling the data, there’s nothing stopping you from just doing metadata harvesting. I would suggest running a few tests to confirm that real data that could be harmful is actually showing. Having first run a few tests on my side, I was happy with what was showing but this was for my use case - you may feel otherwise.
Sampling and Profiling of the Data is not required to harvest and stitch lineage either. Every use case is different, and there is good value you can get from just metadata and lineage harvesting.
Not entirely sure what you mean by your last question, mind elaborating?
noor
OP5 years ago · Editedprofiling and sample data are pre-cursor for automatic data classification seems. But then read the statement -
“Data classification must be manually triggered from every table, schema or database.”
Looks like lot of manual work here is what I meant…
christopherthullen1
·5 years ago · EditedI agree that “automatic” data classification is not very automatic when it has to be triggered at the table level. I understand that one of the new versions of Collibra (cloud) will allow triggering at the schema (or higher?) level. I am looking forward to it.
noor
OP5 years ago · Editedpersisting the profiling results and sample data is mandatory for automatic data classification. However, the below statement from the documentation doesn’t sound that promising to leverage all these features combined - profiling, sample data, auto classification.
'Automatic vs. manual classification :
Data classification must be manually triggered from every table, schema or database."
noor
OP5 years ago · Editedand the suggestion from Steven Wood (from Collibra) in the below article deals only with hiding sample data. Still the profiling statistics like MIN, MAX etc. could have actual data displayed.
https://datacitizens.collibra.com/forum/t/hiding-data-profiling-example-data-for-asset-type-table/296
I am wondering if the sample data & profiling results store are really required. which personas among the data citizens could be really interested in looking at the sample data & profiling information?
Br,
Noor.