Re:Show HN: DataProfiler – What's in your data? Extract sche...
by BugsJustFindMe on Monday May 10, 2021 @11:30AM
(#27107120)
They use a machine learning model to determine categories that have concrete membership rules?
[ Parent
| on HN ]
Re:Show HN: DataProfiler – What's in your data? Extract sche...
by ZeroCool2u on Monday May 10, 2021 @11:50AM
(#27107380)
"Note: The Data Profiler comes with a pre-trained deep learning model, used to efficiently identify sensitive data (PII / NPI). If desired, it's easy to add new entities to the existing pre-trained model or insert an entire new pipeline for entity recognition."
No, they seem to use the ML model for identifying data that typically may be considered private, sensitive, or PII, which can be very difficult to define depending on your data and your organization. For example, identifying phone numbers, social security numbers, addresses, etc. All of those things can take a variety of formats and writing a regex or something to identify them feels a bit silly when you can easily train a decent classification model to do this mostly automatically and then catch this kind of data in new use cases where the recognition pattern you may have written before doesn't necessarily work.