Ask HN: How to avoid sensitive data being part of LLM training data?

Posted by deepakthakur on Monday January 01, 2024 @02:39AM
from the ask-hn dept.

How are you making sure your sensitive data & PII doesn't become part of LLM training data. When training data is small, manual verification is possible. But when data size is huge, is there a way to mask/filter out PII/sensitive data.

( 2 comments | on HN )

Re:Ask HN: How to avoid sensitive data being part of LLM tra... by toomuchtodo on Monday January 01, 2024 @10:52PM (#38837894)

At the orgs I contribute at, we use data security posture management tools (DSPM) to filter sensitive data from model training ingest. Mostly regex string equivalents, but also heuristics and context aware tagging of data.

[ Parent | on HN ]

Re:Ask HN: How to avoid sensitive data being part of LLM tra... by incogitor on Tuesday January 02, 2024 @11:28AM (#38843420)

There are a number of PII filters and libraries available on GitHub.

[ Parent | on HN ]