It's a great question. Redaction limits the LLMs ability to draw on the underlying training data on the subject. This can work to the developers benefit in many cases, like asking questions about your own provided context.

Many developers have gotten away from relying on LLMs for facts, toward providing LLMs with facts and having those facts repurposed.

For example, if you ask an LLM about a famous person, like Wayne Gretzky, it may give you a good answer but there is a chance it may hallucinate key details like the number of points he had in his NHL career.

To combat this, you can provide the LLM with a biography of Wayne Gretzky and you may get more factual answers, but the LLM may still hallucinate if you probe for facts that were not provided.

If you redact his name instead, for example asking “Who is [Name1]?” the LLM will be unable to answer the question without further context. But now, if you provide the redacted biography the LLM can answer the question while relying only on the provided context (the biography will contain information about [Name1]). If the question falls outside of the context the LLM will not be unable to answer, which is often the desired result. In other words, the LLM is unable to rely on the training data about Wayne Gretzky because it is only dealing with [Name1] along with redacted locations, organizations, occupations, etc from the biography about [Name1]. You force the model to rely on the provided facts.

The use cases we see are people providing legal contracts and financial statements where names and currencies get redacted, and the LLM must work with the redacted values and any other context provided.

that's actually pretty brilliant. I can imagine this also being useful for adding a chatbot for a website's content and really trying to limit the responses to only the content from the website as much as possible.

Damn, that is actually a really cool approach.

I suppose most LLMs are not smart enough to make the connection and can be probably told to avoid doing it, but I would imagine that it's not impossible for it to figure out that Name1 is likely Wayne Gretzky from context?

Edit: Yep, it's definitely a problem unless the facts are also anonymized I guess: https://chat.openai.com/share/84dbe124-dca7-46e3-be73-79b194...

I redacted the full wikipedia paragraph with the API. Like, the nickname "The Great One" is a pretty major tell!

[NAME_GIVEN_1] [NAME_FAMILY_1] CC ([NAME_GIVEN_2] [NAME_FAMILY_2]; born [DOB_1]) is a [ORIGIN_1] [OCCUPATION_1] and [OCCUPATION_2]. He played 20 seasons in the [ORGANIZATION_1] ([ORGANIZATION_2]) for four teams from [DATE_INTERVAL_1] to [DATE_INTERVAL_2]. Nicknamed \"the Great One\",[1] he has been called the greatest [OCCUPATION_3] ever by many [OCCUPATION_4], [OCCUPATION_5], The Hockey News, and by the [ORGANIZATION_2] itself,[2] based on extensive surveys of [OCCUPATION_6], [OCCUPATION_7], [OCCUPATION_8] and [OCCUPATION_9].[3] [NAME_FAMILY_1] is the leading goal scorer, assist producer and point scorer in [ORGANIZATION_2] history,[4] and has more career assists than any other [OCCUPATION_10] has total points. He is the only [ORGANIZATION_2] [OCCUPATION_10] to total over 200 points in one season, a feat he accomplished four times. In addition, [NAME_FAMILY_1] tallied over 100 points in 15 professional seasons, 13 of them consecutive. At the time of his retirement in [DATE_INTERVAL_2], he held 61 [ORGANIZATION_2] records: 40 regular season records, 15 playoff records, and 6 All-Star records.[2]

> Based on the information provided, NAME_GIVEN_1 NAME_FAMILY_1, also known as NAME_GIVEN_2 NAME_FAMILY_2, played in the ORGANIZATION_1, which is also referred to as ORGANIZATION_2. He played for four teams within this organization over the course of 20 seasons, from DATE_INTERVAL_1 to DATE_INTERVAL_2.

Hey that's actually pretty good.

You can use CapeChat UI to mess around with it: https://chat.capeprivacy.com/

Or you can also create a free API key here: https://app.capeprivacy.com/api-keys to use the interactive API directly: https://api.capeprivacy.com/v1/docs#/Privacy/DeidentifyText

Click "Authorize" on the top right to add the key, and then click "Try it out" on any of the endpoints.

Exactly. A super famous person like Wayne Gretzky is really hard to protect.

For fun, you can try to tease ChatGPT with information like. "Who is [Name1]?", it won't know, but then add "[Name1] is considered the greatest [Occupation1] in the history of the [Organization1]". Greatest is now a big clue. Add "[Name1] has the most points in history". Points is a big clue, it's some kind of game or sport.. etc. It will eventually figure it out, but I've seen it guess wrong with like Michael Jordan instead.

> De-identification

> Re-identification

Wouldn't these two features address your concern? ChatGPT gets a generated unique ID that is still a consistent value for each card, just not the number itself. Then when the results are returned, that generated ID is turned back into the real card number.

This only becomes a problem when the de-identified data itself is needed to answer a question, like tell me how many Visa cards were used in these transactions by checking the card numbers.

That's right. So in the case of credit card numbers we redact it like [CREDIT_CARD_NUMBER_1], [CREDIT_CARD_NUMBER_2], etc so the LLM can still answer prompts like "how many", but it can't answer prompts like "sort". But you can use OpenAI function calling API to do the sort, where your function re-identifies, sorts, and then de-identifies again.