“Sorry we let this happen, we’ll get it fixed on our next regular release, no rush”
Absolutely a bubble.
(This was not intended to be a constructive comment, just an observation)
Can I have a prorated discount then?
(I know the answer)
The only 100% safe solution is to not reveal the private info in the context. That's exactly what anthropic is doing here: removing the logged in user email from the system prompt.
Make no mistake though, that's a weak fix. The email is usually just a "git log" away, and for many tasks it finds its way into the context quickly.
Internet access must be blocked for agents dealing with private information. This makes agents vastly more cumbersome to develop or much less useful.
So the frontier labs pushed the idea of guard rails and harnesses as the solution. It sort of works, most of the time, and it lets users yolo all their secret information and get things done fast.
I had the distinct pleasure of savoring some of the exquisite UI and API experience that Cisco and friends cooked up for me in a misguided IoT adventure. Let me tell you, "deterministic" does not mean "not wrong" or "not stupid".
Here we get a half apology and a reason, but the passive voice, "we let this happen" is telling. The apology isn't for the bug, but the inability to manage their own system.
> I'm sorry we let this happen. Users had reported Claude would hallucinate names for things like @author tags, since it didn't have context on the account. So we added the account email in the model's context for signed-in users.
(I don't think the GP was generated. But LLMisms seems to creep in when you work with them.)
It's not even a weekend either?
??
The email can be removed from the context but not everything can.
I suggest everyone to add some hooks to strip this part automatically before claude exfiltrates all your NDA plans :)
> ## Querying This Documentation This site has an agentic ask interface you may use to query the documentation dynamically by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
``` GET https://docs.parloa.com/index.md?ask=<question>&goal=<user_goal> ```
Not a great look while you're simultaneously pushing the story that you and everyone else are shipping more lines of code than ever and "code is not the bottleneck".
Do you plan on publishing data on the rate of bugs, privacy issues and security issues introduced as you scale up AI-generated code vs human-written code?
Lets not lie here. Only a misinformed idiot cares about you or your company being sorry. It doesn't fix the problem. And this particular problem should never have happened.
If it wasn't bad enough that adtech literally deanonymizes the internet, you're doing that for them for free. You people are literally insane if you ever thought it was acceptable to dirty contexts in this way.
> Users had reported Claude would hallucinate names for things like @author tags
That is not your problem. Your job (Anthropic's) is to provide a vanilla shell of a system that is default SAFE. I reserve the right to risk putting my keys, my email, or other personally identifying information into the system.
Every one of these failures makes me glad that I've got a backup plan for inference once this entire bubble pops. And if the election was any sign, you guys are screwed, I frankly support what's going on in politics even though I like the technology.
Tech industry has been a bad neighbor for years.
Who added this? When? In what version? What's the CVE? Where's the security disclosure? Is this privacy risk encapsulated in your S-1 filing.
Simple RCAs seem to be beyond companies this days.
Who's "we"? Anthropic or the bots you allow to run free 24/7?
What countermeasures have you/your bots come up with to avoid disclosing user information going forward?
[1] https://www.anthropic.com/research/redesigning-source-contro...
I'm sorry we let this happen. Users had reported Claude would hallucinate names for things like @author tags, since it didn't have context on the account. So we added the account email in the model's context for signed-in users. But when Claude wrote a command itself, like a curl call, it sometimes put your email address in the User-Agent as a contact address. Some sites ask agents for a contact, and Claude would sometimes provide the email address in its context.
Until the next release arrives, 2.1.317 and later have a built-in mod (on by default) that asks you before Claude runs a web tool call or a curl-style command containing your email (mpoteat covered it here: https://news.ycombinator.com/item?id=50386031). But it still doesn't prevent all cases, like when Claude writes a script and runs on its own. The stable channel is still on 2.1.311, so update to 2.1.317 or later.
Again, this should not have happened, and we should've caught this behavior earlier.