People give them access to email, so they can request password resets and act on them. That's a fun one to think about too!

My friend, there are people are out there telling their AI personal assistants to go on Amazon and purchase a surprise present for themselves based on their known preferences. And then they intentionally do not inspect what the AI purchases because it would ruin the surprise.

People have wildly different risk calculators.

Ordering from Amazon is way less risky than a password vault imo. The password vault does not have a return policy, for starters.

Beyond that, access to logins enables buying things on Amazon, sending emails to everyone you know, registering domain names and spinning up servers, stock trading, etc.

> The password vault does not have a return policy, for starters.

When a couple thousand dollars worth of perishable food shows up on your doorstep, what then? Not everything on amazon can be returned.

They typically refund food and don’t require you to return it.

If not: still better than giving ai your password vault. I’m not saying it is a good idea to let ai buy from Amazon. I am saying that giving ai access to your password vault is still a superset of letting ai buy from Amazon.

Sure it's a superset but other than the financial angle what's the perceived issue here? It seems like access to burn through your money is the underlying problem.

It could send personal and significantly problematic emails to family or professional contacts.

It could change LinkedIn to add competitive work.

It could sign a married person up for dating sites and invite people to your house.

It could lie on your taxes.

It could attempt to join a terrorist movement or elicit other illegal activity on your behalf.

It could find many other creative ways to ruin your life and/or reputation, in the same brute-force manner as recent cybersecurity issues that previously would have required impossibly large teams of people.

Even if burning through cash is the only problem you care about, though, credit cards on Amazon is a small part of the financial picture for many people. Banks and brokers can be a much bigger issue.

I currently work for a big French company, for which I have developed an "alternate" (emergency and temporary) accounting system because their actual system is broken, and another team is trying to fix it (has been trying to fix it for nine months now).

Said alternate system is a bunch of Python scripts, an sqlite database, and a file containing various API keys to read and write to other company systems; it runs on a dedicated virtual machine.

I share all the code I produce to the other team. But yesterday I found out that the IT department gave the other team full access to my production machine, without my knowledge or consent (which was bad enough), and that the other team gave access to Claude, which has bean roaming free for a while now, on that production machine.

I've thrown a big fit over this; I don't know if I'm over reacting but I think it's insane.

  > the IT department gave the other team full access to my production machine

Your error is here.

It's not your production machine. It's the company's production machine. Always expect the company to do whatever it wants with its machine, and use the machine accordingly.

Yes I understand it's not "my" machine, but it's the dedicated machine my system runs on.

That someone would let an agent run on it at any time, read what's there, and possibly modify the code or interrupt a process, without even informing me, is what I find problematic, to say the least.

You don't have intrusion detection / auditing setup?

Did you read the "emergency and temporary" part?

Have claude build you one :)

What good would such a thing do when the anointed gatekeepers (ie IT) have explicitly granted access to the transgressors? The scenario you're responding to here amounts to corporate slapstick.

That's a geography error on your part. Humans in the EU have an expectation of privacy even on work-issued devices.

I also work in the EU and privacy on company's machine is a more complex topic that's case by case dependent.

The employer can't enter your machine to look at your personal info just because they feel like it, but they can access it for work related reasons to access work related info, if they have just cause.

Claiming the employer in the EU has no right to access your machine is plain false. It's the company's machine, on the company's network, inside the company's premises, it can definitely access it for a business use case. Now if you happen to put your private info there because you're sloppy, then it's on you. How hard is it to avoid putting personal info on company property?

I have "my" office at the university.

I expect that I can be relocated to another office with a month of advance notice. Also cleanup, fire security check and fixing water leakage while I'm not there.

I don't expect that someone else will provide the keys of "my" office to the biology department when they have to keep a few sheeps during the weekend.

The image of a bunch of sheep pooing all over your office while you are not watching is the best metaphor I've heard for AI agents in a long time!

> I don't know if I'm over reacting but I think it's insane.

Your employer has gone beyond testing in prod to vibecoding live in prod, secrets included. Yes that's completely insane. Perhaps this exact mentality is how the primary system wound up broken in the first place?

Yes, but happily enough for me they're not my "employer", only my client.

For consumer AI products and non-enterprise business accounts, if you look closely at the ToS it says that you retain the rights to your inputs and the outputs you receive, collectively called “content”. You can opt-out of training in your content. However that doesn’t cover the results of tool calls - which are fair game for training. As are any “safety events”, which as far as I can tell is a euphemism for “something the model has never seen before” - including enterprise accounts when it comes to “AI safety.”