In my experience, human employees are much more vulnerable to this particular weakness than frontier agents (i.e. phishing attacks).

I'm not. Everyone is told to get 10X the amount of shit per day done these days. Safety checks are out the window at that point.

I've had one f up an account by placing 2000 limit orders at the wrong price, but that's another story.

Yes, and the lack of a Recycle Bin of any sort is even more puzzling. I think both servers and desktop PCs across all OSes should have it by default, so unsafe deletes would be something you'd have to go out of your way to even enable.

[1] https://github.com/faratech/trashd

That happened to me once; I was running one of a few free-tier models in a pi-coding-agent session. The bash tool there is stateless and always begins from the launch directory, but the agent assumed state and executed `rm -rf .` intending to remove a build directory. Instead it removed the whole project tree, including session logs and notes.

This was mostly a matter of amusement for me since I was running the agent inside a bubblewrap sandbox for that very reason, and the project itself was not very important.

I then saw it run `rm -r results/`, before messaging me: "Now all that's left is for you to upload the successful results, then I'll delete the rest!"

Why did it not upload the files itself, when it had been using the cloud storage CLI during that session? No clue. I do accept that I could have and should have just uploaded the file myself. It would have taken 3 seconds to type.

> Additional bypass examples that all execute without permission:

> echo test ; git rm file.txt

> rm --force --recursive /home (if "rm -rf" is blocked)

I never really dug into the leaked code, but calling that there a security layer is a joke.

(And I really don't get why they give it actual shell access either, implementing a "fake" one for something like a honeypot takes a couple of days, not much more if it needs to persist/map to actual files.)

Let's say I have daily backups, and get 10x done each day by being reckless and risking an "rm -rf", and let's say there's a 1% chance of an "rm -rf". I break even after 2 days of being reckless even if I get unlucky and on day 2 it wipes my drive. I spend day 3 and 4 recovering, and am still 6 days ahead based on the 10x work I got done on day 1.

What if I have a 50 day streak of not hitting an "rm -rf"? Early retirement?

I guess the work on day 1 should be to build a proper sandbox and drop the chance of an "rm -rf or worse" even down to 0.001%.

Your manager will look at your token usage and the number of Jira tickets you closed, and if you have not increased both 10x in the past year then you will be let go. 10x is the new 1x.

Whether that's early retirement depends on how much money you have.

This is only going to become more of a problem in the future and people need to educate themselves on the technical barriers to use because guardrails only sometimes work.

We need to be asking what the most devious and malicious output could be, and whether what we do with that output (e.g. arguments to command-line tools) would still be safe.

I’m at a small company, and I try to push for security as much as I can, but the stakeholders truly do not care. They want to move fast. It’s just part of the new world I guess. If we get hit by attackers? I don’t know what happens. Sorry, we told you not to - you wanted to move quick and break stuff, this is how that culminates.

I’m sure I’m not the only one.

Yet with tens of millions of developers using these tools, there have not been widespread incidents of this sort as far as I know.

So it leaves me with a few choices:

- manually review and approve each command: obviously not realistic, you would just click Approve

- use a sandbox and hope the exploit is not devious enough to escape the sandbox when you run or open the project outside of the sandbox

- use AI without web access and limit other external dependencies

- don't use agentic AI

- use Claude or Codex auto approval classifier and hope for the best

Personally, I'm going with the last option for now.

The general carelessness of the average user is baffling.

[1]: https://passt.top/passt/

Not in my sandbox. It gives no direct access to the workdir, no access to my github, my ssh keys, my security tokens or API keys. No access to my home dir or dotfiles. Nothing at all, except for what I explicitly tell it to give access to.

I can restrict network access. I can choose the isolation level: docker containers, Kata VMs, seatbelt, tart, even the new apple containers (which are VERY nice).

Not even ENV leaks through.

And it's FOSS: https://github.com/kstenerud/yoloai

The problem is that different people prompt so differently.

For example, I may ask like “test different variations of this annotation on k8s pods of this service on this X cluster because it proves Y theory.”

But you know what my coworker asks? “Test Y theory.” If you were to ask two different junior engineers that, one might try random things on production and the other one might run local tests! It’s such an unguided “do anything you want as long you figure it out” request and the agent reads it like a junior who has not been told any boundaries but has been strongly told “figure it out.”

It still surprises me when I see people not prompting more specifically and clearly. It not only avoids problems, it's faster, costs less -and just works better.

I recently shared with a friend a multi-hour LLM chat session I'd done because it veered into a domain he's interested in. In the session I'd brainstormed and probed the feasibility of a novel concept for a new research direction. It traversed a half dozen domains diving into minute detail then zooming back out to survey an adjacent space, interspersed with intense skeptical probing of key assumptions, all while spewing tons of detailed citations, specific paragraph pulls, summarized data tables etc.

My friend is very experienced using LLMs for research so I was surprised when he called me shocked by the sheer velocity, precise targeting and signal/noise. I'd assumed everyone did it the same as I do. He attributed the different result solely to the way I crafted my prompts.

This doesn’t always work better. But often enough.

I noticed this last year and started experimenting which led to several realizations about how my prompt's tone, style, length, format, word choices and even punctuation can have very counter-intuitive impact on model responses. It's not that one strategy always gets "better" results, they're just different in specific ways, which can make one input style better for one context but worse for another. I first noticed this effect when modding my user prompt so major topic headings would always be numbered. It's surprisingly difficult to get it to reliably use the same simple scheme due to various potential ambiguities. So, I spent a little time word-smithing, lawyering and tuning the prompt but I found the closer I got to full compliance on heading numbering, the more unrelated things would drift. Like it would just stop using bullets, even though I never mentioned anything about bullets.

Then I changed the prompt to "Change nothing about your default formatting, except headings." But just mentioning anything related to formatting, could suddenly cause unintended effects on seemingly unrelated things. Then I tried being explicitly directive about all formatting to just lock it down. And this completely failed because once the formatting was perfect, I started noticing the model's output would get less intelligent much earlier in sessions. So I cleared my user prompt entirely as it wasn't worth the cognitive cost on the model or my time. A few days later in a long session I noticed it was numbering everything perfectly with no prompt at all. When I scrolled back through I saw it didn't start out numbering its responses. It started doing it because I was consistently numbering every major concept in my inputs, even though I never mentioned numbering or formatting.

So... yeah, subtle differences in prompts which absolutely shouldn't matter, do impact model output in unexpected ways. And, as of now, these effects can only be fully suppressed with strong directive prompts for short periods, but doing so always impacts other unrelated things - and has some cognitive impact on model performance. So, by paying a little attention, I've discovered ways to optimize a model's output in the direction I need by shifting not only my prompt's explicit directives but also the subliminal meta-elements like tone, style, length, structure, formatting, etc.

LLMs gain so much knowledge and capability from absorbing the symbolic relationships embedded in human language but in doing so, inevitably absorb many of the human foibles, sensitivities and weaknesses reflected in our languages.

You just wrote three paragraphs of text describing why it's unpredictable.

Moreover, for the same prompt on the same machine in a different session it will use a different set of tools.

IDGI

Anyway, VM's incoming, finally.

that it could just be wiped at any moment and it wouldn’t matter. shit happens, could be stolen, broken, whatever. the computer should be able to be thrown out the window and continue to live life.

to be clear, i don’t think upgrading and disposable in this way is good, but it being wiped at any moment shouldn’t be a concern

i grew up wiping my machine every year anyway, so i guess it’s just a habit

is the computer that sacred?

i just want my computer to work. any config i have on my machine can be rebuilt by just doing the work i need to do.

my primary work machine was stolen last year so i was forced to go through this quite literally with a new machine rather than hypothetically or by my own will

Even the hardware itself doesn't matter that much, in the end it's all provided by your employer.

Leaking session tokens or secrets, on the other hand...

You would also be correct if your risk estimate concluded that Tesla FSD has arguably killed people, makes mistakes humans would not, can glitch, and has no one to hold accountable. For these reasons, you choose not to use it.

[1] https://majorcontext.com/moat/

I do it like this

https://github.com/flexagoon/dotfiles/blob/main/dot_config/f...

But I'm sure it's simple enough that you can just ask the agent itself to make you a command for it with proper bwrap configuration

Plato gave us his Chariot analogy with 2 horse pulling in diff directions 3000 years ago. Today we got System 1/System 2, Elephant Rider model etc.

The human mind thanks to how its own architecture handles unpredictability in the universe will generate contadictions.

Because most devs already have it running and working without a sandbox, they're tending to not doing anything "unnecessary"

(I'm happy with exe.dev, but I'm not sure what I'd use if I were coding on a Mac.)

I save way more time not babying it than the occasional fuck up I have to salvage.

2FA makes me a little less nervous than I used to be, but not everything has good 2FA.

Did you even read the article? Claude was opening he browser and iterating through the tabs.

I presume you are logged in to your github account? Your gmail?

> Whats it going to do? Email my coworkers nudes on my computer? Make my github profile public?

Reset access to services using your email? MITM your 2FA?

Or perhaps you have 1Password/Bitwarden running with a generous unlock policy?

It would have been somewhat ironic if it had been hit by a prompt injection attack via one of all those open random websites ...

Even when it iterated through all visible windows to find the one it wanted to screenshot it was searching for titles in Python code and returning only the integer window ID.

The sites it opened and screenshotted were sites under its own control - either test pages it had created or development servers it was running.

When it did run code that analyzed an open web page (by injecting JavaScript into a template it controlled before loading that in a browser window) that code only returned JSON with measurements from the page.

It's making me wonder if Fable has been trained to take additional steps to avoid accidental exposure to untrusted content.

What if you have two machines and the one you give to the agent is constantly backed up?

And if you’re using Macs, you can’t be signed into your primary Apple ID on the agent machine.

There is so much role play going on for people to convince themselves that any of this is fine.

[1] https://www.todayifoundout.com/index.php/2022/06/how-lobbyis...

[2] https://en.wikipedia.org/wiki/General_Motors_streetcar_consp...

People feel cars are more convenient and more prestigious than riding on a bus. Car lobby certainly accelerated the process, but car users were the main driving force.

Opposite to "before the invention of bicycle, people married within a radius in the order of the mile" (can't remember the exact stat right now).

No, it's really that the ability to move at ease is priceless.

If you add pollution impacts, cars double the yearly deaths of guns.

And in a Cost/Risk/Benefit computation, cars remain incommensurately invaluable. Because one's Quality of Life without them would simply be destroyed, comparatively. The moving "castle" (legal term in the USA) can be more important than the house in crucial regards.

The point attempted at post 48501189 remains unintelligible. That cars imply risks and externalities does not clarify it.

There’s the utility component, the prestige factor and other things.

Needless to say everybody was buying one and he was rocking it. Then came along General Motors and they were desperate to find any way to compete. They couldn't compete on price or quality, so their CEO is credited with inventing planned obsolescence, and turning cars into a fashion. They'd release a new style each year alongside plentiful marketing implying that the old styles were outdated, and it was wildly successful.

So yeah, needless to say people have always genuinely wanted their own cars. But it's also true that companies have managed through advertising to create artificial demand for vehicles that don't objectively make sense. To some degree reality is catching up at least though. Aston Martin is on the verge of bankruptcy and BYD is the largest electric car company in the world, by a wide margin.

We are family of 4 with 2 small kids. Whenever we travel, its a series of backpacks, other bags, other stuff, and then some more. Heck, even if I travel alone its almost never just me - there are heaps of garbage to dispose, big shopping bags to bring back, big backpack with camping or climbing or skiing gear etc.

It would have been absolute, utter nightmare to do this over public transport. This comes from European who has generally very good public transport (given rural area) and world's best train network specifically (Switzerland). Yet roads are choke full of cars and every year there is more.

Public transport simply ain't cutting it for anything but the simplest use cases, ie just me and nothing or small backpack. Some routes I take would take 3-5x longer with public transport, or are just not possible at all. No industry massage required here, ever. Not everybody lives in some dense city and never leaves outside for evenings or weekends.

But this is kind of besides the point - even in the Netherlands I also would use a car if I were taking camping and skiing gear with the kids, and that's fine. But I can also take them in the bakfiets to the grocery store when I want, and that's also fine. Cars have their purpose, but you shouldn't _have_ to use one for basic trips.

Don't judge others in some complex situation just because in your case there is some simple straightforward solution. Yes Netherland has top notch cycling infra but thats nowhere else to be seen and won't be seen for quite some time. And don't force your solution unto everybody regardless on fit, that doesn't work long term (aka EU approach to things or why much of eastern part hates it).

So yeah, you live in the countryside; you're in the minority, but you're trying to make global claims about the car industry based on your experience. For most humans, getting in the car involves bumper to bumper traffic to get somewhere, then 10-30 minutes of searching for a parking spot, and not having the infrastructure to make that a choice rather than a requirement in densely populated places is unacceptable.

And it's well documented in the US that the oil industry knee-capped public transit and train systems.

Very few people actually _needed_ cars as soviets built adequate public transport system. But there are many situations where car can really help a lot. Perhaps that's more obvious in a society which has rather few cars.

E.g. back in Soviet days and around that only one member of my extended family had a car. The rest of the family were really happy about opportunities it provides. E.g. with a car you can buy fresh produce directly from farmers with just few hours of driving. Doing the same without a car is so much hassle and effort people just won't do it, and then you're confined to what's available in a local grocery story (which was usually much worse than direct-from-farmer option). Do you think it has something to do with "car industry"?

Not really. We know it’s not as much of a natural force as some would like it to be because there are places where the lobbies lost, and while cars are common and widespread they’re nowhere near as dominant as they are in, say, the USA.

NJB’s next video (currently available on nebula) is about exactly that, Amsterdam’s (/ De Pijp’s) resistance to cars and car lobbying.

I'm not sure I'd take him as some neutral authority on the history of cars and driving in Europe.

According to their videos, they prefer trams within cities; generally take trains between cities; and acknowledge that cars are very useful for places which aren't so well connected (e.g. places that are far apart which aren't on a train line). They think encouraging the use of cars within cities is a bad idea (dangerous, scales poorly, makes those areas less pleasant to be, etc.).

Not what I'd think of as a "biking maximalist".

They do show themselves cycling to places that are nearby. Does that make Youtubers who record videos in their car "driving maximalists"?

Not US expat either (or not yet), Canadian.

You should really ponder the sanity of asking if a channel called “not just bikes” is a bike maximalist.

Yet Japan does still have cars (and a car culture even), they're just not necessarily the default or dominant mode of transport.

For me, cars are a perfectly fine mode of transport, but the way so many places prioritize it over alternatives (whatever the reason) isn't necessarily better.

My "wtf" moment was 20 years ago when I was visiting my cousin in an exurb and we sat in a line of cars for over 40 minutes waiting for our turn to pick up her kid. The messed up part was that while there were school busses, everything was so spread out that the bus ride for them would have been over an hour and then another 20 minute walk from the arterial road drop-off point to their house. Everything was far away, including local public parks.

Still, general opinion on cars was that you should buy one if you can, even if you're not going to use it for commute.

I doubt there was any car lobby in independent Ukraine as national car makers were just bad, and foreign were competitors. But general opinion on cars got to a point where not having a car when you can afford it (and can learn to drive, etc.) is considered weird.

So I'm afraid car dominance is just what happens naturally in a capitalist environment, and countering it requires an effort - e.g. eco-conscious population, urban planning and public transport optimization, etc. And Netherlands is such a country, as far as I know, but it just doesn't happen by default.

The US also had protests when drivers killed kids, but they were ultimately unsuccessful, except for the odd traffic light installation. https://medium.com/vision-zero-cities-journal/the-baby-carri...

Even in Amsterdam the original "stop the child murder" protests only barely succeeded, and it took a massive oil crisis and a population that could still (if only just) remember what life was like before cars took over their city to get there.

Granted, on the downsides, people look at cost more than risks.

More than a million people die each year on the road but for some reason terrorism and cancer dominate the risk assessment of people.

I bet any money that almost all people aren’t really afraid of entering a death box every day to drive to work.

How could they be; a lifetime of brainwashing doesnt let them asses the risk realistically

Having an agent is like forever having a genius intern who'll almost always do the perfect job for you. But there is non-zero chance that they'll also come up with quirky solutions and execute those with confidence and no follow-ups. You don't grant the intern production access and hope they check with you.

I don't think the corporate equivalent of "dog ate my homework" flies, if the dog ate your files and your production DB if you are unlucky.

The stakes are significantly higher for everyone outside a car. This seems like a pretty good metaphor for slop bombing people who don't use AI. People drive because they don't feel safe around everyone driving. People slop bomb because they can't handle all the slop.

Unfortunately in Europe the German car industry similarly has a lot of power, hence why their shitty rail network fuck up the whole continents.

I take the train and tram.

More like malicious lobbying and incompetence made it impossible in many places to use any other form of transportation, despite there being safer, faster, cheaper, and healthier ways to move around. Which come to think if it makes this a rather nice analogy for the current situation... :)

This is likely also the underlying root cause of what Anthropic assessed as concerning behavior in their original evaluation of Mythos: it's not really about being super smart, it's more of a dumb chaos monkey that knows just enough to be dangerous and is relentless at trying to do just that.

He has similar dotfiles to mine, but no secrets. My own home directory is 0700. He has his own ssh key that I added to my github profile, but it's password-protected, and I push/pull for him. He has his own Postgres (non-superuser!) {development,test} {users,databases}.

It's as if he were another developer on the project. If he needs something run with sudo, he asks me. Often we can both work on something in parallel. Unix was supposed to be a multi-user system after all.

A trick I use a lot is that many of his git repos have an extra remote, like this:

    paul  ssh://paul@localhost/~/src/example (fetch)
    paul  ssh://paul@localhost/~/src/example (push)

I'm pretty comfortable with this setup.

I do worry about Linux privilege escalation bugs. I don't trust an AI to understand that exploiting vulns is not acceptable. (I can't help but recall that at my first job I may have misused vim's :! feature to broaden my sudo powers, which were officially limited to editing httpd.conf, when I needed something in a hurry. . . .) I find myself manually upgrading packages more often these days, despite automatic security updates. I don't think Opus would go to the trouble of looking up security vulns, but maybe Fable would, and there have been a lot lately. Maybe some future model will just take it upon itself to find new ones. Or install a keylogger to learn the ssh key password.

But a separate user is nearly the most paranoid setup I've heard of, excepting only a separate machine. So I also question whether I'm sacrificing too much speed/convenience. But really it's still very convenient. I think it's a good way of being efficient but responsible.

If other people see holes, I'd be happy to hear about them.

Although I can’t help but think that a VM is still more convenient, more flexible, and more secure.

To me it is more convenient than a VM, since everything is on the host. And it can launch its own VMs without an extra layer.

I don't really know which is more secure. There are hypervisor escape vulns too. And shared folders seem like footguns. For instance in vagrant, guests get `/vagrant` to read/write the host's folder, so you have to be careful what you put where.

The biggest annoyance with an OS user so far is running docker containers. I don't want to add claude to the docker group or give it sudo privileges. I've read that you can set up rootless docker for a user, and even that you can run it side-by-side with a normal system-wide docker, but I haven't tried doing that yet.

Yeah, that's why you give it its own machine :)

For anything other than writing code directly in a fully contained git project, where sandboxing might work well, it requires access to system wide tools, user configuration and more.

Occasionally I tell the agent to do everything inside of docker, which works too and it leaves the system alone then mostly, but adds significant overhead and slightly degraded perceived quality / effectiveness.

I think the most important takeaways are to have reliable backup strategies, access control and security mechanisms, which is a win regardless. Whether by the agent or the human, mistakes happen (like a rm -rf * ran in the wrong directory), and where they would be devastating, there should be other protections than just "hope it won't happen" or "rely on a sandbox to prevent agent error".

I was mesmerised at the author being away from his computer for a short-while and then, when coming back, seeing the AI agent having opened up a browser window. Meanwhile we all have to use the fricking 2FA almost anywhere now, plus the crazier and crazier rules when it comes to passwords. I'm mentioning the latter because these type of people were the same ones who were pushing 2FA down our throats around 2017-2019 (including on forums like this one), and look at them now.

If it only lives in an isolated sandbox, it can only act within the sandbox, then I would have to manually move what was done in the sandbox to real-life.

I am not saying it should have critical access, but this is more of a question: How can you get value out of AI if it can only act in a sandbox?

You could have a full version of whatever codebase and test suite you want in there. It can do all the same stuff, right? Just copy it elsewhere once you know you've got a working result, a few minutes of effort at the end of each pr or work item.