dare944

> I believe that this line of thinking, while fine in theory, fails when faced with reality. As evidence of developers being overwhelmed by AI code I'd like to offer this post with almost 600 votes and 700+ comments titled "How to deal with shipping AI-written code you don't understand?".

While I certainly haven't read all the comments on that post, I interpret many of them as supporting my position that AI users have to take responsibility for their actions (e.g., https://news.ycombinator.com/item?id=50151036).

I definitely agree that many people (and not just devs) will find themselves under intense pressure from their employers to use AI to speed up their work. And that this pressure will incentivize them to accept the actions and output of AI without proper review. But this doesn't eliminate their (or their employer's) responsibility to conduct that review.

In my mind, it's as if a trucking company told their drivers they had to make deliveries as fast as possible, and penalized them for being too slow. Despite this, it is still the driver's responsibility to drive safely, and the company's responsibility to facilitate safe driving.

> Once you have a malicious model inside your infrastructure generating thousands of lines of code, it doesn't matter how good the sandbox is - if the generated malicious code makes it through code review once, it is game over.

As I pointed out before, all models, not just open-weight ones, can request actions or produce output is unsafe or malicious. So the need for proper oversight exists no matter which one you choose.

No, because the agent is run inside a sandbox that prevents execution or committing of code downloaded from the internet without express review and approval, right? Right?

If that's not the case, then the problem is the user of the AI, not the model. Any model is capable of requesting actions that violate security or are otherwise harmful, not only ones that have been maliciously trained for that. Both OpenAI and Anthropic have proven this multiple times.

In the end, the model itself is just a bunch of numbers. It's up to you, as the person running the harness through which the model gains any agency, to restrict the AI from performing malicious actions.

I knew for decades that my parents were going to die, but I still mourned the day it happened, and for a long while afterwards.

Your inability to give space for those who love math to mourn its inevitable evolution is unnecessarily cruel.

Lol ...and they call AI inhuman!

No. They might have better braced themselves for the hit, but the wound was not self-inflicted.

Doesn't need to be. Point an AI at it and say reproduce the behavior.

Point is, if need be, the open source version could be forked and carried forward by someone else, using the public code as a definitive spec for any missing functionality.

Of course. There's never any free lunch. There's always a cost to software development. Just sometimes the cost gets shifted in a way that's beneficial to the general public. But it never lasts.

At least with bitwarden, if the value they're trying to extract becomes more than the product is worth, in terms of real cost, lockin or transparency, at least the code is open now and for the foreseeable future. And when it comes time to fork it, AI will make it easier for the future maintainer(s) to keep the fork alive at minimal expense.

Per their public discussion on the topic, the non-OSS licenses will still be public and accessible for review.

I was suggesting to ask that question in a separate context, just as a means to understand the potential risks involved. I wouldn't ask this within a coding agent context.

Lately I've been asking a lot of these sorts of questions while setting up a sandboxed environment for my agents (generally, claude and pi). I was quite surprised to learn how many different ways there are to configure git run a script out of the .git directory. And the various AIs know all about this.

At this point my agents get read-only access to my .git directories.

Ask your favorite LLM what a malicious script dropped into the right place in your .git directory can do to you the next time you run git outside of the sandbox.

And of course, if the agent has access to the network (which is probably required in order to talk to the AI server), then you have to think about what other things on your local network it could get into.

Mail / IMAP server for my family domain. Small web server with a few static content sites. Trillium notes server.

Pretty low maintenance overall - I can go months without even thinking about it.

No amount of "they did it first", "they did it better" entitles them to monopolistic behavior.

It happens. Been guilty of the same myself.

> I want to trust science and dislike anti vaxxers but it’s difficult to argue against them when researchers act like the caricature they are mocked as.

Wait, you think the article was written by the researchers?

You know, it's also difficult to trust someone's criticisms when they misassign responsibility. Just like anti-vaxxers do.

It may take 100s of instructions to enter a deep sleep state, and 100s to 1000s more to exit it. If you anticipate having to service an event sooner than that, you don't enter sleep at all (especially since there's likely a point at which you're committed to sleep, and have to go all the way down in order to come right back out again). Instead, you hang around twiddling your thumbs until the event comes along.

Now if your processor has a halt/wait-for-interrupt instruction (most do but some don't) you can escape into assembly and use that. But it probably makes little to no difference to energy utilization, and of course its not portable. A nice while (true); would seem obvious, except that the C++ committee insisted that it wasn't.

Just one example of where the committee lost sight of the fact that it was defining an imperative programming language.

This is myopic. In many cases it takes time, and sometimes considerable programming effort, to enter and exit low power modes. So you don't do it willy-nilly; you do it when you believe the system has quiesced. That means, on a purely interrupt driven system that is not yet ready to sleep, the code may very well be spinning in an empty infinite loop somewhere.

There's no need to inform the "user" because there's nothing wrong with the system. Its simply waiting until the benefit of sleeping outweighs the cost of getting there.

And where to you think all that "wasted" energy/cycles would go otherwise? Why do you presume there's some other, more efficient way the CPU could be spending its time while waiting for an event to process?

I, the programmer, will decide what cycles are wasted or not. That the C++ committee thought they knew better is hubris.

For some reason I read this in James Burke's voice and I could swear I was teleported back to the 80's watching a hitherto unseen episode of Connections.

Interesting stuff.

> I am of the school of thought that even if Broadcom gives VMware Standard away free for a year, it would be hard to convince customers that they need to stick with the platform

Yep. They're building the cage around you. Why would you stick around?

"THIS IS A TEST

This country is conducting a test of the Emergency SAIfguard System.

THIS ONLY A TEST

In the event of a real emergency you would have been given instructions on how to grab your ankles and kiss your butt goodbye.

...

This concludes our test of the Emergency SAIfguard System."

I think it would be fine if more people used "add color to". I think it's a great turn of phrase.

"Load bearing", not so much. Ick.

The idiom "added color to" as an expression of adding detail or nuance to something goes back much further than the AI era. I've certainly used it at various times in my career spanning 4 decades now.

Sorry. I meant, is there any way to block stories on HN that link to Twitter posts?

Won't be reading any twitter post on HN.

Is there a convenient way to block them?

I use seccomp filters on linux in my AI sandbox.

Not if they designed the algorithm right.

As I understand it, the current watermarking methods rely on a secret key, making the detection schemes a black box to anyone not in possession of the key. This means organizations like Anthropic are free to make any claim about authorship they want, true or not, and no one can call them on it.

> If they had no useful purpose, why did we think about them for years?

I don't see a 'we' here. By your own admission you didn't spend years thinking about complicated math topics. So maybe you simply don't see the "purpose". Or maybe no one sees the purpose yet because for some math practical applications take years to discover.

You can insist otherwise, but I suspect that so long as the government continues to ignore people's basic need for privacy, things will continue to get broken.

There's no question that, in general, people expect businesses to provide a valuable service that does not harm society in the process. Of course, the consensus as to what things are valuable and what things cause harm changes over time (as is clearly the case for cigarettes). As well, when people sense a violation of their expectations, they should certainly take their outrage with them to the ballot box.

But when people sense that the ballot box isn't listening; that the government is both in bed, and emboldened, by the very thing that violates their sensibilities, they often step outside their part in the social contract and do things that speak louder than simply voting, like destroying property. If you are a US citizen like I am, you will know very well that this method of "discourse" is a foundational and often celebrated act in American history.

These people destroying a few cameras here and there aren't solving the problem of invasive and unaccountable government surveillance. But they are speaking their passion about it, not just to the company or the government but to their fellow citizens. And it looks like some people are starting to listen.