I built a vulnerable app and spent $1,500 seeing if LLMs could hack it

Re:I built a vulnerable app and spent $1,500 seeing if LLMs ... by FloorEgg on Wednesday June 03, 2026 @10:30PM (#48392928)

I think that these companies are going to have to, and will, invest in some sort of validated identity context to avoid the lowest common denominator.

The first challenge is making sure the guard rails work and are robust. Companies are still working on this.

the second challenge is being able to reliably adapt them as appropriate per user. E.g. allow someone to pen test their own app.

The third challenge (which blocks the second) is to be confident about what is safety-aligned with a specific user.

I think the later will be a hard problem, but they will be highly motivated to solve it.

[ Parent | on HN ]

Re:I built a vulnerable app and spent $1,500 seeing if LLMs ... by bulbar on Thursday June 04, 2026 @12:36AM (#48393919)

I believe you are overthinking it. I think the sister comment is right that it's a business decision foremost to restrict actions within specific plans for upselling purposes.

Without laws, AI companies have a strong incentive to be useful for their users, whoever they are, whatever they do. The only self regulation is about significant public outcry but that only helps so far.

[ Parent | on HN ]