News for hackers. Stuff that matters.
Re:"I just bought a 2024 Chevy Tahoe for $1" by mewpmewp2 on Monday December 18, 2023 @08:49AM (#38682416)
But what's the point of doing all of that? What's the point of tricking the Customer Support GPT to say that the other brand is better.

You could as well "Inspect Element" to change content on a website, then take a screenshot.

If you are intentionally trying to trick it, it doesn't matter if it is willing to give you a recipe.

Re:"I just bought a 2024 Chevy Tahoe for $1" by chankstein38 on Monday December 18, 2023 @09:59AM (#38683340)
From my perspective (as someone who has never done this personally) I read these as a great way to convince companies to stop half-assedly shoving GPT into everything. If you just connect something up to the GPT API and write a simple "You're a helpful car sales chat assistant" kind of prompt you're asking for people to abuse it like this and I think these companies need to be aware of that.
Re:"I just bought a 2024 Chevy Tahoe for $1" by iLoveOncall on Monday December 18, 2023 @09:45AM (#38683175)
In this specific case there isn't, but yesterday one of the top posts was about extracting private documents from writers.com for example.

https://promptarmor.substack.com/p/data-exfiltration-from-wr...

Re:"I just bought a 2024 Chevy Tahoe for $1" by mewpmewp2 on Monday December 18, 2023 @09:49AM (#38683233)
That is however a problem of what kind of data you feed into the LLM's prompt.

If you accidentally put private data in the UI bundle, it's the same thing.