News for hackers. Stuff that matters.
Re:A ChatGPT mistake cost us $10k by primitivesuave on Monday June 10, 2024 @12:41AM (#40630222)
I spotted the error instantly. With all due respect to your team - this has nothing to do with ChatGPT and everything to do with using a programming model that your team does not have sufficient expertise in. Even if this error managed to slip by code review, it would have been caught with virtually any monitoring solution, many of which take less than 5 minutes to set up.
Re:A ChatGPT mistake cost us $10k by arjvik on Monday June 10, 2024 @01:04AM (#40630337)
To be fair, if I wasn't looking for this bug I never would have spotted it. That being said, you're entirely right that any monitoring or even the most basic manual testing should have instantly caught this.
Re:A ChatGPT mistake cost us $10k by Akronymus on Monday June 10, 2024 @03:44AM (#40631173)
Those kinda issues are why I ALWAYS make an integration test with calling the same insert multiple times.

I did step into that particular trap more than once (passing the result, rather than the function)

Re:A ChatGPT mistake cost us $10k by astromaniak on Monday June 10, 2024 @01:26AM (#40630458)
Easy enough, ChatGPT can write verification code along the main line. Just ask.
Re:A ChatGPT mistake cost us $10k by rvnx on Monday June 10, 2024 @02:36AM (#40630835)
+ humans could have done this mistake as well
Re:A ChatGPT mistake cost us $10k by LigmaBaulls on Monday June 10, 2024 @02:43PM (#40636737)
I spotted it right away. If you plan to use a library in your project RTFM
Re:A ChatGPT mistake cost us $10k by fragmede on Monday June 10, 2024 @02:49AM (#40630906)
Interestingly, you know who else spotted the error? ChatGPT-4o. Annoyingly you can't share a chat with an image in it, but pasting in the image of the bad code, and prompting "whats wrong with the code" got ChatGPT to tell me that:

* UUID Generation in Primary Key: The default parameter should use the callable uuid.uuid4 directly instead of str(uuid.uuid4()). SQLAlchemy will call the function to generate the value.

* Date Default Value: server_default=text("(now())") might not work as expected. Use func.now() for server-side defaults in SQLAlchemy.

* Import Statements: Ensure uuid and text from sqlalchemy are imported.

* Column Definitions: Consider using DateTime(timezone=True) for datetime columns to handle time zones.

It then provided me with corrected code that does

    id = Column(String, primary_key=True, default=lambda: str(uuid.uuid4()), unique=True, nullable=False)
where the addition of lambda: fixes the problem.
Re:A ChatGPT mistake cost us $10k by Kesty on Monday June 10, 2024 @03:04AM (#40630978)
ChatGPT-4o might spot it when asking about the code directly, but this was a conversion from js to python, errors where chatgpt/copilot or any other AI will allucinate or make mistakes to be as close as the original code are very common in my experience.

The other common issue is if the original code has thinsg chatgpt doesn't like (misspell, slightly wrong formatting) it will fix it automatically, or if he really think you should have added a particular field you didn't add.

Re:A ChatGPT mistake cost us $10k by kachapopopow on Monday June 10, 2024 @02:04AM (#40630673)
Having no real experience with python I would assume uuid.uuid4() was some schema definition (like in prisma), so honestly the fact that this bug exists is not surprising at all and I would have done the same mistake myself, but yah one kubectl logs would have been able to catch it immediately.

...also from next.js and prisma to python? ...what?

Re:A ChatGPT mistake cost us $10k by aforwardslash on Monday June 10, 2024 @03:26AM (#40631088)
This. And it seems the team wasn't able to do basic troubleshooting from either database or application log. This was a simple error - what will happen when transient errors (such as implicit locks on tables, etc) occurs. These guys shouldn't be writing code - at all.
Re:A ChatGPT mistake cost us $10k by KennyBlanken on Monday June 10, 2024 @01:45AM (#40630582)
It's not some innocent mistake. The title is purposefully clickbait / keyword-y, implying that it was chatgpt that made the 'mistake' for SEO and to generate panicked clicks.

"We made a programming error in our use of an LLM, didn't do any QA, and it cost us $10k" doesn't generate the C-suite "oh shit what if ChatGPT fucks up, what's our exposure!?" reaction. There's a million middle and upper management posting this article on LinkedIn, guaranteed.

It's like the Mr. Beast open-mouth-surprised expression thumbnail nonsense; you feel incredibly compelled to click it.

While we're on the subject: LLMs can't make "mistakes." They are not deterministic.

They cannot reason, think, or do logic.

They are very fancy word salad generators that use a lot of statistical probabilities. By definition they're not capable of "mistakes" because nothing they generate is remotely guaranteed to be correct or accurate.

Edit: The mods boosted the post; it got downvoted into oblivion, for obvious reasons, and then skyrocketed instantly in rank, which means they boosted it: https://hnrankings.info/40627558/

Hilarious that a post which is insanely clickbait (which the rules say should result in a title rewrite) got boosted by the mods.

I'm sure it's a complete coincidence that the story was apparently authored by someone at a Ycombinator company: https://news.ycombinator.com/item?id=40629998

Re:A ChatGPT mistake cost us $10k by Akronymus on Monday June 10, 2024 @03:45AM (#40631181)
> It's like the Mr. Beast open-mouth-surprised expression thumbnail nonsense; you feel incredibly compelled to click it.

Sponsorblock is great for combating that. (Altough, I conciously avoid channels that mostly do clickbait anyways)

Re:A ChatGPT mistake cost us $10k by cbozeman on Monday June 10, 2024 @02:32AM (#40630815)
The only part of your post that I do not agree with is:

> It's like the Mr. Beast open-mouth-surprised expression thumbnail nonsense; you feel incredibly compelled to click it.

I feel incredibly compelled to ignore it.

Re:A ChatGPT mistake cost us $10k by vsuperpower2020 on Monday June 10, 2024 @02:07AM (#40630685)
>By definition they're not capable of "mistakes" because nothing they generate is remotely guaranteed to be correct or accurate.

This makes no sense. Only things that are guaranteed to be correct or accurate can make mistakes? Everyone knows what "mistake" means in this context. Nobody cares what your preferred definition of mistake is.

Re:A ChatGPT mistake cost us $10k by moritzwarhier on Monday June 10, 2024 @08:08AM (#40632693)
A mistake is usually seen as something that happens when someone (or metaphorically also, something) makes an error, but is capable of solving similar problems through understanding.

Hard to put in words.

But that's roughly what is concerning about the "AI makes mistakes" narratives.

It implies they are caused by a (fixable) fault in reasoning or memory.

LLM "AI" will always respond that it made a "mistake" when you correct it.

It is trained to do so, and humans often behave similarly.

It is hard to come up with a good definition for "mistake", yes.

That does not change that using this in case of LLM hallucinations is misleading.

Re:A ChatGPT mistake cost us $10k by southernplaces7 on Wednesday June 12, 2024 @04:48AM (#40655957)
A good analogy would be nature: Is it making a mistake when something created by it fails in some way? Not really, since there's no conscious intent behind it and thus no correct or incorrect reasoning or a guarantee of either. LLMs share that trait.
Re:A ChatGPT mistake cost us $10k by brabel on Monday June 10, 2024 @02:23AM (#40630766)
> By definition they're not capable of "mistakes" because nothing they generate is remotely guaranteed to be correct or accurate.

By that logic, nothing is capable of making mistakes :D.

> Hilarious that a post which is insanely clickbait (which the rules say should result in a title rewrite) got boosted by the mods.

You have a distorted view of what clickbait is and the rules of this site. I suggest you go calm down and try to stop hating on a technology which is just that: a technology! Like any other, it can be misused, but think about why exactly you feel so passionate about this particular technology.

Re:A ChatGPT mistake cost us $10k by multjoy on Monday June 10, 2024 @03:07AM (#40630990)
Why do you feel bound to defend this particular technology?

It is, after all, incapable of feeling hurt.