News for hackers. Stuff that matters.
Re:A ChatGPT mistake cost us $10k by throw156754228 on Monday June 10, 2024 @03:22AM (#40631064)
Should've been picked up at unit, way before monitoring.
Re:A ChatGPT mistake cost us $10k by igammarays on Monday June 10, 2024 @04:20AM (#40631337)
The error didn't show up until multiple entities were created. I can forgive someone for not having unit tests for scaling issues.
Re:A ChatGPT mistake cost us $10k by huygens6363 on Monday June 10, 2024 @07:22AM (#40632360)
Creating more than one entry is not a “scaling issue”.
Re:A ChatGPT mistake cost us $10k by f35caleb on Monday June 10, 2024 @08:50AM (#40633040)
Agreed. I'm chuckling because I was wrestling with this exact same bug on a FastAPI project last night. I caught it because I have a habit of submitting API endpoints multiple times with the same data to see how the database reacts. Got a key collision when I tried to submit the endpoint the second time and figured out that setup didn't create a new UUID each time.

Unit tests are good, yes. Monitoring is also good. But just taking 30 seconds to do some manual testing will catch a LOT of unexpected behavior.

Re:A ChatGPT mistake cost us $10k by withinboredom on Monday June 10, 2024 @10:27AM (#40634010)
I generally follow this process:

1. Get it working: write the code for the desired behavior, not worrying about making it beautiful, testable, whatever.

2. Get it working well: manually testing and finding edge cases, refactoring to get it testable and writing tests to solidify behavior.

3. Get it working fast: optimizing it to be as fast as I need it to be (can sometimes skip this step). No tests should change here, but only new tests.