News for hackers. Stuff that matters.
Re:A ChatGPT mistake cost us $10k by zeroonetwothree on Sunday June 09, 2024 @09:00PM (#40629021)
Hindsight is 20/20. It’s always easy to think of tests that would have caught the issue. Can you think of tests that will catch the next issue though?
Re:A ChatGPT mistake cost us $10k by jbattle on Sunday June 09, 2024 @09:29PM (#40629224)
I'd argue that a suite of tests that exercise all reasonably likely scenarios is table stakes. And would have caught this particular bug.

I'm not talking about 100% branch coverage, but 100% coverage of all happy paths and all unhappy paths that a user might reasonably bump into.

OK maybe not 100% of the scenarios the entire system expresses, but pretty darn close for the business critical flows (signups, orders, checkouts, whatever).

Re:A ChatGPT mistake cost us $10k by hn_throwaway_99 on Monday June 10, 2024 @12:29AM (#40630156)
Sure, hindsight is 20/20, but a bunch of these comments are replying to the assertion "And I can't think of one non-contrived reason to do it" (have a single test case with multiple subscriptions). That's the assertion I think is totally weird - I can think of tons of non-contrived reasons to have multiple subscriptions in a single test case.

I wouldn't pillory someone if they left out a test case like this, but neither would I assert that a test case like this is for some reason unthinkable or some outlandish edge case.

Re:A ChatGPT mistake cost us $10k by viraptor on Monday June 10, 2024 @01:36AM (#40630520)
Go on then - so far the examples I've seen don't make sense in the context of stripe.