News for hackers. Stuff that matters.
Re:Claude Fable 5 by teiferer on Wednesday June 10, 2026 @02:09AM (#48472119)
> wrote a fuzzer for its prototype which explored and verified that its reasoning was correct. It absolutely nailed it.

For such a data structure, "nailing it" means a formal proof of correctness. Fuzzing, as useful as it is, is merely throwing dirt at the wall and seeing if anything sticks.

Re:Claude Fable 5 by josephg on Wednesday June 10, 2026 @02:13AM (#48472143)
I’ll ask it for a formal proof when I get home and see how it goes.

I’ve read plenty of papers with “formal proofs of correctness” that turned out to have huge flaws. Machine verifiable proofs I trust. But I’ve personally found more bugs with fuzzing than I have via proofs.

Re:Claude Fable 5 by teiferer on Thursday June 11, 2026 @05:22PM (#48496590)
Oh I actually mean machine checked. Indeed, formal pen-and-paper proofs can have flaws, since they are essentially code without test coverage.
Re:Claude Fable 5 by noduerme on Wednesday June 10, 2026 @02:15AM (#48472164)
In the real world, many of us don't have the time to create formal proofs. But our instinct in testing where edge cases may exist in code that we wrote is a type of refactoring that happens in our brains during the coding process. Hand the coding off to a machine and you have no idea where to start looking for the flaws.
Re:Claude Fable 5 by bluGill on Wednesday June 10, 2026 @09:12AM (#48475793)
> Hand the coding off to a machine and you have no idea where to start looking for the flaws.

I have found this quickly becomes false. I have learned I cannot review llm generated code as if it is written by a trusted senior developer (where I often just do a quick look, see nothing obvious and hit approve). Once you start reading the code in depth with the goal of understanding you quickly see the places where flaws are likely. Sure I start with no clue where to look, but it doesn't take long to see things.

Re:Claude Fable 5 by noduerme on Thursday June 11, 2026 @04:25AM (#48487761)
Yes but it takes much longer to trace them. Because the LLM code almost always gravitates toward data blobs and highly dynamic objects and spaghetti that takes a ton of cognitive load to understand what their failure modes are. Even when it does document them.
Re:Claude Fable 5 by teiferer on Thursday June 11, 2026 @05:23PM (#48496610)
> In the real world, many of us don't have the time to create formal proofs

Of course not. That's why they are so rare. But I thought we live in an AI era now where this kind of stuff can be done by a machine.