Re:Auto-research with codex: How I achieved a 232x Faster Ke...
by qarl2 on Saturday August 15, 2026 @09:28AM
(#49310374)
> It still produces a lot of crap in the later steps (understanding the implementation itself)
I've had success here by adding a phase called "grounding" that attempts to verify its "understanding" by creating tests that modify the running executable to ensure its made the right inference.
Is this variable really MARIO_X? Change it and see if Mario moves. Etc.
As an example in Donkey Kong - the system had trouble deciding if an array controlled barrels or fireballs. There was conflicting evidence.
After many trips through the loop - it realized it does BOTH, depending on which level you're on.
So the "understanding" grows with each iteration.
Even the cheap LLMs are great in doing the awful crud work in the beginning: finding offsets, firmware update file structures, brute forcing checksums, etc.
It still produces a lot of crap in the later steps (understanding the implementation itself) but I'm happy doing this stuff myself then.