> It strikes me that if their models were good enough for recursive self improvement
I just can't imagine this is true when their overreach problem (doing more than you ask), hallucination problem (lying), redefinition problem (doing something completely orthogonal to what you asked), and harm problem (doing something that is exactly the opposite of what you asked) are all far too high at this point, and by most measures - all of the recent models are worse on most (or all) of these metrics.
They are giving up here in RLHF to benchmaxx the metrics people are looking at to pretend like they're making huge gains, when they're largely running in circles.
They are getting better at pass@1 at the expense of far worse p90s on real agentic tasks.
I just can't imagine this is true when their overreach problem (doing more than you ask), hallucination problem (lying), redefinition problem (doing something completely orthogonal to what you asked), and harm problem (doing something that is exactly the opposite of what you asked) are all far too high at this point, and by most measures - all of the recent models are worse on most (or all) of these metrics.
They are giving up here in RLHF to benchmaxx the metrics people are looking at to pretend like they're making huge gains, when they're largely running in circles.
They are getting better at pass@1 at the expense of far worse p90s on real agentic tasks.