Re:Neo (Autonomous ML engineer) is leading the MLE Bench wit...
by andsoitis on Tuesday August 19, 2025 @01:45AM
(#44948601)
That’s not the GitHub for Neo, is it?
When I go to the website I can Join waitlist - https://heyneo.so/
Smells vaporware-y, fwiw
Re:Neo (Autonomous ML engineer) is leading the MLE Bench wit...
by Vij137 on Tuesday August 19, 2025 @01:50AM
(#44948624)
Wow..Vaporware? how?
Did you even check the leaderboard on openAI's official github?
If not, first check that before calling it whatever.
Here:https://github.com/openai/mle-bench
They are not just distributing trophies to anybody. It was built over 10 months with a small team and some GPUs.
Re:Neo (Autonomous ML engineer) is leading the MLE Bench wit...
by andsoitis on Tuesday August 19, 2025 @04:52PM
(#44956160)
How can I try Neo myself? I couldn't find a way.... as I said, the official Neo website just talks about a waitlist, so seeing benchmarks and no way to try it out or use it in some fashion makes it.... not useful to me? Unless I misunderstand how it is integrated into other systems (which the Neo website doesn't tell me either).
Seeing a benchmark for something that is hidden behind a veil isn't worth much.
If I am missing something key, I sincerely apologize, but I really tried to find a way to interact with or use or download Neo or something, but it is just a void, other than the empty website and this benchmark.
Re:Neo (Autonomous ML engineer) is leading the MLE Bench wit...
by Vij137 on Tuesday August 19, 2025 @07:52PM
(#44957341)
No worries. I appreciate the concerns you shared and we will work on them. We are releasing access next week to our waitlist. Please signup and share your id, i will bump it up.
Re:Neo (Autonomous ML engineer) is leading the MLE Bench wit...
by gauravvij137 on Tuesday August 19, 2025 @02:02AM
(#44948676)
Did you miss reading the official leaderboard score on openai/mle-bench page where it clearly states that Neo has the best score on mlebench?
Neo is now the SOTA Machine Learning Engineering agent on OpenAI's MLE-Bench.
For context: Microsoft's RD agent is at 22.4%.