News for hackers. Stuff that matters.
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by lionkor on Wednesday May 14, 2025 @08:36PM (#43990645)
Show the training set, and PROVE that the tasks and answers aren't in there. I don't understand why this is not a default first step for proving that this is creating new knowledge.
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by Davidzheng on Wednesday May 14, 2025 @08:59PM (#43990786)
Well that's harder than maybe solving well-known open problems (whose soln's are presumably not in training set lol) but it seems that their examples are not clearly breaking sota, especially on matmul
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by thatguysaguy on Wednesday May 14, 2025 @09:39PM (#43991010)
Are you claiming that for the open problems they give record-breaking solutions for, there were just answers on the web waiting to be found?
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by lionkor on Thursday May 15, 2025 @03:10AM (#43992564)
No, I'm saying they have a massive database of solutions (the training set) and don't even bother proving that their solution isn't in there. I'm not claiming something, they are failing to provide some necessary information here
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by thatguysaguy on Thursday May 15, 2025 @10:17PM (#44001263)
Many of the problems (such as the matmul and packing problems) have existing "scoreboards". It would be quite surprising if so many records had already been broken without it coming to the attention of the people who care about the problem.
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by lionkor on Friday May 16, 2025 @08:41PM (#44011109)
Not really; people solve problems all the time, rarely do I check it a solution I worked hard on is known by some scoreboard
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by thw09j9m on Wednesday May 14, 2025 @11:09PM (#43991445)
It's Google. Assume the training set contains, as a subset, the entirety of all public digitized information. How would you like to them to share it?
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by kookamamie on Thursday May 15, 2025 @12:33AM (#43991867)
As a tar, please.
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by lionkor on Thursday May 15, 2025 @03:11AM (#43992574)
If they wanna do research where they claim they did something novel, without showing that they didn't just "look it up" in their massive training set, then yes, they should share what is and what isn't contained within.
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by tananaev on Wednesday May 14, 2025 @08:41PM (#43990664)
How can you actually verify it, even if they provide something?
Re:AlphaEvolve: A Gemini-powered coding agent for designing ... by lionkor on Thursday May 15, 2025 @03:11AM (#43992582)
That's my point; you can't. They have no idea if their model came up with any of this or not.