News for hackers. Stuff that matters.

A comment on FunSearch: Making new discoveries in mathematical sciences using LLMs. Parent comment

Re:FunSearch: Making new discoveries in mathematical science... by Imnimo on Thursday December 14, 2023 @04:06PM (#38647240)
Critically, though, the LLM is not acting as a domain expert here. It's acting as a code mutator. The expertise it brings to the table is not mathematical - Codey doesn't have any special insight into bin packing heuristics. It's not generating "suggestive hints and promising leads", it's just help the evolutionary search avoid nonsense code mutations.
[ Parent | on HN ]
Re:FunSearch: Making new discoveries in mathematical science... by nopinsight on Thursday December 14, 2023 @04:28PM (#38647507)
An LLM serves as a sort of “expert” programmer here. Programming itself is a pretty complex domain in which a purely evolutionary algorithm isn’t efficient.

We can imagine LLMs trained in mathematical programming or a different domain playing the same role more effectively in this framework.

[ Parent | on HN ]
Re:FunSearch: Making new discoveries in mathematical science... by Imnimo on Thursday December 14, 2023 @04:48PM (#38647764)
I disagree that the type of coding the LLM is doing here is "expert" level in any meaningful sense. Look for example at the code for the bin-packing heuristics:

https://github.com/google-deepmind/funsearch/blob/main/bin_p...

These aren't complex or insightful programs - they're pretty short simple functions, of the sort you typically get from program evolution. The LLM's role here is just proposing edits, not leveraging specialized knowledge or even really exercising the limits of existing LLMs' coding capabilities.

[ Parent | on HN ]
Re:FunSearch: Making new discoveries in mathematical science... by nopinsight on Thursday December 14, 2023 @04:56PM (#38647866)
To be clearer, the word “expert” in my comment above is in quotation marks because an LLM has more programming expertise than a non-programmer or an evolutionary algorithm, but not anywhere near a true expert programmer.
[ Parent | on HN ]