Re:The last six months in LLMs, illustrated by pelicans on b...
by diggan on Sunday June 08, 2025 @11:54AM
(#44217725)
> The number of subject-verb-objects are near infinite. All are imaginable, but most are not plausible
Until there is enough unique/new subject-verb-objects examples/benchmarks so the trained model actually generalized it just like you did. (Public) Benchmarks needs to constantly evolve, otherwise they stop being useful.
Re:The last six months in LLMs, illustrated by pelicans on b...
by demosthanos on Sunday June 08, 2025 @12:57PM
(#44218087)
To be fair, once it does generalize the pattern then the benchmark is actually measuring something useful for deciding if the model will be able to product a subject-verb-object SVG.
Re:The last six months in LLMs, illustrated by pelicans on b...
by zahlman on Sunday June 08, 2025 @01:10PM
(#44218176)
I can't fathom this working, simply because building a model that relates the word "ride" to "hump" seems like something that would be orders of magnitude easier for an LLM than visualizing the result of SVG rendering.
The number of subject-verb-objects are near infinite. All are imaginable, but most are not plausible. A plausibility machine (LLM) will struggle with the implausible, until it can abstract well.