News for hackers. Stuff that matters.
Re:Qwen4-Max by powersnail on Monday November 16, 2026 @08:03AM (#50517177)
Look at the page, though. If the data is in their training set, then the training must not have been very good, cause the images have very salient defects, and random artifacts popping up.

As far as I've seen, among the popular models today, there are still significant gaps in the quality of bike-riding pelikan svgs, making the benchmark still useful. Maybe not as a differentiator of frontier models, but certainly still informative for lesser competitors.

Re:Qwen4-Max by paimapi on Monday November 16, 2026 @09:58AM (#50518284)
Unless the website is pulling hundreds of thousands of views per pelican, I sort of doubt any firm is actually training or post-training to optimize for this benchmark. They'll more generally optimize for better SVG rendering and I imagine these pelicans arose out of the annoyance we all had many months ago with how utterly bumbling and forgetful models used to be when completing tasks like that.