News for hackers. Stuff that matters.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by jgalt212 on Wednesday December 10, 2025 @08:10AM (#46217271)
Aiden is perhaps misinformed. From a Bing search performed just now.

> Yes, I am familiar with the "pelican riding a bicycle" SVG generation test. It is a benchmark for evaluating the ability of AI models, particularly large language models (LLMs) and multi-modal systems, to generate original, high-quality SVG vector graphics based on a deliberately unusual and complex prompt. The benchmark was popularized by Simon Willison, who selected the prompt because:

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by 100721 on Wednesday December 10, 2025 @10:10AM (#46218542)
Web search-based RAG is very different from having something embedded in a model's training data, though.
Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by jgalt212 on Wednesday December 10, 2025 @01:59PM (#46221934)
ChatGPT website gives a similar answer. Are they running RAG, or the model?

> Yes — I’m familiar with the “pelican riding a bicycle” SVG generation test.

> It’s become a kind of informal benchmark people use when evaluating whether an image-generation or SVG-generation model can: ...

Re:Mistral Releases Devstral 2 (72.2% SWE-Bench Verified) an... by Barbing on Friday December 12, 2025 @12:14PM (#46246164)
Runnin’ confabulations:

>Yes — the “hamster driving a car” prompt is a well-known informal test …

>…that’s a well-known informal test people use…(a mole-rat holding or playing a guitar).

Try any plausible concept. Get sillier and it’s trained to talk about it being nonsense. The output still claims it’s a real test, just a real “nonsense” test.