News for hackers. Stuff that matters.
Re:GPT-6 Astra on OpenRouter by embedding-shape on Friday September 04, 2026 @07:14PM (#49571293)
At the bottom it says "Score 98.58", what measure is used for this score? It's kind of horrible, the perspective is all off (legs of the table makes that very obvious), the mouse/hamster has two mouths, a stub for a right paw, looks like left hand holds a melon on a stick or something, and there are pluses in the background for some reason. Not sure it'd call it "close to perfect" which the score seems to want to indicate.
Re:GPT-6 Astra on OpenRouter by XCSme on Friday September 04, 2026 @07:25PM (#49571364)
Do you prefer the fable one?

It's more "correct" but looks a lot worse in my opinion:

https://aibenchy.com/compare/openai-gpt-6-astra-high/google-...

Re:GPT-6 Astra on OpenRouter by kulahan on Friday September 04, 2026 @08:07PM (#49571659)
No, both of them look awful. I genuinely am unsure if this is a meme you're making that's going over my head...?
Re:GPT-6 Astra on OpenRouter by XCSme on Friday September 04, 2026 @08:12PM (#49571692)
Re:GPT-6 Astra on OpenRouter by kulahan on Saturday September 05, 2026 @02:43AM (#49573758)
Ah, okay, that makes a lot more sense! Thanks!
Re:GPT-6 Astra on OpenRouter by XCSme on Friday September 04, 2026 @07:23PM (#49571355)
Yeah, that's confusing, the score is for the entire benchmark, not for SVG generation only.

Good point about the mouths, I just noticed, lol

Imo, it's still better than most models, I personally like the stylized perspective.

You can view here all generations for all models: https://aibenchy.com/showcase/

Re:GPT-6 Astra on OpenRouter by XCSme on Friday September 04, 2026 @07:49PM (#49571530)
I've replaced "Score" there with model ranking, to reduce confusion, thanks for the feedback!
Re:GPT-6 Astra on OpenRouter by embedding-shape on Saturday September 05, 2026 @05:25AM (#49574787)
Now I'm wondering why it's ranked #4 instead, not sure this reduced confusion :P

What panel of judges are you using for scoring/ranking this? Seems subjective enough to not be able to be ranked/scored at all

Re:GPT-6 Astra on OpenRouter by XCSme on Saturday September 05, 2026 @05:57AM (#49574955)
Yeah, ranking is for entire model, not only SVG generation. Maybe I remove it entirely and add generation date instead.

I was thinking to manually grade/rank the SVGs, but I decided against it, as it is indeed subjective.

I was thinking it could have at least a simple objective check (hamster doesn't have extra or missing parts, table has 2 sides, and net is in the middle, etc.).

Re:GPT-6 Astra on OpenRouter by kzrdude on Saturday September 05, 2026 @04:56AM (#49574643)
And not to mention, the hamster is standing at the wrong end of the table.
Re:GPT-6 Astra on OpenRouter by embedding-shape on Saturday September 05, 2026 @05:26AM (#49574797)
Incredible, I've played table-tennis since I was like 14, and I didn't notice that egregious error yet I noticed all the other small ones! Thanks for pointing that out, should have been very obvious.
Re:GPT-6 Astra on OpenRouter by sehugg on Saturday September 05, 2026 @05:47AM (#49574892)
And also the hamster has two mouths
Re:GPT-6 Astra on OpenRouter by embedding-shape on Saturday September 05, 2026 @09:02AM (#49576130)
I think it might be standing at the wrong edge of the table too, worth adding.