News for hackers. Stuff that matters.
Re:GPT‑6.2 Astra and Sol by simonw on Tuesday November 17, 2026 @01:16PM (#50535742)
Re:GPT‑6.2 Astra and Sol by Hacktrick on Tuesday November 17, 2026 @01:19PM (#50535795)
I'm not sure if this is a useful benchmark any longer
Re:GPT‑6.2 Astra and Sol by InsideOutSanta on Tuesday November 17, 2026 @01:45PM (#50536218)
They're extremely useful. In fact, I refuse to make infrastructure decisions in a pelican-blind environment.
Re:GPT‑6.2 Astra and Sol by p11p on Tuesday November 17, 2026 @01:19PM (#50535791)
Are you getting any signal out of this anymore? All I can see is "good pelican" and "dogshit pelican". These ones look pretty good.
Re:GPT‑6.2 Astra and Sol by EugeneOZ on Tuesday November 17, 2026 @01:22PM (#50535834)
I do. 6.2 Sol xhigh and max show the results comparable to Astra 6 max, and Astra 6.2 only shows any improvement on "max" reasoning effort level.
Re:GPT‑6.2 Astra and Sol by adverbly on Tuesday November 17, 2026 @01:24PM (#50535881)
You're kidding right?

These are some of the worst pelicans I've seen in a while.

Pay attention to the stacking layers near the legs. Sol literally never got it right, and Astra regressed on it multiple times...

Also what is that bending bar from Astra?

Re:GPT‑6.2 Astra and Sol by jay_kyburz on Tuesday November 17, 2026 @01:57PM (#50536405)
When are the Pelicans going to stop being rendered 2d like a child's drawing. We are a million miles away from what a professional artist would come up with.
Re:GPT‑6.2 Astra and Sol by EugeneOZ on Tuesday November 17, 2026 @01:20PM (#50535807)
still the best benchmark! thanks
Re:GPT‑6.2 Astra and Sol by firemelt on Tuesday November 17, 2026 @01:21PM (#50535824)
any index link for all ur pelicans i want to see fable and opus
Re:GPT‑6.2 Astra and Sol by simonw on Tuesday November 17, 2026 @02:03PM (#50536493)
Re:GPT‑6.2 Astra and Sol by tintor on Tuesday November 17, 2026 @01:21PM (#50535828)
Sol 6.2 max is still failing on pelican benchmark. Left leg is jammed in chain.

Astra 6.2 max did a similar mistake on the left bicycle pedal.

Re:GPT‑6.2 Astra and Sol by frangonf on Tuesday November 17, 2026 @01:31PM (#50535996)
But Astra 6.2 low and medium didn't make this mistake.
Re:GPT‑6.2 Astra and Sol by BoldColdHold on Tuesday November 17, 2026 @01:23PM (#50535848)
Wow the yellow is leaking into their llms too.
Re:GPT‑6.2 Astra and Sol by vopi on Tuesday November 17, 2026 @01:23PM (#50535863)
This might be in other pelicans, but 6.2 Sol/Astra is the first time I'm noticing a model shade the leg further in the frame darker to mimic depth (or a shadow).

It's also interesting you can clearly tell 6.2 Astra does a much better job of the above. 6.2 Sol attempts it at higher reasoning levels, but it's more crude.

Re:GPT‑6.2 Astra and Sol by I_Write_It on Tuesday November 17, 2026 @01:24PM (#50535879)
They all seem very good, with a lot of attention in detail ! Thx for the Pelicans!
Re:GPT‑6.2 Astra and Sol by adverbly on Tuesday November 17, 2026 @01:26PM (#50535903)
What's with everyone saying these Pelicans are great?

These are some of the worst that I've seen in a while.

Pay close attention to the stacking layers around the feet, pedals (sometimes there aren't even two pedals), wheel, and two legs. Some pretty obvious mistakes in most if not all of these...

Re:GPT‑6.2 Astra and Sol by svachalek on Tuesday November 17, 2026 @01:41PM (#50536142)
The highest Astra images give the pelicans wings instead of arms, which does seem like a valid interpretation.
Re:GPT‑6.2 Astra and Sol by preommr on Tuesday November 17, 2026 @01:48PM (#50536269)
I have to agree, those results are surprisingly bad and I am not seeing an improvement.

For sol 6.2, some of them are slightly better at higher cost, some of them are way worse and only slightly cheaper (but comparable to terra in quality while being more expensive).

I guess we'll see as more information comes out, but so far not looking good.

Re:GPT‑6.2 Astra and Sol by RGS1811 on Tuesday November 17, 2026 @01:35PM (#50536046)
The emergence of the beach setting and the scarf is interesting.
Re:GPT‑6.2 Astra and Sol by kzrdude on Tuesday November 17, 2026 @02:18PM (#50536714)
I think a scarf or similar is relatively common with bicycle cartoons, to indicate speed or movement.
Re:GPT‑6.2 Astra and Sol by globular-toast on Tuesday November 17, 2026 @01:47PM (#50536241)
Do we know why they always end with a scarf?
Re:GPT‑6.2 Astra and Sol by pinkyclouds2 on Tuesday November 17, 2026 @01:54PM (#50536364)
winter is coming
Re:GPT‑6.2 Astra and Sol by ex-aws-dude on Tuesday November 17, 2026 @01:55PM (#50536378)
Gets cold in new york
Re:GPT‑6.2 Astra and Sol by phoghed on Tuesday November 17, 2026 @02:22PM (#50536776)
They are a classic motif for pilots, birds fly, and there are famous cartoon birds with a scarf, like the ones from The Rescuers or Duck Tales come to mind.
Re:GPT‑6.2 Astra and Sol by ryanschaefer on Tuesday November 17, 2026 @02:11PM (#50536608)
I think this is the first time I’d call the pelican benchmark solved. All levels of effort look good.
Re:GPT‑6.2 Astra and Sol by prmoustache on Tuesday November 17, 2026 @02:56PM (#50537217)
Pelican is one legged on gpt 6.2 sol below xhigh.
Re:GPT‑6.2 Astra and Sol by arcastroe on Tuesday November 17, 2026 @02:58PM (#50537254)
one defect I noticed is that the back leg still bends a different direction than the front leg in many of those images. In both Astra and Sol.

Something a human can recognize as inconsistent, but not the SOTA leading frontier model.

Re:GPT‑6.2 Astra and Sol by boutell on Tuesday November 17, 2026 @02:15PM (#50536668)
Interesting that pelican quality does not go up monotonically within Astra. Honestly, I think low was the best one.
Re:GPT‑6.2 Astra and Sol by glenjamin on Tuesday November 17, 2026 @02:15PM (#50536669)
Interestingly, on the mech thing it seems like only Fable really nailed the brief - the others might look cooler, but Yonkers and the mech aren't directly connected in the same way.
Re:GPT‑6.2 Astra and Sol by boutell on Tuesday November 17, 2026 @02:19PM (#50536731)
Fable gets at the idea in a much more entertaining way than the others.
Re:GPT‑6.2 Astra and Sol by prescriptivist on Tuesday November 17, 2026 @02:52PM (#50537168)
Turning Beech Hill into shoulder mounted cannons is pretty clever.
Re:GPT‑6.2 Astra and Sol by BrokenCogs on Tuesday November 17, 2026 @02:21PM (#50536753)
No idea what a yonkers is but that fable 5.2 animation blows the gpt 6.2 models out of the water
Re:GPT‑6.2 Astra and Sol by dllu on Tuesday November 17, 2026 @02:30PM (#50536878)
Pelicans still not saturated. No chain or front fork offset, and Astra added a rather disconnected detail on top of the pelican's head. Still, we've come a long way since the early days. Very exciting for the future of pelican cyclists!
Re:GPT‑6.2 Astra and Sol by fekunde on Tuesday November 17, 2026 @02:35PM (#50536945)
Wow, surprisingly Gemini 4 Carbon really nailed the Yonkers prompt. I like how the sections of Yonkers held shape in the mech. Fable tried but gemini is more detailed.
Re:GPT‑6.2 Astra and Sol by okrad on Tuesday November 17, 2026 @03:04PM (#50537334)
Looks like high/xhigh both get the back leg z-index incorrect relative to the back wheel of the bike and frame.
Re:GPT‑6.2 Astra and Sol by clzerv on Tuesday November 17, 2026 @03:05PM (#50537361)
The Fable5.2 version is the only one that allows user interaction with the model, it's also very clean and minimal. I've been using Sol-5.1-high for a dumb web project and it tends to create really baroque code and UIs. I haven't used Claude in a while and this makes me want to try it again.

It seems like the models have a kind of built in aesthetic sense and I wonder what part of the training influences this. Maybe if more RL time was spent on artistic tasks the models could not only make better visuals but also code.

Currently they all seem to care little about code quality and maintainability (by humans), just functionality, which is fine I guess if you can expect to vibe code forever. But what if there is a critical bug and the models are temporarily unavailable for some reason? It would be hopeless to expect a human to wade through the giant bowl of clanker soup and fix it anytime soon.