georgeven

This is my recent PhD work. I want to show that AI can be used for good, rather than just optimizing ads and keeping people on TikTok for longer. Modern VLMs (LLMs with vision capabilities) are just good enough for labeling that they can replace route-label work for bioacoustics, and I find that distilling these labels into a bioacoustic encoder -- such as SongMAE, my previous work -- can work fairly well. Interestingly, the student exceeds the teacher model, and doesnt asymptote at teacher performance. This work was actually roughly inspired by Simon Willison (pelican guy), he made a post that showed Qwen 27B is an effective labeler for images.

ARC-AGI-3 is actually about action efficiency not solve rate, current LLMs just make a bunch of moves that inefficient, thereby lowering their score

What are you building if you dont mind sharing? I am moving and all these online applications and such are so annoying.

No! because they did not release GPT-2 XL until analogs appeared a year later. So the SOTA model during GPT-2 days was closed weights.

PCIE 16 gig is <300 dollars these days a piece

on eBay you can buy 4 of them for 1000. PCIE 16 gigs.

I have a 1500 dollar machine that can run it at 50 tok/s (3 V100s)

innovation requires monopoly is a Theil one liner -- probably true too

I wonder if you can sue him in civic court, could teach him a lesson.

The issue then becomes it writes insane one liners that are impossible to read. Would be difficult to set up an RL environment where the goal is correctness and readability.

the codex limits are actually pretty high too. You might want to check it out.

I have known about TBPN since early summer last year. They are widely known about in Cali startup culture at least.

its probably not even a conscious decision from dwarkesh to be hyperbolic. pod casters who are hyperbolic are just simply watched more

he was not sure, but if i recall correctly, he put the probability at something like 90 percent of being able to do non verifiable tasks.

it's 20,000 in 2026, with the price of tokens halving every year (at a given perf level), this will be around 1,000 dollars in 2030

Interesting. Everyone in my circle said the opposite.

It was actually nearly 5 years ago!

dude you are spinning up my models! Emailing you!

I found codex cli to be significantly better than claude code. It follows instructions and executes the exact change I want without going off on an "adventure" like Claude code. Also the 20 dollars per month sub tier gives very generous limits of the most powerful model option (5.2 codex high).

I work on SSL bio acoustic models as context.