I think the human brain uses something on the order of 10-20W so 13.5W seems like there's probably some room to improve on a 4b model!

423. Could y'all please start prefixing comments with the number of joules you consumed in generating it, so that I can decide if it's worth reading?

[delayed]

Eleventy billion:watch snl celebrity jeopardy.

^what every advertiser will do if we actually do this lol

Statements like these often overlook that a human brain is pretty slow for a lot of tasks. It's hard to make direct comparisons, but for language (related cognitive work) we're at maybe a few tokens per second.

I wonder how even a team of humans (a meat mixture of experts) would perform on the AI benchmarks, measured in energy use per intelligence index/benchmark point.

[delayed]

Back in July, I timed myself manually looking at some data files to create an index out of them, and out of curiosity asked some AI to do the same.

It took me an hour (61 minutes) and I made 4 mistakes in about 80 entries. It took DeepSeek-V4-Flash 3 minutes, it cost $0.034, and only made 2 mistakes. There is no contest. Humans have lost.

Uh, yeah, humans lost at data processing about halfway through the last century. The calculator can calculate fast than you. Great that you've finally discovered what computers can do, but it doesn't mean what you seem to think it means.

These days I like to think of myself as a 100T dense model with 20 t/s prefill and 2 t/s decode. I am an RL(HF/VR/CR) policy rollout at this point.

I know some people that would beg to differ. They’re at least 7B class.

[delayed]

Not really. Models of that size is reasonably often moe. It is also 100b even if it only has 20b active at a time.

13.5W is Jetson nano territory.