Re:AI Is Writing Its Own Kernels, and They Are 17x Faster
by pos456 on Thursday November 20, 2025 @05:30PM
(#45998762)
Calling beam search 'AI' is doing a lot of heavy lifting here. This is just superoptimization with a very expensive heuristic function.
[ Parent
| on HN ]
Re:AI Is Writing Its Own Kernels, and They Are 17x Faster
by jryio on Thursday November 20, 2025 @07:03PM
(#45999625)
That's correct - however as other commenters have noted. Doing this by hand is extremely challenging for human engineers working on tensor kernels.
The expense calculation might be
expense of improvement = (time taken per optimization step * cost of unit time ) / ( speedup - 1)
The expensive heuristic function is saving wall time well also being cheaper in cost of unit time. And as the paper shows the speed up provided for each unit time multiplied by unit cost of time is large.
Re:AI Is Writing Its Own Kernels, and They Are 17x Faster
by greeravoctado on Thursday November 20, 2025 @07:29PM
(#45999829)
Usually the rate of overall improvement for this type of optimization is less than Moore law rate of improvement, thus not worth the company investment. 17x micro-benchmarks don't count. Real improvements come from architectural changes, for example: MoE, speculative multi-token prediction, etc.