News for hackers. Stuff that matters.

A comment on iPhone 17 Pro Demonstrated Running a 400B LLM. Parent comment

Re:iPhone 17 Pro Demonstrated Running a 400B LLM by alpineman on Tuesday March 24, 2026 @04:36AM (#47499970)
Agree with the first part - but I can run GPT OSS 20b, a highly capable model on my laptop with 32GB of RAM at speeds that for all practical intents is as fast as GPT-5.4 and good enough for 90%+ of non-technical use cases.

As such I can't agree with "The only hope for a handheld execution of a practical, and capable AI model is both an algorithmic breakthrough" - we are much closer than 15/20 years to get these on a phone

[ Parent | on HN ]
Re:iPhone 17 Pro Demonstrated Running a 400B LLM by zozbot234 on Tuesday March 24, 2026 @05:28AM (#47500283)
With this work you can run a medium-sized model like GPT OSS 20b at native speed even while keeping those 32GB RAM almost fully available for other uses - the model seamlessly starts to slow down as RAM requirements increase elsewhere in the system and the fs cache has to evict more expert layers, and reaches full speed again as the RAM is freed up. It adds a key measure of flexibility to the existing AI local inference picture.
[ Parent | on HN ]