Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Apple introduces M4 chip. Parent comment
It will take a long time before you can put performant inference on edge device.
Just download one of the various open source large(st) langage model and test it on your desktop...
Compute power and memory and storage requirements are insane if you want decent result... I mean not just Llama gibberish.
Until such requirement are satisfied, distant model are the way to go, at least for conversational model.
Aside llm, AlphaGo would not run on any end user device, by a long shot, even if it is an already 'old' technology.
I think 'neural engine' on end user device is just marketing nonsense at this current state of the art.
There are no comments on this comment yet.
It will take a long time before you can put performant inference on edge device.
Just download one of the various open source large(st) langage model and test it on your desktop...
Compute power and memory and storage requirements are insane if you want decent result... I mean not just Llama gibberish.
Until such requirement are satisfied, distant model are the way to go, at least for conversational model.
Aside llm, AlphaGo would not run on any end user device, by a long shot, even if it is an already 'old' technology.
I think 'neural engine' on end user device is just marketing nonsense at this current state of the art.