News for hackers. Stuff that matters.
Re:The Coming of Local LLMs by BulgarianIdiot on Tuesday April 11, 2023 @03:40PM (#35530377)
The first step is for it to be viable for smaller models on desktops. The rest will follow, as hardware catches up. Hybrid analog-hybrid NN hardware is on the horizon, maybe in 3-4 years. This would allow GPT-4 level performance on an iPhone with plausible battery life.

The current hardware of course can't pull anything like this yet. But iPhone supports on-device facial recognition, object recognition, dictation and translation, so small steps...

Re:The Coming of Local LLMs by boppo1 on Wednesday April 12, 2023 @12:44AM (#35535113)
Who is making said hardware?
Re:The Coming of Local LLMs by BulgarianIdiot on Thursday April 13, 2023 @09:42AM (#35555315)