Re:Stanford Alpaca, and the acceleration of on-device LLM de...
by mikek on Monday March 13, 2023 @04:52PM
(#35142397)
Apple comes to mind.
[ Parent
| on HN ]
Re:Stanford Alpaca, and the acceleration of on-device LLM de...
by BryantD on Monday March 13, 2023 @05:49PM
(#35143357)
Agreed. Apple will run models remotely if necessary, but from a PR perspective they align with their stated intentions when they can run locally.
Re:Stanford Alpaca, and the acceleration of on-device LLM de...
by mustacheemperor on Monday March 13, 2023 @08:27PM
(#35145441)
Oh, how I hope there is a project ongoing at Apple right now to implement some kind of on-device LLM that can drive the phone a la Siri. Any time I open the Bing app on my iPhone to answer a question I think of how useful it could be to ask it to update my calendar, fill out a dinner order, etc. How much time do we collectively waste putzing around with delivery apps etc?
Re:Stanford Alpaca, and the acceleration of on-device LLM de...
by smoldesu on Monday March 13, 2023 @08:48PM
(#35145671)
It would be neat, but iPhone hardware has limited memory (and iOS limits that usage further). It would be much faster and higher-quality to run it as an API request like it is today.
Re:Stanford Alpaca, and the acceleration of on-device LLM de...
by yieldcrv on Tuesday March 14, 2023 @02:33AM
(#35148078)
Given the incentive, it can push the hardware in this direction.
Or they’ll do a home network device like implement it like a HomeKit hub. So the plugged in resource has more computational power and feeds it to iphones when nearby. Disconnected, the iphone uses a much more rudimentary one or falls back to Siri or a cloud service.
Re:Stanford Alpaca, and the acceleration of on-device LLM de...
by gpderetta on Tuesday March 14, 2023 @04:40AM
(#35148764)
Your phone can remotely connect to your home device when needed.