News for hackers. Stuff that matters.
Re:Stanford Alpaca, and the acceleration of on-device LLM de... by mikek on Monday March 13, 2023 @04:52PM (#35142397)
Apple comes to mind.
Re:Stanford Alpaca, and the acceleration of on-device LLM de... by BryantD on Monday March 13, 2023 @05:49PM (#35143357)
Agreed. Apple will run models remotely if necessary, but from a PR perspective they align with their stated intentions when they can run locally.
Re:Stanford Alpaca, and the acceleration of on-device LLM de... by mustacheemperor on Monday March 13, 2023 @08:27PM (#35145441)
Oh, how I hope there is a project ongoing at Apple right now to implement some kind of on-device LLM that can drive the phone a la Siri. Any time I open the Bing app on my iPhone to answer a question I think of how useful it could be to ask it to update my calendar, fill out a dinner order, etc. How much time do we collectively waste putzing around with delivery apps etc?
Re:Stanford Alpaca, and the acceleration of on-device LLM de... by smoldesu on Monday March 13, 2023 @08:48PM (#35145671)
It would be neat, but iPhone hardware has limited memory (and iOS limits that usage further). It would be much faster and higher-quality to run it as an API request like it is today.
Re:Stanford Alpaca, and the acceleration of on-device LLM de... by yieldcrv on Tuesday March 14, 2023 @02:33AM (#35148078)
Given the incentive, it can push the hardware in this direction.

Or they’ll do a home network device like implement it like a HomeKit hub. So the plugged in resource has more computational power and feeds it to iphones when nearby. Disconnected, the iphone uses a much more rudimentary one or falls back to Siri or a cloud service.

Re:Stanford Alpaca, and the acceleration of on-device LLM de... by gpderetta on Tuesday March 14, 2023 @04:40AM (#35148764)
Your phone can remotely connect to your home device when needed.
Re:Stanford Alpaca, and the acceleration of on-device LLM de... by smoldesu on Tuesday March 14, 2023 @10:29AM (#35152100)
Maybe it will use lasers to paint a smiley face on my wall!

Only two things are for certain; you'll have to pay for it, and it will not be cheap.

Re:Stanford Alpaca, and the acceleration of on-device LLM de... by gpderetta on Tuesday March 14, 2023 @04:39AM (#35148760)
Apple Siri as a device you can run on your home as a digital assistant.