Re:Apple introduces M4 chip
by strangescript on Tuesday May 07, 2024 @03:26PM
(#40290404)
The fundamental problem with this strategy is model size. I want all my apps to be privacy first with local models, but there is no way they can share models in any kind of coherent way. Especially when good apps are going to fine tune their models. Every app is going to be 3GB+
[ Parent
| on HN ]
Foundation models will be the new .so files.
And fine tuning datasets will be compressed and sent rather than the whole model
Re:Apple introduces M4 chip
by strangescript on Wednesday May 08, 2024 @10:40AM
(#40298626)
This would be interesting but also feels a little restrictive. Maybe something like LoRa could bridge the capability gap but if a competitor then drops a much more capable model then you either have to ignore it or bring it into your app.
(assuming companies won't easily share all their models for this kind of effort)
Re:Apple introduces M4 chip
by SpaceManNabs on Tuesday May 07, 2024 @06:52PM
(#40292454)
I don't think HN understands how important model distillation still is for federated learning. Hype >> substance ITT
You could always mix and match. Do lighter task on device and outsource to cloud if needed
Gemini nano is 1.8B 4 bit parameters, so a little under a GB. And hopefully each app won’t include a full copy of their models.