News for hackers. Stuff that matters.
Re:A look at Apple's technical approach to AI including core... by FL33TW00D on Friday June 14, 2024 @04:45AM (#40678900)
I described this system word for word 2 years ago, glad to see it come to fruition on the only software stack integrated enough to do it.

https://fleetwood.dev/posts/a-case-for-client-side-machine-l...

Re:A look at Apple's technical approach to AI including core... by aitooltime on Friday June 14, 2024 @06:03AM (#40679303)
Thanks for posting the article.

Some predictions in the article:

1. Partial client side execution of the models, then main model execution in cloud. (Example: text encoder for Stable Diffusion runs local, UNet runs cloud)

2. Local LoRAs for “dynamic ML”

3. WebGPU will be new standard to unlock models in the web

Even though the article had some good calculations and overview, I am not sure word-for-word is how I would describe its relationship to the OP.

Re:A look at Apple's technical approach to AI including core... by jimmySixDOF on Friday June 14, 2024 @04:59AM (#40678985)
If you described Speculative Decoding, 3bit Quantization, and Adapter based weight selection all two years ago then your AI Lab must be so sota /s
Re:A look at Apple's technical approach to AI including core... by reqo on Friday June 14, 2024 @05:14AM (#40679058)
I am curious, what do you mean by "Adapter based weight selection"?
Re:A look at Apple's technical approach to AI including core... by supriyo-biswas on Friday June 14, 2024 @05:36AM (#40679166)
That's apparently Apple's term for LoRA[1].

[1] https://arxiv.org/abs/2106.09685

Re:A look at Apple's technical approach to AI including core... by jimmySixDOF on Friday June 14, 2024 @08:18AM (#40680168)
Sort of but Adapters allow for multiple weight adjustments (think loras) for specific skills so it is more like extra optimized mixture of experts or multi agent approach. They have a slide with adapters listed like summarization, prioritization, tone (happy, business, etc), editor, etc) -- this is not to be mixed up with Intents which is how on device apps publish their capabilities to the Intelligence system for real npu os level multi agent tool use.