News for hackers. Stuff that matters.

A comment on Llama 2 – Meta AI. Parent comment

Re:Llama 2 – Meta AI by smoldesu on Tuesday July 18, 2023 @01:50PM (#36776573)
If they want to own the whole stack, I don't think they have much to work with. Their highest-end server chip is a duplex laptop SOC, with maxed-out memory that doesn't even match the lowest-end Grace CPU you can buy (nevermind a fully-networked GH200). Their consumer offerings are competitive, but I don't think Apple Silicon or CoreML is ready to seriously compete with Grace and CUDA.
[ Parent | on HN ]
Re:Llama 2 – Meta AI by samwillis on Tuesday July 18, 2023 @02:01PM (#36776746)
While Apple silicone may not be there for training, I think it's probably there for inference. I expect next years device models to launch with exclusive support for Apples own LLM based Siri.
[ Parent | on HN ]
Re:Llama 2 – Meta AI by smoldesu on Tuesday July 18, 2023 @02:14PM (#36776938)
Sure. Haswell CPUs from 2014 are "there" for inference if they have AVX support and 8gb of RAM. Inferencing isn't the problem though, not on M1 or Macbooks from 2016. Scaling a desirable (and hopefully open) GPGPU programming interface is. This is bottlenecked by both hardware and software decisions Apple has made, making a "home grown" competitive model much more unlikely in my eyes.

I agree that there is an incentive to put AI models on your OS. I just don't think Apple can own the whole stack if they want to play ball right now.

[ Parent | on HN ]