Re:Apple's AI Strategy in a Nutshell
by ZeroCool2u on Tuesday June 11, 2024 @07:16AM
(#40644896)
I mean TPU's are generally the only off the shelf viable and reliable alternative to Nvidia GPU's for training and there are a lot of former Google staff at Apple that have experience with TPU's, so I wouldn't be surprised.
Re:Apple's AI Strategy in a Nutshell
by msoad on Tuesday June 11, 2024 @06:30AM
(#40644591)
The article mentions that Apple can use OpenAI responses for further training their own models. I don’t want to say that’s impossible but that means they really screwed OpenAI in this deal. OpenAI terms don’t allow training on responses
Re:Apple's AI Strategy in a Nutshell
by elicksaur on Tuesday June 11, 2024 @09:00AM
(#40645720)
>But Apple can also collect data on how users utilize GPT-4o versus Apple’s models, and perform gap assessment. This becomes valuable training data for Apple.
I’d read that as: they know what queries users choose to route to OpenAI, so they can identify where their models are being perceived as less capable.
I don’t think the author is revealing non-public knowledge of the contracts.
Re:Apple's AI Strategy in a Nutshell
by glimshe on Tuesday June 11, 2024 @06:43AM
(#40644679)
Apple's move is interesting, despite being a lot more reserved than this article's fanboy take on the impact of the move. For me, it is at least another clear blow on the "AI is just hype" arguments I still see around here.
Re:Apple's AI Strategy in a Nutshell
by troupo on Tuesday June 11, 2024 @06:51AM
(#40644733)
A yes, a "clear blow". Because we can definitely deduce it from... an over-produced video with sloppy AI-generated images and some heavily edited actions/responses from a product that will not be released until fall (and even then, not with all features).
What other "clear blows" have there been so far that this one is "yet another"?
Re:Apple's AI Strategy in a Nutshell
by ydnaclementine on Tuesday June 11, 2024 @07:00AM
(#40644786)
Am I crazy to think that the claim of private cloud compute running M chips is marketing fluff? Is their ARM based SoC really as performant as a GPU at the scale they need?
Re:Apple's AI Strategy in a Nutshell
by xcv123 on Tuesday June 11, 2024 @07:31AM
(#40644995)
No one said they are M chips. These would be custom made for servers and AI inferencing. Also note their "ARM based SoC" has a GPU with 800 GB/sec bandwidth in the M2 Ultra.
Re:Apple's AI Strategy in a Nutshell
by xcv123 on Tuesday June 11, 2024 @08:17AM
(#40645337)
That guy is just some random blogger and I wouldn't take that to literally mean they use the same chips as a Macbook. Apple only says they use "Apple Silicon". Common sense to assume it is designed for servers and AI inferencing.
Re:Apple's AI Strategy in a Nutshell
by sholladay on Tuesday June 11, 2024 @07:47AM
(#40645105)
Because these days a lot of the money a developer makes isn’t made up-front, it’s made through interactions in the app. Siri isn’t going to read you the ads in the apps it’s using behind the scenes.
Maybe this will cause a shift back towards up-front pricing, which personally I would welcome.
Re:Apple's AI Strategy in a Nutshell
by stanislavb on Tuesday June 11, 2024 @07:17AM
(#40644914)
Yup, this was the second thing I thought that's happening. Also, I wasn't sure whether to upgrade from 13 to 16. However, the AI angle tends everything towards buying a new phone.
Re:Apple's AI Strategy in a Nutshell
by eddyg on Tuesday June 11, 2024 @07:32AM
(#40645005)
The parent comment makes it sound like everybody has to buy new hardware to use AI. Plenty of users with 2-3 year old hardware will be able to use it. As well as all the existing iPhone 15 Pro owners.
Re:Apple's AI Strategy in a Nutshell
by dymk on Tuesday June 11, 2024 @07:35AM
(#40645026)
Apple will be selling iPhone 15s for the next 3 months that won’t get the AI features they just announced. When you buy an iPhone, you generally get software updates giving you new features for a few years at least. What they did with AI is kind of shitty, given their track record.
Re:Apple's AI Strategy in a Nutshell
by beretguy on Tuesday June 11, 2024 @07:58AM
(#40645191)
I mean, I’m glad not to have this AI stuff. I’m glad I have a buffer of iPhones that don’t support this AI when I will be forced to downgrade to a newer device from first SE.
Re:Apple's AI Strategy in a Nutshell
by troupo on Tuesday June 11, 2024 @09:40AM
(#40646063)
I'd like to have a smarter Siri though. Guess I'm stuck with an idiot who can't even understand the word "bedroom" anymore (though it worked fine until a recent update)
Re:Apple's AI Strategy in a Nutshell
by stavros on Tuesday June 11, 2024 @08:49AM
(#40645625)
On the other hand, if your phone can't run a model, it can't run a model, you can't ship people a new AI coprocessor to solder onto their iPhone's motherboard.
Re:Apple's AI Strategy in a Nutshell
by dymk on Tuesday June 11, 2024 @09:40AM
(#40646071)
That’s true, but they’ve made a lot of noise about “private cloud compute” to run models too large for your device. They’re making the odd decision to not offer it as a subscription for devices less than a year old.
They’ve also certainly been working on LLMs for longer than the iPhone 15 and 14 have existed. They knew what it’d support before announcing those devices.
Re:Apple's AI Strategy in a Nutshell
by m0llusk on Tuesday June 11, 2024 @07:37AM
(#40645045)
The larger context is that Apple gets most of its revenue from iPhone sales and use which have seen tepid growth. This strongly indicates reductions in valuation.
Re:Apple's AI Strategy in a Nutshell
by LaserPineapple on Tuesday June 11, 2024 @08:12AM
(#40645294)
"Also, Siri’s agentic features - if they work as advertised - can increase Apple’s leverage over App Publishers, because now the AI - not the user - is the entity opening and clicking on the apps."
This was really interesting to me. How does one develop an app for Siri (or an AI agent in general). Is there a standard way to communicate and expose the functionality of your app?
Re:Apple's AI Strategy in a Nutshell
by IMTDb on Tuesday June 11, 2024 @08:21AM
(#40645371)
> It’s now clear that Apple knows how to train frontier model-quality models, but it’s simply choosing to lay low. Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality
I really don't see where this comes from. Apple has been deploying Siri for a decade. Despite this, Siri is still a steaming pile of cow dung. In few months, OpenAI built a working Siri, something that no-one at Apple was remotely close to achiving.
The fact that Apple signed a deal with OpenAI and includes GPT-4o as an alternative option is a clear sign that Apple server-side models are really not anywhere near GPT-4o. If they were, Apple wouldn't have signed this deal which is so unlike them.
To me, it really looks like Apple is late to the party in terms of LLM. They are betting that within a few years, high quality models will be commoditised and that having an ecosystem that leverages them properly will be the differentiator. Until then they are reluctantly incorporating the market leader in order not mis the train.
Re:Apple's AI Strategy in a Nutshell
by red2awn on Tuesday June 11, 2024 @08:32AM
(#40645460)
The article seems to be based on misinterpreted information:
> Apple’s model has extremely low latency (0.6 milliseconds to first token), outperforms similar sized Phi and Gemini models from Microsoft and Google.
From [1] it is 0.6 millisecond per prompt token so unless the prompt is one token the latency would be higher.
> Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality
The benchmarks from [1] never mentioned GPT-4o, the best model they compared to for the server model is GPT-4-0125. If their model almost matches GPT-4o they wouldn't need to integrate with ChatGPT.
Re:Apple's AI Strategy in a Nutshell
by elicksaur on Wednesday June 12, 2024 @12:04AM
(#40654381)
Oh, I see, I misread. Thought it meant 0.6ms per output token. Now I get that it’s saying “prompt token”, so if your prompt is 100 tokens, that’s 60ms.
That seems pretty fast. 1.6k tokens for a 1s time. Do other models compare to that? I’m not sure what the current top ranking for this metric looks like.
Re:Apple's AI Strategy in a Nutshell
by JSR_FDED on Tuesday June 11, 2024 @09:18AM
(#40645860)
Apple has been laying the foundation for “agentic” use of apps for a long time. All of the functions that apps make available to Shortcuts today will be usable by Apple Intelligence. I wonder if they already had that use case in mind when they came out with Shortcuts?
Re:Apple's AI Strategy in a Nutshell
by troupo on Tuesday June 11, 2024 @09:43AM
(#40646096)
> I wonder if they already had that use case in mind when they came out with Shortcuts?
They didn't come out with shortcuts. They bought an app called Workflow that was leagues ahead of anything Apple was providing on the automation front in iOS.
And Apple never knew what to do with Shortcuts. They bought the app in 2017 and didn't integrate Siri with it until 2022. And Shortcuts still remain limited, and barely usable.
TPU as best as I can see is always google. Either in cloud or their coral devices
I’d read that as: they know what queries users choose to route to OpenAI, so they can identify where their models are being perceived as less capable.
I don’t think the author is revealing non-public knowledge of the contracts.
What other "clear blows" have there been so far that this one is "yet another"?
So it only needs to be so efficient that it will still cost less than those other options.
https://www.wsj.com/tech/ai/apple-is-developing-ai-chips-for...
No real details.
>These data centers will also run completely on Apple’s M chips
Maybe this will cause a shift back towards up-front pricing, which personally I would welcome.
Sell more devices by using privacy and security as an argument for making new features available only on new iPhones.
A very smart move from Apple.
Indeed. Very smart.
They’ve also certainly been working on LLMs for longer than the iPhone 15 and 14 have existed. They knew what it’d support before announcing those devices.
I fixed that for you in the parent. Thanks for pointing it out.
Why does the latter follow from the former? I guess the author has Apple stock.
This was really interesting to me. How does one develop an app for Siri (or an AI agent in general). Is there a standard way to communicate and expose the functionality of your app?
I really don't see where this comes from. Apple has been deploying Siri for a decade. Despite this, Siri is still a steaming pile of cow dung. In few months, OpenAI built a working Siri, something that no-one at Apple was remotely close to achiving.
The fact that Apple signed a deal with OpenAI and includes GPT-4o as an alternative option is a clear sign that Apple server-side models are really not anywhere near GPT-4o. If they were, Apple wouldn't have signed this deal which is so unlike them.
To me, it really looks like Apple is late to the party in terms of LLM. They are betting that within a few years, high quality models will be commoditised and that having an ecosystem that leverages them properly will be the differentiator. Until then they are reluctantly incorporating the market leader in order not mis the train.
Probably due to internal political wars. Apparently they had an internal lightweight version outperforming existing Siri, but the team never got anywhere with it: https://daringfireball.net/linked/2024/06/06/how-the-wall-st...
> Apple’s model has extremely low latency (0.6 milliseconds to first token), outperforms similar sized Phi and Gemini models from Microsoft and Google.
From [1] it is 0.6 millisecond per prompt token so unless the prompt is one token the latency would be higher.
> Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality
The benchmarks from [1] never mentioned GPT-4o, the best model they compared to for the server model is GPT-4-0125. If their model almost matches GPT-4o they wouldn't need to integrate with ChatGPT.
[1]: https://machinelearning.apple.com/research/introducing-apple...
That seems pretty fast. 1.6k tokens for a 1s time. Do other models compare to that? I’m not sure what the current top ranking for this metric looks like.
They didn't come out with shortcuts. They bought an app called Workflow that was leagues ahead of anything Apple was providing on the automation front in iOS.
And Apple never knew what to do with Shortcuts. They bought the app in 2017 and didn't integrate Siri with it until 2022. And Shortcuts still remain limited, and barely usable.