I’m not even sure why those two things would be asked as a single question. It seems like a very unnatural way to pose those two questions. Most humans would trip on that, especially if it was asked verbally.
I'd assume GP only gave an example. As a pretty frequent user, I can unfortunately only confirm that Siri trips over almost every multi-part question.
This would be forgivable if there weren't multiple voice-based AI consumer products available that can handle these kinds of requests perfectly.
If they wanted an LLM answer they could have got one. They went out of their way just to take shots at Apple.
Besides that, many people don’t install any apps, and Apple not pre-installing a reasonable LLM to cater to that market just seems incredibly out of character.
And there’s enough credible reporting and personnel reshuffling happening to suggest that it’s not available yet because they failed to make it work, not because they didn’t try.
(Well, with multiple direct objects, anyway.)
While Siri only does one thing at a time, I trust the answer more, because it’s doing the actual math and not just guessing what the most likely answer is, like an LLM. We need to pick the right tool for the right job. Frankly, I don’t think an LLM is the right tool for conversations like this, and jumbling multiple questions into a single question is something people do with LLMs to get more use out of them during the day, this is an adaptation to a limitation of the free tier (and sometimes speed) of the LLM.
Amazon already reworked Alexa to be backed by a LLM months ago, and they were delayed doing it.
You’re telling me that Apple isn’t capable of the same to Siri?
Amazon just needs a generic LLM. Apple, from the sound of it, is trying to create deep integration with the OS and on-device data. That’s a different problem to solve. They also seem to be trying to do it while respecting user privacy, which is someone most other companies ignore.
I don’t see what the big deal is. I’d rather wait for something good than have them rush out a half-ass “me too” chatbot, that is indistinguishable from the dozens of other chatbots I can simply download as an app for that.
If we believe what Craig Federighi said, they had something, it just wasn’t up to their standards when talking about rolling it out to a billion devices. Which is fair, I run into bad data from ChatGPT and other LLMs all the time. Letting it mature a little more is not a bad thing.
ChatGPT spent a couple months getting my dad pumped up for an elective open heart surgery; he was almost arrogant going into it about how the recovery would go, thinking ChatGPT gave him all the info he could possibly need and a balanced view of reality. Reality hit him pretty hard in the ICU. He sent me some of the chats he had, it was a lot of mutual ego stroking. He was using ChatGPT to downplay the negatives from the doctors and boosting the positives. While it’s good to feel confident, I think it went too far. I spent the whole week in the hospital trying to pull him out of his depression and recalibrating the unrealistic expectations that ChatGPT reinforced. I hope Apple finds a way to be more responsible. If that takes time, great.
Are all Android users using Gemini exclusively? Are all Windows users only using Copilot? Where is the native Linux desktop LLM?
I really don’t understand this criticism. Would it be nice if Siri could do more, sure. Do I have tolerance for Siri to start hallucinating on simple problems it used to use real math for, no. Do I have other options to use in the meantime to get the best of both worlds, absolutely. Where is the hardship?
Much like with the internet, Apple didn't need to re-invent every website to own it all. From Apple platforms a user can access Amazon, Google, or whatever else. Apple didn't create the internet, they sold a gateway to it. AI could be done largely the same way. This way it doesn't matter who wins, Apple can support it. At the end of the day, an LLM doesn't exist on its own, it needs to be accessed through hardware/software people enjoy using, and not be yet another device to charge and carry. Apple has a very popular phone and the most popular wearable. This positions them very well. They are often late to the party, but tend to be best dressed. The first iPhone didn't even have video, and people clowned them for it, and now iPhone video is largely considered one of the best in the smartphone world.
Build a crude router in front of it, if you must, or give it access to "the old Siri" as a tool it can call, and let the LLM decide whether to return its own or a Siri-generated response!
I bet even smaller LLMs would be able to figure out, given a user input and Siri response pair, whether the request was resonably answered or whether the model itself could do better or at least explain that the request is out of capabilities for now.
Both of these approaches were tried internally, including even the ability for the LLM to rewrite siri-as-a-tool's response, and none of them shipped, because they all suck. Putting a router in front of it makes multi-turn conversation (when Siri asks for confirmation or disambiguation) a nightmare to implement, and siri-as-a-tool suffers from the same problem. What happens when legacy siri disambiguates? Does the LLM try to guess at an option? Does it proxy the prompt back to the user? What about all the "smart UI" like having a countdown timer with Siri saying "I'll send this" when sending a text message? Does that just pass through? When does the LLM know how/when to intervene in the responses the Siri tool is giving?
This was all an integration nightmare and it's the main reason why none of it shipped. (Well, that and the LLM being underwhelming and the on-device models not being smart enough in the first place. It was just a slower, buggier siri without any new features.)
The answer is that they need to renege on the entire promise of a "private" siri and admit that the only way they can get the experience they want is a _huge_ LLM running with a _ton_ of user context, in the cloud, and don't hinder it all with backwards compatibility with Siri. Give it a toolbox of things it can do with MCP to your device, bake in the stock tools with LoRA or whatever, and let it figure out the best user experience. If it's a frontier-quality LLM it'll be better than Siri on day one, without Apple having to really do anything other than figure out a good system prompt.
The problem is, Apple doesn't want to admit the whole privacy story is a dead-end, so they're going to keep trying to pursue on-device models, and it's going to continue to be underwhelming and "not meeting our quality bar", for the foreseeable future.
But regarding Apple not wanting to admit that client side compute isn't enough: Haven't they essentially already done that, with Private Cloud Computing and all that? I believe not even proofreading and Safari summarization work fully on-device, at least according to my private compute privacy logs.
Yes, but isn't that infuriating? The technology exits! It even exists, as evidenced by this article, in the same company that provides Siri!
At least I feel that way every time I interact with it – or for that matter my Google Home speaker, ironically made and operated by the company that invented transformer networks.
Err, what? As a native English speaker human that's a pretty confusing question to me, too!
"As of 2022, there were about 400 million native speakers of English. Including people who speak English as a second language, estimates of the total number of Anglophones vary from 1.5 billion to 2 billion."
Second, all popular models I tested did well with that query, including Gemini on Android (aka "ok Google"), except Apple's.
Some modern Apple devices support "local Siri", but it's a limited subset of both voice recognition performance and capabilities.
Probably wouldn't have made a difference but the second half of that statement isn't exactly clear. 85 degrees what?
I also think when you're chaining these two separate calculations together you get a problem when it comes to displaying the results.
So yeah, Apple is way behind on this stuff.
So, it's nice to see Apple is doing research and talking about it, but we're out here waiting, still waiting, for anything useful to make of it all on our thousand-dollar devices that literally connect us to the world and contain our entire life data. It's what I would've expected from one of the most valuable companies in the world.