News for hackers. Stuff that matters.
Re:Apple wants AI to run directly on its hardware instead of... by tech_ken on Thursday December 21, 2023 @02:37PM (#38725641)
> “As those things get scaled up, they start to make a meaningful change in the user experience and enable new innovation which has the potential to create a new upgrade cycle in smartphones.”

Interesting read. "Create a new upgrade cycle" is a phrase that's almost too on the nose, but will be interesting to see if the AI-powered feature set genuinely justifies an upgrade. I have a hard time imagining someone wanting to shell out 4 digits to be able to run ex. stable diffusion locally if they weren't already in the market for a new phone, but maybe I'm underestimating the AI hype wave right now. I could see predictive typing or improved voice assistants being the two biggest opportunities for a killer feature, but even then not sure if the value proposition would change that much, especially if the battery life takes any hit whatsoever.

Re:Apple wants AI to run directly on its hardware instead of... by msoad on Thursday December 21, 2023 @02:53PM (#38725887)
Siri that actually works sounds very appealing to be honest. All the integration and UX is there. It just doesn’t work today. So an LLM based Siri that can understand my request is worth an upgrade (for me)
Re:Apple wants AI to run directly on its hardware instead of... by rollcat on Thursday December 21, 2023 @03:11PM (#38726168)
On the contrary, Siri already works perfectly (or rather, used to work) for a lot of things... It's just Apple keeps on randomly changing stuff, and every time they change something, shit breaks.

"Hey Siri, good evening" - "I turned on the `good morning` scene."

"Hey Siri, timer five minutes" - "I set an alarm for seventeen hundred hours"

"Hey Siri, weather" - turns on the lights in the bathroom

I wish I was joking or exaggerating... The last thing I need is an LLM, just please fix the obvious problems and then maybe iterate on that

Re:Apple wants AI to run directly on its hardware instead of... by amluto on Thursday December 21, 2023 @05:09PM (#38727754)
> “Hey Siri, timer five minutes" - "I set an alarm for seventeen hundred hours”

I think I figured out the issue with this one, finally: Siri understands “Set a timer for 6:30” as a request for a timer for however far in the future 6:30 is. And it also understands “set a timer for 6” similarly.

And Siri doesn’t seem at all intelligent about when to stop transcribing.

So Siri has a rather large chance of interpreting “set a timer for 5 minutes” as “set a timer for 5” and ending up with utter nonsense.

A LLM that jointly handles language and audio might do much better.

Re:Apple wants AI to run directly on its hardware instead of... by n9 on Thursday December 21, 2023 @05:47PM (#38728179)
Something's not right. I say "hey siri 10 minute timer" or somesuch like five times a day. I'd play with your timing and word order. For instance my kid has trouble making music work with siri in the car, but if I say "hey siri play the album technique by the band new order" just like that it works 100% of the time.
Re:Apple wants AI to run directly on its hardware instead of... by oidar on Thursday December 21, 2023 @04:06PM (#38726957)
Additionally, in a multi-person household adding things to a shopping list should take any authentication. Not everybody has an iDevice or needs to have an iDevice - my petsitter, babysitter, maid and random family members should be able to add stuff to the shopping list just because they are in the room with the homePod. On Alexa, it works - but we have to deal with all the advertisements for crappy AliExpress junk. As soon as Apple figures out shopping lists for whoever is in the room, I'll be dropping all my Alexa devices.
Re:Apple wants AI to run directly on its hardware instead of... by VogonPoetry on Thursday December 21, 2023 @07:59PM (#38729515)
Not sure exactly which way round you are asking -- should requests be tied to the person or be generic?

In the HomePod settings there is a section for "Personal Requests". When enabled, adding reminders will then go to the device / account of the person speaking. I'm not sure what the behavior is when set to off. You can change the setting for each HomePod.

Re:Apple wants AI to run directly on its hardware instead of... by oidar on Thursday December 21, 2023 @09:38PM (#38730281)
In an ideal world, anyone who can access the HomePod physically _should_ have the ability to contribute items to the shopping list, regardless of whether that list is being shared among multiple users or not. At present, in order to add items to a shared "shopping list" in Reminders, each individual must authenticate themselves through voice recognition on their personal iOS device. It's not possible to add items to the "shopping list" without this initial authentication. Additionally, enabling personal requests is a prerequisite for adding items to a reminders list on HomePod devices. Even when it's set up correctly, it takes nearly a 20 second lag to add things to a list. With Alexa, it's instant without the need for authentication.
Re:Apple wants AI to run directly on its hardware instead of... by ip26 on Thursday December 21, 2023 @04:14PM (#38727036)
It works great if you know the exact incantation, but the discoverability of those incantations is terrible, and that's the exact problem.
Re:Apple wants AI to run directly on its hardware instead of... by rollcat on Thursday December 21, 2023 @05:36PM (#38728087)
I spent the past 30+ years typing exact incantations into a computer. I give Siri an exact incantation today and it works, I give Siri the same exact incantation tomorrow and it breaks.
Re:Apple wants AI to run directly on its hardware instead of... by ceejayoz on Thursday December 21, 2023 @03:30PM (#38726443)
Yeah, I'll get random weeks in which "text <wife>" works fine, and other weeks where it says "which one?" and gives me her email and phone as different contacts. No changes on my end that'd cause it.
Re:Apple wants AI to run directly on its hardware instead of... by raccoonDivider on Thursday December 21, 2023 @04:04PM (#38726936)
Makes me wonder how voice assistants handle people with multiple partners. Maybe "which one" is actually the right answer for some users?
Re:Apple wants AI to run directly on its hardware instead of... by kstrauser on Thursday December 21, 2023 @04:02PM (#38726907)
My wife is "Jen". I used to work with "Joe Chan". To this day, if I say "text Jen ...", there's about a 20% chance it'll reply with "OK, saying ... to Joe Chan."

Dude. How about optimizing for the option that I've actually communicated with this decade, and 4 times already today?

Re:Apple wants AI to run directly on its hardware instead of... by merpnderp on Thursday December 21, 2023 @02:56PM (#38725936)
Exactly, just figure out how to run Whisper on my phone and it would be the best phone on the market. Whisper plus Mixtral on my phone and it could do anything.
Re:Apple wants AI to run directly on its hardware instead of... by n9 on Thursday December 21, 2023 @05:45PM (#38728158)
Not sure what you're needing that you're note getting -- there is certainly a lot that it doesn't do -- but Siri works like a champ for me these days for maybe 70% of the things that I would do with my phone or ever computer without it. I initiate all calls with it, do tons of texting with it, cue up all my music in the car with it, constantly add things to reminders lists ('hey siri... add soy sauce to the grocery list' results in soy sauce on my grocery list... in the condiments sublist)

I hadn't noticed the improvements until I started using my apple watch a lot... I had given up on it a few years back, I guess.

Re:Apple wants AI to run directly on its hardware instead of... by hx8 on Thursday December 21, 2023 @03:14PM (#38726205)
There aren't many more single new features that are going to justify upgrading that will exist on the mobile platform. Apple is playing a game where they try to add enough features to prevent the upgrade cycle from becoming too long. Encourage people with 2 or 3 year old phones to upgrade this cycle instead of delaying a generation.

Honestly, Apple's business model is harder if everyone wants to upgrade to a certain new version. The M1 Macs were so good, everyone upgraded their Intel Macs and then mac sales dropped. It's better logistically if the sales are more predictable and consistent year-to-year.

Re:Apple wants AI to run directly on its hardware instead of... by tech_ken on Thursday December 21, 2023 @03:31PM (#38726449)
That's an interesting point, so more of a game of amortizing upgrades over a few years rather than instigating a massive one-off purchasing craze. I could see how a drip of sufficiently useful AI features and improvements over like '24-'27 would bolster the market. Even then though, this seems like an opportunity to hold rather than really reverse the trend; at some point I think Apple is going to have to reckon with the fact that the market is more or less saturated, though I could definitely be wrong here.
Re:Apple wants AI to run directly on its hardware instead of... by hx8 on Thursday December 21, 2023 @04:22PM (#38727149)
Apple knows the cell phone market is saturated. That's why they have iPads, AirPods, Apple Watches, MacBook Airs, etc. Their plan is to create new accessories (and services) that pair with the iPhone. This generates a moat around existing iPhone users.
Re:Apple wants AI to run directly on its hardware instead of... by tech_ken on Thursday December 21, 2023 @04:31PM (#38727263)
I guess if they start layering LLMs or similar into their service or accessory interop stack I could see how having the phone running the model could be a big win for latency and availability. Like if Apple Calendar uses a GPT to create a .ics from the invitation text my friend sent me then it could be very handy to do that locally, moreso if I'm using my Apple Watch to actually signal that I want the .ics created.
Re:Apple wants AI to run directly on its hardware instead of... by n9 on Thursday December 21, 2023 @05:49PM (#38728199)
I think people (and the market) make this assumption but the correction isn't Apple doing something differently it is the market and public assumption seeing that when you sell something that lasts longer than a year and works really well you need to accomodate the ebbs and flows of sales across a 5 year period instead of quarterly and YoY.
Re:Apple wants AI to run directly on its hardware instead of... by MBCook on Friday December 22, 2023 @12:16AM (#38731183)
I agree, I think this is what they meant. A feature for the people who have an iPhone 8 or 9 who say “why would I upgrade, my current phone is fine”.

If you can use this tech to make a feature that they really want you can trigger upgrades that otherwise may not happen for many more years. We’ve seen that before with things like when Apple first released a bigger phone. That triggered a LOT of “early” upgrades that wouldn’t have otherwise happened.

Re:Apple wants AI to run directly on its hardware instead of... by ForkMeOnTinder on Thursday December 21, 2023 @02:54PM (#38725894)
fwiw Google is attempting the same thing, but they're not doing much better.

https://www.notebookcheck.net/MrWhosetheboss-video-reveals-G...

> Google's Pixel 8 Pro Tensor G3 off-loads all generative AI tasks to the cloud

Re:Apple wants AI to run directly on its hardware instead of... by rainclouds on Thursday December 21, 2023 @02:55PM (#38725913)
Critical.

I’d like my data to stay local.

I probably don’t need cloud processing for any regular usage.

The privacy to life benefit equation is broken in society atm.

This will change.

Re:Apple wants AI to run directly on its hardware instead of... by tech_ken on Thursday December 21, 2023 @03:21PM (#38726311)
It didn't seem like training will occur locally, just serving prompt responses from a GPT model (maybe I misunderstood the article). My personal privacy calculus places a lot of less value on keeping prompts local, especially if some kind of E2EE can be provided between me and the serving model. I understand that not everyone feels this way, but from the perspective of market adoption I don't see the average user caring too much whether a prompt is kept local or sent to the cloud (if encrypted or similar). To me it seems like the most critical determinants would be response latency and battery life, plus the basic performance of the thing.
Re:Apple wants AI to run directly on its hardware instead of... by mitjam on Thursday December 21, 2023 @03:41PM (#38726604)
NVIDIA‘s newest GPUs support confidential computing which can be used for E2EE but also for intellectual property protection. https://www.nvidia.com/en-us/data-center/solutions/confident...
Re:Apple wants AI to run directly on its hardware instead of... by tech_ken on Thursday December 21, 2023 @04:22PM (#38727154)
If it's just serving prompt responses I'm not sure if the IP protection is a huge dealbreaker for me as a downstream user. The most critical use case I can currently imagine is better predictive text for my keyboard, and that one seems pretty light on the IP violations (for me personally). The biggest thing for me here is E2EE if the text is leaving my phone.