News for hackers. Stuff that matters.
Re:Apple: Transformer architecture optimized for Apple Silicon by davnicwil on Thursday March 23, 2023 @07:04PM (#35282726)
I really think you have hit the nail on the head here. Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices.

We must be talking orders of magnitude differences in operational cost, not to mention completely unique features like privacy. The very definition of disruption, waiting in the wings.

Not only that, but this model of 'wait in the wings and pounce when the tech is really there and absolutely nail it' is Apple's forté, aligned completely with the cultural and strategic DNA of the company.

Re:Apple: Transformer architecture optimized for Apple Silicon by jorvi on Thursday March 23, 2023 @07:30PM (#35283009)
Siri was supposed to become locally processed but sometimes can’t even set a timer because of a connection time-out. Or she’ll use her fancy ML speech recognition model to turn “set a timer for 3 minutes 10 seconds” into “search for trinity tensor”.

So much for an “unfathomable moat”.

Re:Apple: Transformer architecture optimized for Apple Silicon by belugacat on Thursday March 23, 2023 @07:34PM (#35283041)
Siri is so bad it needs to be scrapped and rethought from the ground up (it can’t even give me the time when internet is down at my rural house). No one at Apple who wants a career there would dare propose that. The switch to a LLM architecture could be the perfect transition point for this.
Re:Apple: Transformer architecture optimized for Apple Silicon by 015a on Thursday March 23, 2023 @09:12PM (#35283799)
If Siri can't give you the time without the internet, I think you need to update iOS.

There are definitely two tiers of Siri queries. There are queries like "set the brightness to 10%" or "set a timer for 5 minutes" which absolutely and consistently work without internet, and have for several years, and if you're legitimately having a different experience then its possible a cosmic ray hit your iPhone (or, realistically, you're running into a strange and rare bug which is not indicative of the general experience and will be fixed in three to five years or maybe never). There are also queries like "create a new note" (from the Notes app) which should be able to work offline, but don't. And, naturally, there are queries like "when did Resident Evil 4 come out" which wouldn't reasonably work without internet (but, if you're curious, she does get it right).

In other words, Siri clearly does some local guessing as to whether she can answer a query without the internet, and some queries appear to be miscategorized into the second bucket. My leading theory on why this happens, which may be incorrect, but it seems like: if an app has any Siri functionality which requires the internet to answer, all of the queries which are responded to by that app have to require the internet. It doesn't matter where the processing ends up actually happening to respond to the query, it just shuts the query down. Its weird, but its consistent with the behavior I've seen.

The more important point: Siri's real and weird limitations don't seem to have much to do with limitations in local processing. They've said that the speech interpretation all happens on-device. They encrypt practically all of your data that does get shipped to their servers, so a query like "Open the note titled 'Hello World'" is probably also being processed on-device. But: Siri still requires the internet for that query. That doesn't seem like a significant limitation with their ML algorithms or silicon or anything meaningful; it seems like just a case of dumb coding, which is certainly something Apple is no stranger to.

Re:Apple: Transformer architecture optimized for Apple Silicon by king_magic on Thursday March 23, 2023 @10:42PM (#35284410)
> If Siri can't give you the time without the internet, I think you need to update iOS.

interesting - latest iOS in airplane mode - "hey siri what time is it?" - "you need to turn off airplane mode to do that"

but - "hey siri set a time for 5 minutes" - works just fine

Re:Apple: Transformer architecture optimized for Apple Silicon by smoldesu on Friday March 24, 2023 @12:18AM (#35285034)
They must be using the esteemed time delivery API, https://www.whattimeisitrightnow.com/
Re:Apple: Transformer architecture optimized for Apple Silicon by Kye on Thursday March 23, 2023 @07:40PM (#35283099)
It was pretty bad when they turned on local processing on my 6S Plus. Since getting a new Pro 14 Max, it's as good as it was when it had to hit a server. I think maybe they tried too hard to wedge it into older phones' chips.
Re:Apple: Transformer architecture optimized for Apple Silicon by getcrunk on Thursday March 23, 2023 @07:33PM (#35283037)
You see the only reason it does that is because your … holding your phone wrong
Re:Apple: Transformer architecture optimized for Apple Silicon by highwaylights on Thursday March 23, 2023 @07:46PM (#35283144)
Maybe, but Apple doesn’t have a search to rival Google (or even an assistant, given the state of Siri).

Focusing on privacy and on-device learning is great, but when the strength of these models is in consuming all the data they can hoover up your motive is at odds with your philosophy.

Re:Apple: Transformer architecture optimized for Apple Silicon by 015a on Thursday March 23, 2023 @08:42PM (#35283599)
They have every query their users enter into Spotlight. They have all the results it returns (who cares if they originated from Google, ok maybe they do, maybe not using it for anything is part of their deal with Google). They have your contacts, calendars, messages, email, music, workout history, tasks, real time location, which Siri already uses to e.g. recommend a destination when you get in your car to a degree of accuracy that straight spooks me sometimes.

Apple has access to SUBSTANTIALLY more fine-grained personalized data than Google does. Full stop. That's a weird thing to assert in the tech crowd, but think about it deeply and you'll realize its true.

Google has islands of services that they've done an extremely good job of building bridges between. They have some islands that Apple has nothing like (YouTube is the biggest one) (Search is a huge island, but only indefensible if the interfaces customers use to access it don't leak the queries and results; no one opens a browser and types "www.google.com" on mobile and even if they did, Apple controls the browser renderer, they could lift everything if they wanted to, they won't, I'm simply illustrating how much Apple should fuckin scare Google). But the waters between those islands are patrolled by Samsung, Xiaomi, Oppo, and Vivo, and there's sharks in there as well (uh, the metaphor is falling apart, the sharks are "Google's tumultuous history with privacy and user blowback if they overstep").

Apple is a continent. Many of their cities are ports that connect to some of Google's islands, but Google's product still has to be checked by customs.

You're right that even given that power, hoovering it up is at-odds with Apple's philosophy. Or, is it? I think "hoovering it unencrypted to the cloud" definitely is; but that's the point OP was making: if we're extremely close, as a species, to solving "make AI work", one of the challenges for the next five years is inevitably going to be "make it more personal". Its awesome that I can ask ChatGPT to write a date comparison function. It'll also be awesome if I could ask Siri "when did sarah and I talk about getting a cat" or something.

That requires personalized data to set the context. If Apple can swing at the fences and say "everything is on-device, nothing leaves, we can't see it, and now you can ask Siri that, and by the way there's new functionality built-in to iOS that apps can leverage to integrate with Siri's new LLM capabilities just like ChatGPT plugins"; that's an extremely compelling product. Extremely. And I know they would ingest that data, that they would do that, because they already do! Go ask Siri to call Sarah, or if you have any meetings tomorrow (assuming you're using Apple Calendar), and it will respond. I don't know where you're getting this take that they don't "hoover data"; they ingest everything from all their first party apps into Siri's `DATA MATRIX`. Its just, you know, 2010s era querying and data crunching.

Don't get me wrong, Google's gonna do a lot of this too. But Google is playing from the position of "we have a model, we have the data, we just need to pay hundreds of millions of dollars a year maintaining all these servers". Apple is playing from the position "our customers are paying us for the silicon to run this, we have the data, we just have to figure out the model". If its not obvious at this point: the models/algorithms/etc are not a moat. They're going to be commoditized with time. Publicly accessible data isn't a moat. Personal data is a moat, because people care about privacy. And cost effective training and inference silicon is a moat, because its Physical, and like literally One Company on the planet makes it, and they're in a big time situationship with Tim Cook.

Re:Apple: Transformer architecture optimized for Apple Silicon by oidar on Thursday March 23, 2023 @11:49PM (#35284887)
GPT-4 seems pretty "moatish".
Re:Apple: Transformer architecture optimized for Apple Silicon by hobs on Friday March 24, 2023 @12:02AM (#35284950)
Apple does not have the queries that their users enter into spotlight, They dont have all the results it returns.

You dont spawn a network request on every local search on an apple device.

Re:Apple: Transformer architecture optimized for Apple Silicon by saagarjha on Friday March 24, 2023 @01:41AM (#35285464)
Typing into Spotlight sends a request to Apple?
Re:Apple: Transformer architecture optimized for Apple Silicon by 015a on Saturday March 25, 2023 @03:35PM (#35306299)
Yes you do. It trivially does. Search for "Burger King" and see how many results that generates which couldn't possibly be served locally (undownloaded Apps from the App Store, Maps, search recommendations, etc).

They may not be storing these queries or using them for further analytics, training, etc. But they could.

Re:Apple: Transformer architecture optimized for Apple Silicon by smoldesu on Friday March 24, 2023 @12:03AM (#35284956)
> And cost effective training and inference silicon is a moat, because its Physical, and like literally One Company on the planet makes it,

I got Nvidia on my paper. Did I do the math wrong?

Re:Apple: Transformer architecture optimized for Apple Silicon by jocaal on Friday March 24, 2023 @02:37AM (#35285753)
Google has TPUs, Amazon has tranium and inferentia
Re:Apple: Transformer architecture optimized for Apple Silicon by 015a on Saturday March 25, 2023 @03:41PM (#35306384)
TSMC manufactures the silicon for nearly all of the current generation high end silicon. The only exceptions I've been able to find are, naturally, Intel's CPU lineup (which has nothing to do with AI) (and, to be clear, Intel Arc is also manufactured by TSMC), and the Snapdragon 8cxg3, which is manufactured by Samsung; and the "advanced-ness" of the 8cxg3 is pretty miserable compared to A16, M2, RTX 4000, RDNA3, or Tensor Processing Units.

Apple has purchased the entirety of TSMC's 3nm production for the next 1 to 2 years. They can do that, and can continue to do that, because they have a ton of money. They have a ton of money for buying chips because there isn't some crazy business middle-logic justifying the cost of these chips' performance; they buy chips, they sell chips to consumers. In comparison, literally zero other customers of TSMC derive the majority of their hardware revenue from selling to consumers. Companies like Nvidia make some money this way, but most of their money goes to data center sales, which has their own business justification for buying them which fluctuates (ChatGPT subscriptions? training models? is the past model good enough? etc).

Re:Apple: Transformer architecture optimized for Apple Silicon by threeseed on Thursday March 23, 2023 @08:06PM (#35283302)
They do have a search engine though with Siri Suggestions. Which has a user base of ~1.5b+ people.

We only see the tip of the iceberg so who knows what else it is capable of.

Re:Apple: Transformer architecture optimized for Apple Silicon by lotsofpulp on Thursday March 23, 2023 @08:11PM (#35283347)
“Hey siri, what is today’s date”

“Sorry, I’m having trouble connecting to the network”

Re:Apple: Transformer architecture optimized for Apple Silicon by dmd on Thursday March 23, 2023 @08:19PM (#35283409)
“hey siri, what’s the weather today?”

“Now playing Eminem Love the way you lie featuring Rihanna”

Re:Apple: Transformer architecture optimized for Apple Silicon by lotsofpulp on Thursday March 23, 2023 @08:22PM (#35283437)
Hey siri, turn the TV off.

Which TV? Bedroom or Living Room or Everywhere?

(Only 1 of the 2 TVs is ever on)

I won’t spam this thread anymore, but I would be pleasantly surprised if it improved.

Re:Apple: Transformer architecture optimized for Apple Silicon by illiarian on Friday March 24, 2023 @02:37AM (#35285756)
And AppleTV itself is hot garbage for this kind of stuff even though it's a veritable supercomputer that should be the hub for Siri and home automation.

- Hey Siri, <do something that involves AppleTV in any capacity>.

- I'm sorry, one of your devices is off

Re:Apple: Transformer architecture optimized for Apple Silicon by scarface74 on Thursday March 23, 2023 @08:31PM (#35283515)
And that has nothing to do with an LLM. That has to do with speech to text algorithms. It also doesn’t take advanced AI to do a simple intents based answers.
Re:Apple: Transformer architecture optimized for Apple Silicon by nicky0 on Friday March 24, 2023 @04:22AM (#35286437)
Yes we are aware that Siri doesn't use LLM yet. That is what this thread is about.
Re:Apple: Transformer architecture optimized for Apple Silicon by epistasis on Thursday March 23, 2023 @09:31PM (#35283932)
Me: Hey Siri, call <my city> Toyota.

Siri: I'm sorry, that contact is not in your list

Me: Siri, what is the number for <my city> Toyota.

Siri: <my city> Toyota's phone number is 123-456-7890 [said too fast to remember or write down in one go]

Me: Siri, call <my city> Toyota

Siri: I'm sorry, you do not have that contact number.

Me: &$@@&/&&/&!!!

Re:Apple: Transformer architecture optimized for Apple Silicon by gnicholas on Thursday March 23, 2023 @10:52PM (#35284463)
I work around this by asking it to find Toyota in <my city>. Then if it finds the dealership, it asks if I want to navigate there, or call them.

I find the worst errors to be when I ask it for information, or to send a text, and it instead places a phone call to someone. It will even call people that I have never called on my phone (a fact it should know), without asking first.

Re:Apple: Transformer architecture optimized for Apple Silicon by fingerlocks on Thursday March 23, 2023 @10:54PM (#35284470)
Just tried this exact interaction. Siri gave me a list of nearest Toyotas and made the call. Seems to be working just fine.

Starting to think all these Siri complaints are either made up or really outdated.

Re:Apple: Transformer architecture optimized for Apple Silicon by epistasis on Thursday March 23, 2023 @11:23PM (#35284696)
If you have the magic phrase to make it work, I'd like to know. I tried for about five minutes while driving yesterday, to no avail.

For me, Siri is like an 80s text adventure game, except I was better at those.

Edit: just now I was able to call by interacting a second time with Siri, and using the physical button. Using the button never would have occurred to me while driving.

Re:Apple: Transformer architecture optimized for Apple Silicon by lotsofpulp on Thursday March 23, 2023 @11:08PM (#35284586)
I only use Siri with the original Homepods. It has always been terrible. You cannot even tell it to start playing movies on the TV that you have already purchased from Apple itself in the TV app. It will start to play some random song or something.

I should be able to say hey siri, start playing <name of media> on <name of Apple TV>, and it should be able to start the TV and start playing it.

Re:Apple: Transformer architecture optimized for Apple Silicon by highwaylights on Friday March 24, 2023 @04:20AM (#35286414)
A friend has one that constantly plays U2 out of nowhere. He doesn't even listen to U2, and maintains that he's never requested it.

This isn't in response to a prompt either. He'll unpause the connected TV and it'll just start blasting U2.

Re:Apple: Transformer architecture optimized for Apple Silicon by UncleEntity on Friday March 24, 2023 @09:38AM (#35288899)
Yes, the U2 Songs of Innocence album.

In case someone doesn’t know they gave that album to all iTunes users as part of a promotional thing when it came out.

On my iPhone if I connect the Bluetooth headset and accidentally push the call/cancel button out comes U2…

Re:Apple: Transformer architecture optimized for Apple Silicon by nicky0 on Friday March 24, 2023 @04:21AM (#35286431)
You must be the one user that Siri works perfectly for.
Re:Apple: Transformer architecture optimized for Apple Silicon by RulerOf on Friday March 24, 2023 @06:29AM (#35287233)
I spent a decade lowering my expectations, and they're still not low enough.
Re:Apple: Transformer architecture optimized for Apple Silicon by gcanyon on Thursday March 23, 2023 @10:56PM (#35284495)
The perfectly obvious use cases Siri overlooks are maddeningly numerous. My go to example was "text this photo to <contact name>" For years I checked each new iOS release to see if that was enabled, and for close to ten years, no. I'm not sure whether it was the most recent, or the one before (I gave up somewhere along the way) but now you can (finally) do it.
Re:Apple: Transformer architecture optimized for Apple Silicon by Tagbert on Friday March 24, 2023 @12:47PM (#35291832)
"HeySiri, send this photo to name in messages." "Ready to send" "Send it" "It's sent"

done

Perhaps the verb "text" is unclear to Siri?

Re:Apple: Transformer architecture optimized for Apple Silicon by gcanyon on Friday March 24, 2023 @03:36PM (#35294352)
Like I said: it does it now. It didn't from inception until something like 2021.
Re:Apple: Transformer architecture optimized for Apple Silicon by fsckboy on Thursday March 23, 2023 @09:31PM (#35283936)
Siri never tells me it has trouble connecting to the network, it says "something went wrong"
Re:Apple: Transformer architecture optimized for Apple Silicon by astrange on Thursday March 23, 2023 @10:12PM (#35284209)
If you're asking for an LLM this will get worse because the first L means "large", as in too large to fit in a phone without a network connection.
Re:Apple: Transformer architecture optimized for Apple Silicon by highwaylights on Friday March 24, 2023 @04:17AM (#35286402)
two minutes later

"Hey Siri, what is today's date?"

"That's sweet, but I think of you as a friend"

"What? The date, Siri, what is the date?"

"You're so sweet"

"Sigh"

Re:Apple: Transformer architecture optimized for Apple Silicon by scarface74 on Thursday March 23, 2023 @08:30PM (#35283501)
Apple gets $18B a year from Google to be the default search engine on Apple devices. Why would Apple build a search engine that would probably be inferior and not be able to monetize it?
Re:Apple: Transformer architecture optimized for Apple Silicon by nojito on Thursday March 23, 2023 @09:51PM (#35284066)
They already have a search engine?

Just swipe down on any iPhone.

Re:Apple: Transformer architecture optimized for Apple Silicon by scarface74 on Thursday March 23, 2023 @10:01PM (#35284126)
Yes and at one point someone wrote a web front end for it. It’s laughably bad.
Re:Apple: Transformer architecture optimized for Apple Silicon by behnamoh on Thursday March 23, 2023 @11:25PM (#35284716)
I had to disable several features of iOS search due to ridiculously long delay and lag.
Re:Apple: Transformer architecture optimized for Apple Silicon by fnordpiglet on Thursday March 23, 2023 @08:04PM (#35283288)
I think this is wrong because you can always build a search engine - it’s not hard - and you can supplement the learned models with cryptographically secured private data to build a private personalized model. In fact this seems in all ways superior to any alternative, and strikes at the heart of the Google spy ware ad spam business model.

And seriously, did you really try to assert the Google whatever it is has outperformed Siri? I’d love to meet someone that uses it so I can verify this statement. But with the advent of Chatgpt Alexa, Siri, and Google whatever all look like freshman projects. The next generation of assistants have just begun, and I assure you, the state of the past has nothing to do with what comes next. OpenAI hit a giant reset button across an awful lot of stuff.

Re:Apple: Transformer architecture optimized for Apple Silicon by coolspot on Thursday March 23, 2023 @08:09PM (#35283334)
> And seriously, did you really try to assert the Google whatever it is has outperformed Siri?

Anything outperforms Siri. Google Assistant is head and shoulders above Alexa and Siri.

Re:Apple: Transformer architecture optimized for Apple Silicon by schrodinger on Thursday March 23, 2023 @08:29PM (#35283493)
I have both and find that 90% of the time my commands are “remind me ___”, “wake me up at ___”, “what’s the weather today”, and “play ___ on Spotify”. For these, I can’t tell the difference between google and Siri in terms of quality (but Alexa was worse).
Re:Apple: Transformer architecture optimized for Apple Silicon by bigDinosaur on Thursday March 23, 2023 @10:20PM (#35284269)
I was just setting reminders today and had Siri repeatedly fail to do what I want. The only use case I have for it is setting timers, which is does very well. I'd use it for much more if it was as good as ChatGPT.
Re:Apple: Transformer architecture optimized for Apple Silicon by fnordpiglet on Thursday March 23, 2023 @11:55PM (#35284914)
This is entirely my point. They’re all crap at a fundamental level, none of them outperform any other. But the next generation, which will assuredly be backed by a LLM, will be shockingly powerful.
Re:Apple: Transformer architecture optimized for Apple Silicon by abudabi123 on Friday March 24, 2023 @01:46AM (#35285479)
The MKBHD personality is onto Monte Carlo testing in his device reviews. He has an intuition for good UIX. Google and Apple should pay people like MKBHD and his production crew a billion dollars every year to film achievable, desirable text, voice, camera A.I. assisted use-cases.
Re:Apple: Transformer architecture optimized for Apple Silicon by fnordpiglet on Friday March 24, 2023 @10:24AM (#35289452)
They don’t work like that though. Instead they spend $3b / year on hiring fungible cog SDE 2 job code 67483’s and look at gantt chart roadmaps produced by fungible cog PM 3 job code 74842’s for fungible cog SDM 7 job code 84747’s with the nominal assistance of centralized (I.e., marginalized) fungible cog UX 5 job code 35563’s then wonder why it’s perennially late and complete garbage. Except no one wonders that because fungible cogs don’t have wonder in their job responsibly matrix.
Re:Apple: Transformer architecture optimized for Apple Silicon by arthurcolle on Thursday March 23, 2023 @08:15PM (#35283377)
> Apple has a ridiculous, almost unfathomably deep moat for training and running personalised, customised LLMs and other AI models on the 'edge' with these Apple Silicon chips in all their devices.

Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today. I haven't built anything against Siri APIs, but I've used Siri and various integrations and every single time I give her a shot, she disappoints me. It can do cookie-cutter, super well-traveled code paths that were engineered together for demos, basically just one off tricks without cohesion, but the whole infrastructure around Siri is not open and has not been improved in any way in many OS versions in any way that I've discerned... But I am not an expert with these systems or APIs.

The Neural Engine architecture especially considering M1/M2 hardware leaps (which are absolutely mind-bogglingly impressive) are both really superb technologies from my layman view as a mere software engineer, with improved battery life as well as performance, a real improvement over Intel x86 architectures - I just don't see Apple as a serious AI player right now, despite head start in AI w/ Siri acq, despite these hardware leaps that may make it easier to do cool stuff in the future like this post. The OCR stuff in Photos is cool and useful and I use it every day, usually to look up my Known Traveler Number.

Ramblings over, just wanted to ask you to opine as to Apple LLM related information if you have any additional context!

Re:Apple: Transformer architecture optimized for Apple Silicon by parker_mountain on Thursday March 23, 2023 @08:22PM (#35283430)
> how garbage Siri quality is today

a few notes on this

- siri is incredibly underinvested. They have another team that's building some sort of search and natural language processing engine, that has slowly sapped away some key headcount from the siri team.

- apple doesn't get the full advantage of tons of user data from the wild. this is both a bug and a feature

- the siri api model is clearly generations old, and i suspect that apple has been marshaling its resources into a big leap forward - alongside their hand tracking, ar, and hardware

- apple has shipped everything required for you to point at a light /in your house/ to turn it on or off (and optionally flick). this includes software - the individual components are built and ready - the only thing missing is gluing it together

There's something happening, and I think that the rumored glasses are the hardware totem.

Re:Apple: Transformer architecture optimized for Apple Silicon by afavour on Thursday March 23, 2023 @08:36PM (#35283550)
“Yes Siri is garbage but it doesn’t matter because Apple are probably working on a top secret new thing that will blow everyone else out of the water”

You might be right but there’s no real evidence for it yet.

Re:Apple: Transformer architecture optimized for Apple Silicon by mikepurvis on Thursday March 23, 2023 @09:40PM (#35284001)
Lots of things with Apple come out of nowhere. In this case there's actually quite a bit more evidence than usual because of all the fielded, underutilized hardware that's ready to go— when it launches, it'll likely be available on millions of devices, rather than as something you have to get the latest hardware to have, as is the case when the camera or screen undergoes a drastic improvement, or there's a new feature like wireless charging or NFC payments.
Re:Apple: Transformer architecture optimized for Apple Silicon by LoganDark on Thursday March 23, 2023 @09:42PM (#35284010)
Hasn't Siri remained relatively unchanged for over a decade? I think the most recent innovation was just bringing it to Intel Macs.
Re:Apple: Transformer architecture optimized for Apple Silicon by bigDinosaur on Thursday March 23, 2023 @10:18PM (#35284247)
It has had features removed.
Re:Apple: Transformer architecture optimized for Apple Silicon by PopePompus on Friday March 24, 2023 @02:07AM (#35285589)
Yes, it's been stagnant for so very long that I, and I suspect most users, have learned the very few things that it can do reliably, and no longer even try to do something more sophisticated. If it became more capable, it might take a long while for most users to even notice, because we have stopped hoping it will pleasantly surprise us.
Re:Apple: Transformer architecture optimized for Apple Silicon by sroussey on Friday March 24, 2023 @11:28AM (#35290332)
Most people didn’t notice, but dictation now runs in a hybrid mode — both on device and via server. So you get some (maybe poor) results at first (only results if offline but better than nothing) that get rewritten after a couple seconds from server.

For Siri itself, I think it runs locally now.

Re:Apple: Transformer architecture optimized for Apple Silicon by parker_mountain on Thursday March 23, 2023 @10:09PM (#35284172)
>because Apple are probably working on a top secret new thing that will blow everyone else out of the water”

that's a mischaracterization of what i said. what i said was that they have been clearly hiring key positions and cannibalizing the siri team for something new. that something new will likely be a major release. i also believe that the ar headset is the unifying product under which they're rallying.

Re:Apple: Transformer architecture optimized for Apple Silicon by janekm on Friday March 24, 2023 @04:31AM (#35286492)
You may well be right, but if so that's a huge problem for Apple as by all accounts the AR device is still many years from being ready for release and meanwhile they are in a very precarious situation with Siri...

The first proper integration of a Whisper & a GPT-class LM will be a big step change, a Siri-like AI that you can actually talk to somewhat sensibly and expect it to "understand" more than pre-set phrases... Google could be in a position to release something for Android, and their are Android SoCs with neural accelerators as well (sure nothing as impressive as Apple's chips but Samsung have demonstrated a cut-down SD model on their SoC).

Re:Apple: Transformer architecture optimized for Apple Silicon by parker_mountain on Sunday March 26, 2023 @04:09PM (#35319002)
I don't think the new siri is intrinsicly tied to the device. I think that as part of remaking siri, they're using the new platform as a rallying cry and story as to why it's important. (it's hard to justify features such as making siri spatially aware otherwise)
Re:Apple: Transformer architecture optimized for Apple Silicon by ngai_aku on Thursday March 23, 2023 @09:33PM (#35283950)
> apple has shipped everything required for you to point at a light /in your house/ to turn it on or off (and optionally flick). this includes software - the individual components are built and ready - the only thing missing is gluing it together

Could you expand on this bit? I’m pretty deep in the Apple ecosystem, but I’m not sure what you’re referencing here

Re:Apple: Transformer architecture optimized for Apple Silicon by revscat on Thursday March 23, 2023 @09:52PM (#35284070)
Not OP, but I suspect this is a simple combination of AR and precise device location awareness. The former comes in the form of ARKit. I’m not aware of a framework for locating devices, but it wouldn’t surprise me if there were one.
Re:Apple: Transformer architecture optimized for Apple Silicon by xvector on Friday March 24, 2023 @02:28AM (#35285702)
U1 chip triangulation solves the second part. Would require at least 3 U1-enabled devices in your home.

Alternatively whole-home mapping via AR also solves this. No triangulation needed.

Re:Apple: Transformer architecture optimized for Apple Silicon by parker_mountain on Thursday March 23, 2023 @10:11PM (#35284194)
Homepods have U1 chips. They can position airtags within your house. You can put an airtag under, or adjacent to, a lamp and link it to a smart switch. Your apple watch has gesture detection (still in beta, tbf), as well as U1 and real time positioning capabilities.

Everything required for apple to know not only where you are, but which way your hand is pointing, as well as where "smart devices" are in your house is already being sold and rolled out en masse to the majority of people in the ecosystem.

Re:Apple: Transformer architecture optimized for Apple Silicon by ohgodplsno on Friday March 24, 2023 @03:14AM (#35285989)
$25 for an Airtag, plus say $20 for a smart switch. Plus $200 for a HomePod.

To turn a single lamp on/off by pointing at it, something that nobody ever wants to do (alright, once for the cool factor). If someone wanted to overpay for useless features, they can already go for a Philips Hue.

I can turn off my lamps from anywhere in the world using a $10 Tuta ZigBee bridge and a $8 LIDL light.

Re:Apple: Transformer architecture optimized for Apple Silicon by tlrobinson on Friday March 24, 2023 @05:42AM (#35286942)
You sir, are not Apple’s target market.
Re:Apple: Transformer architecture optimized for Apple Silicon by mgdante on Friday March 24, 2023 @07:36AM (#35287705)
Seems like Apple's target market is very narrow given homepod's weak sales
Re:Apple: Transformer architecture optimized for Apple Silicon by parker_mountain on Friday March 24, 2023 @02:53PM (#35293805)
I think that this is one of those things that sounds like you'd never do it, but...

I've put zigbee switches into every wall switch in my house and now when I stay in hotels, I forget to turn off the lights before getting into bed.

>something that nobody ever wants to do

I think that this gesture, if it works well, is something that will be a killer app for smarthomes. The other point of friction, which is interop, has been more or less solved by thread/matter. Homepods are $100, btw. Not to mention what happens if apple integrates u1 into a smart switch/bulb, which is something that the thread protocol allows for.

Re:Apple: Transformer architecture optimized for Apple Silicon by diffeomorphism on Sunday March 26, 2023 @04:13AM (#35312260)
Meanwhile you could use an iphone as a webcam for years by just buying an app,you could use an ipad as a cintiq by just buying an app, using your phone with your computer with kdeconnect also has been around for years. Pretty much nobody cared. Then apple came out with their branded version and suddenly you have people praising it as the best thing since sliced bread and great "innovation".

So my prediction would be that apple releases a ZigBee bridge and light for $50 in about 3 years and calls it a fancy name. People will tell you how absolutely essential and innovative it is and how you could never get that without an iDevice.

Re:Apple: Transformer architecture optimized for Apple Silicon by post-it on Friday March 24, 2023 @01:58AM (#35285546)
It's probably not glasses. Nobody likes wearing glasses. Every attempt at making tech glasses has failed. It'll probably be the Apple Watch.
Re:Apple: Transformer architecture optimized for Apple Silicon by FredPret on Friday March 24, 2023 @03:30AM (#35286116)
Everybody would love wearing Apple Glasses. Would probably look really cool.

I remember seeing AirPods in somebody's ears for the first time. "What a dork" I thought back then. Now they're cool and everywhere.

Re:Apple: Transformer architecture optimized for Apple Silicon by highwaylights on Friday March 24, 2023 @04:14AM (#35286382)
> Now they're cool and everywhere.

One of these things is true.

Re:Apple: Transformer architecture optimized for Apple Silicon by yurishimo on Friday March 24, 2023 @05:37AM (#35286914)
There is a difference between the subjective and objective "cool." They've sold something like 150 million pairs of Airpods? I think that qualifies as objectively "cool."
Re:Apple: Transformer architecture optimized for Apple Silicon by FredPret on Friday March 24, 2023 @05:46AM (#35286966)
Another heuristic is: are the majority of attractive and young people using/buying/doing it?

AirPods are cool by that measure as well.

Re:Apple: Transformer architecture optimized for Apple Silicon by smugma on Saturday March 25, 2023 @01:13PM (#35304763)
If lots of people are doing or buying something, that’s a signal that it isn’t cool.
Re:Apple: Transformer architecture optimized for Apple Silicon by wwalexander on Friday March 24, 2023 @09:36AM (#35288869)
I would not love wearing them. I waste a significant amount of money on contact lenses each year for the privilege of not having to wear anything on my face.
Re:Apple: Transformer architecture optimized for Apple Silicon by moonchrome on Friday March 24, 2023 @04:32AM (#35286498)
I'm not a VR/AR fan but if Apple can come out with super high quality AR headset that can augment/replace monitors I would buy one instantly. I might be earlier than mainstream adopter but if the tech ever gets there this will be a natural shift.

I won't buy Meta products but I keep waiting for someone to do VR workspace right - where I can truly work and travel from anywhere with a decent chair and a desk for keyboard (no need to lug around monitors, no need for huge table).

Re:Apple: Transformer architecture optimized for Apple Silicon by niklasrde on Friday March 24, 2023 @07:33AM (#35287678)
I think that's it. I had half an hour to kill in a cafe yesterday, didn't bring my kindle, and I thought "if there were compact AR or even VR glasses that I could whip out to emulate a full-size working environment, that would be super handy right now".

Having said that, for me to spend money on it it would (a) also be able to realistically replace my set up at home, which currently consists of 2 HiDPI screens and (b) not be that much more expensive than what that cost me.

Re:Apple: Transformer architecture optimized for Apple Silicon by nicky0 on Friday March 24, 2023 @02:20AM (#35285649)
As an mere user this is what I had assumed. Siri has been stagnant and buggy for years; it makes sense they are working on a big replacement instead of putting resource into incremntal improvements on what seems to be a weak foundation.
Re:Apple: Transformer architecture optimized for Apple Silicon by pantulis on Friday March 24, 2023 @07:39AM (#35287730)
> apple doesn't get the full advantage of tons of user data from the wild.

Do you think OpenAI did? Perhaps Apple doesn't have the vast amount of data that Google has, but if OpenAI managed to have like 200 wikipedia-sized corpuses of different textual data for their English GPT models, that's certainly not out of reach of Apple.

Re:Apple: Transformer architecture optimized for Apple Silicon by cal85 on Friday March 24, 2023 @08:23AM (#35288088)
> apple has shipped everything required for you to point at a light /in your house/ to turn it on or off

Point at it? You mean with a finger? What do you mean they've shipped everything required?

Re:Apple: Transformer architecture optimized for Apple Silicon by feifan on Thursday March 23, 2023 @08:21PM (#35283423)
The part that Siri is bad at will be commoditized — someone will open-source a GPT-4-level language model. And Apple's moat will be being able to run that on-device with all the attendant benefits (privacy, zero marginal cost to the company, availability in more scenarios, etc)
Re:Apple: Transformer architecture optimized for Apple Silicon by arthurcolle on Thursday March 23, 2023 @08:29PM (#35283489)
yeah, makes sense. At least these integrations at a lower level are happening now. I just can't help but feel disappointed that we didn't see earlier cohesion at least a few years ago, maybe around the early or pre-M1 era when there were all those A7, A8 whatever SoCs. But in retrospect, that is basically 10 minutes ago in the history of the universe so yeah, I'm just overly excitable!
Re:Apple: Transformer architecture optimized for Apple Silicon by jocaal on Friday March 24, 2023 @02:30AM (#35285712)
You realize that these language models are like 100's of GBs in size and consumes 10's of GB's of memory. Last time I checked, apple still ships their products with less than the market average in both of these specs. If you want a local running LLM on an iphone, get ready to sell a kidney.
Re:Apple: Transformer architecture optimized for Apple Silicon by ithkuil on Friday March 24, 2023 @04:55AM (#35286642)
(assuming we're not talking about the near future)

I think this can be a scenario of converging incentives: on one side large models will incentivized hardware manufacturers to increase the memory available on the devices, while on the other sides model developers will be incentivized to trim the fat on the models and devise compression mechanisms that don't compromise quality too much.

It's not unthinkable to imagine a hand held device able to run full inference locally a few device generations in the future.

Re:Apple: Transformer architecture optimized for Apple Silicon by l33tman on Friday March 24, 2023 @05:58AM (#35287036)
You can today run an LLM vastly better than Siri on a few GB of RAM using Llama 7B at 4-bit quantization and alpaca.cpp. This is moving so fast, every day there is something new coming. There won't be any moat in LLMs soon or even in dedicated HW as it turns out you don't need that much for "basic intelligence".

Note I'm not suggesting you can pack the full knowledgebase of humanity into those 2GB of RAM, but the key feature of an edge AI is simply to understand instructions, something Siri and Ok Google struggle with at best..

Re:Apple: Transformer architecture optimized for Apple Silicon by BonoboIO on Thursday March 23, 2023 @08:58PM (#35283691)
Same experience.

Alexa is DIMENSIONS better than Siri. Siri can create a timer ... okay even an alarm. That’s it. It is comically bad. Their text to speech is excellent, but the rest is unbelievable bad.

If Apple has some kind of silver bullet, it’s time to put it out or be left behind.

Re:Apple: Transformer architecture optimized for Apple Silicon by bradDonniger on Thursday March 23, 2023 @09:18PM (#35283852)
I don’t have all the details because NDAs but I ask questions and get nods and grins from colleagues and friends at chip companies; Apple (and nVidia and Intels) silver bullet to cloud hosted, software based AI is AI chips.

We’re circling back around to local compute being the default as hardware performance of next gen phones and tablets reaches a “good enough” point for most users.

There will be scientific problems that will require modern server clusters but most consumer facing AI needs will be done on hardware within a decade.

I’m not saying AGI in a decade, I’m saying cutting edge logic embedded in software now will be the basis for logic in chips in the years to come.

Re:Apple: Transformer architecture optimized for Apple Silicon by fbdab103 on Thursday March 23, 2023 @10:12PM (#35284202)
Can we take a minute to comment on how bad the text correction is on the iPhone?
Re:Apple: Transformer architecture optimized for Apple Silicon by Tiktaalik on Thursday March 23, 2023 @11:39PM (#35284805)
Amazon has publicly walked back from Alexa because turns out people don't use it for much beyond creating a timer and it doesn't make money.

I think it's quite likely that Apple realized the same quite a bit earlier and eased up on Siri versus focusing on other things.

Re:Apple: Transformer architecture optimized for Apple Silicon by vbezhenar on Friday March 24, 2023 @12:57AM (#35285240)
Honestly I observe zero progress in the last 10 years. My first iPhone was 4S. My current iPhone is 8. My use-case is absolutely the same. Calls, SMS, Whatsapp, navigation, books, browser. The same with macs. I boot, open IDE, Terminal and that's about it. Yes, hardware's getting faster (which is compensated by software getting slower). But nothing changed dramatically.

I think that the only thing that changed is adoption of fingerprint reader. Both in iPhone 8 and in mac.

AI might be that thing that could tremendously change my usage pattern. But it should be as smart as human assistant. GPT4-level intelligence might have the necessary power for that.

Re:Apple: Transformer architecture optimized for Apple Silicon by gumby on Thursday March 23, 2023 @09:05PM (#35283741)
> > Apple has a ridiculous, almost unfathomably deep moat for training and running [LLMs]...

> Do they? I can completely fathom, given my own anecdotal experiences, how garbage Siri quality is today.

Siri is a dead end. When Jobs bought Siri (what, 10 years ago?) he explicitly junked almost all the AI back end, mainly buying the speech recognition engine. I didn't understand why and still don't (but strangely he didn't ask me :-).

John Giannandrea has run Apple's AI effort for the past five or six years. He is the reason Google has a big AI effort (he consolidated a bunch of AI projects and bought Deep Mind, etc) before he decamped for Apple. For all I know the Siri team isn't even part of his remit.

You can never look into Apple (even if you work there) so one can only speculate based on what visible signs appear. But Siri isn't one of them.

Re:Apple: Transformer architecture optimized for Apple Silicon by duped on Thursday March 23, 2023 @09:28PM (#35283917)
Jobs has been dead for over ten years
Re:Apple: Transformer architecture optimized for Apple Silicon by gumby on Thursday March 23, 2023 @10:50PM (#35284447)
And it feels like nobody has worked on it since then
Re:Apple: Transformer architecture optimized for Apple Silicon by abudabi123 on Friday March 24, 2023 @01:32AM (#35285410)
The cinema industry know how to squeeze a billion dollar a year out of Apple. Do the Apple or Sony ”social engineers“ have total view? When Amazon was sky rocketing and Blue Origin had not reached orbit a niche producer from the cinema industry had Jeff B paying a lot although he might have backed out of that deal like giving up on the Android phone with a lot of cameras.
Re:Apple: Transformer architecture optimized for Apple Silicon by dev_tty01 on Friday March 24, 2023 @10:43AM (#35289681)
>When Amazon was sky rocketing and Blue Origin had not reached orbit...

Blue Origin still has not reached orbit.

Re:Apple: Transformer architecture optimized for Apple Silicon by sidibe on Thursday March 23, 2023 @11:09PM (#35284597)
> He is the reason Google has a big AI effort (he consolidated a bunch of AI projects and bought Deep Mind, etc) before he decamped for Apple

I'm sure he's great but this made me laugh

Re:Apple: Transformer architecture optimized for Apple Silicon by gumby on Friday March 24, 2023 @01:03AM (#35285277)
I aim to please!

(And yes he’s a great guy)

Re:Apple: Transformer architecture optimized for Apple Silicon by richardw on Thursday March 23, 2023 @09:07PM (#35283758)
If you’re the exec in charge of this at Apple, what are you doing right now. It’s not watching OpenAI become what Siri could be unless you have something to do about it. Your motivation levels are extremely high.
Re:Apple: Transformer architecture optimized for Apple Silicon by theptip on Thursday March 23, 2023 @11:49PM (#35284888)
I don’t think Apple necessarily needs to lead the pack on LLM research. They just need to take the SOTA FOSS model and bake it into silicon, using their chip design expertise.

If you do that, you can do inference on mobile devices, which is a huge privacy win; it plus into their general privacy positioning in a big way. If you open up that SDK to developers it would be the iOS app gold rush all over again.

Note, Google is also moving in the direction of on-device inference with Coral, but they obviously will want to transmit the personalized model weights back to the mothership.

Re:Apple: Transformer architecture optimized for Apple Silicon by mannyv on Friday March 24, 2023 @02:29AM (#35285704)
Siri is garbage because it's trying to be a real AI, as it was understood to be at the time. Alexa works because it's not an AI but it seems as if it's an AI.
Re:Apple: Transformer architecture optimized for Apple Silicon by agentcoops on Friday March 24, 2023 @09:29AM (#35288786)
Honestly, it speaks mountains to Apple as an organisation that they did not invest all of their resources into Siri, especially when you consider how Amazon invested in Alexa etc. Everyone has long understood that the "personal assistant" represents really the pinnacle of not just what AI but the personal computer should be able to offer -- the mistake this past decade was thinking that had to do with then current ML paradigms or with these completely useless voice agents. That Apple could realise it was a dead-end and not sink much more investment/engineering resources into the tool any further than what anyone actually uses these tools for (changing song, lowering volume, etc) is outstanding.

Ultimately, I'm very reassured that it's Apple and Microsoft leading the way for personal and business AI assistants respectively. As Stratechery has emphasised, the latter not only already has all the data from most firms outside Silicon Valley, but more importantly has always adhered to and designed their software according to a computer-as-a-tool philosophy versus Google's "ML/AI will do everything for you behind the scenes"; the former has not only made a show of data security but invested massively in the hardware architectures necessary to making on-device AI a possibility. Without at all being a "fanboy", Apple is quite literally the only company I would trust my personal data to for use in a GPT-based assistant.

Re:Apple: Transformer architecture optimized for Apple Silicon by mark212 on Friday March 24, 2023 @11:37AM (#35290502)
Amazon's investment in Alexa is a perfect example of why Apple didn't (and shouldn't have) invested all of their resources into Siri. The Alexa team is getting seriously gutted in these rounds of downsizing [1], or as CNBC puts it "the team behind the technology was a prime target of the largest layoffs in the company’s history."

What they have been invested heavily in is the Apple Neural Engine ("ANE"), special silicon right on the SoC to handle ML / AI code. Optimize on a server, then run the model on your iPhone or probably soon, your Apple Watch.

WWDC this year is going to be very, very important.

[1] https://www.cnbc.com/2023/01/06/amazon-fully-committed-to-al...

Re:Apple: Transformer architecture optimized for Apple Silicon by ThorsBane on Thursday March 23, 2023 @09:37PM (#35283969)
Yeah, Apple is slower to adopt some things but I noticed when they hit it...they knock the ball not just out of the park...they elevate the entire game.

I dominated a 13" mbp m1 for like two solid years on hardcore startup software engineering. The base model. It was almost entirely perfect until the last few months when I upgraded to the new 16".

These are alien devices. They're just not possible yet here I am using one.

Re:Apple: Transformer architecture optimized for Apple Silicon by wetpaws on Thursday March 23, 2023 @11:53PM (#35284901)
Apple's exceptional hardware can only be rivaled by the absolute shitfest quality of their software. So no, I'm not buying it.
Re:Apple: Transformer architecture optimized for Apple Silicon by lunatuna on Friday March 24, 2023 @12:30AM (#35285097)
Exactly. You can see Apple struggle to build anything of quality with their software stack. Good luck making anything from this. So much great HW sadly wasted.
Re:Apple: Transformer architecture optimized for Apple Silicon by yyyk on Friday March 24, 2023 @07:17AM (#35287545)
> Apple has a ridiculous, almost unfathomably deep moat for.. running personalised...

Apple has decent hw but no LLM software. Unfortunately, in LLM space software changes are the ones driving performance for now, since the space hasn't stabilized yet. Since their competitors control the software, they get to adapt it to their hardware. That is, Google and Microsoft are going to adapt GPT/Bard to Qualcomm ARM etc. while Apple is being ignored. Unless Apple gets in with their own LLM (quite possible), their hw advantage will end up not mattering one bit.

Re:Apple: Transformer architecture optimized for Apple Silicon by zitterbewegung on Friday March 24, 2023 @08:16AM (#35288025)
They might not even require the need to train. There are many open implementations of models that don’t have an academic requirement. Right now you can use tensorflow.js which runs on the gpu using webgl of a phone. If you build the API out developers will use it.
Re:Apple: Transformer architecture optimized for Apple Silicon by josephjrobison on Friday March 24, 2023 @02:45PM (#35293696)
In addition, Apple is notoriously slower at adopting the most cutting edge technologies but historically they tend to consider things very thoughtfully and rolls them out in beautiful launches (not always).

Examples would be the fact that Samsung almost always beats them to the punch on camera technology and other whiz-bang features, but Apple eventually adopts to much mass market consumer acclaim and groans from Android techies. Another example is they’re just now considering touch screen on laptops.

A counter example is the TouchBar - which was “innovative” but many didn’t like.

Re:Apple: Transformer architecture optimized for Apple Silicon by waffletower on Saturday March 25, 2023 @02:46AM (#35300129)
iCloud has screamed ineptitude for decades. I don't see how Apple can possibly develop successful products that can't be held in the hand and fetishized by myopic designers. Much like a stereotypical grandparent raised during the Great Depression, Tim Cook still leads like a supply chain miser trying to save a company that has long prospered -- they tragically squander the unprecedented capital at their disposal. Change my mind, Apple please.