Was anyone expecting anything new?
Apple has never been big on living at the cutting edge of technology exploring spaces that no one has explored before—from laptops to the iPhone to iPads to watches, every success they've had has come from taking tech that was already prototyped by many other companies and smoothing out the usability kinks to get it ready for the mainstream. Why would deep learning be different?
I think he is pointing out for people interested in research.
OTOH, it is interesting to see how a company is applying AI to customers at the end. It will bring up new challenges that will be interesting from at least an engineering point of view.
I'm not sure I understand... Considering apple makes the ipad, wouldn't all ipad calculators other than apple's be 3rd party by definition?
Then laugh at Samsung and their flagship line of phones as well, since they haven't had headphone jacks for a while now. "After Note 10 dumps headphone jack, Samsung ads mocking iPhone dongles disappear" (2019):
* https://www.cnet.com/tech/mobile/after-removing-headphone-ja...
"Samsung is hiding its ads that made fun of Apple's removal of headphone jack":
* https://www.androidauthority.com/samsung-headphone-jack-ads-...
> Then laugh at Samsung and their flagship line of phones as well, since they haven't had headphone jacks for a while now. "After Note 10 dumps headphone jack, Samsung ads mocking iPhone dongles disappear" (2019):
I totally do. One of the problems with Apple is the industry seems to mindlessly ape their good and bad decisions. Their marketing has been so good, many people just assume whatever they do must be the best way.
That's not a problem with Apple.
Do you have a crystal ball that lets you know ahead of time which choices are good and bad? Even in retrospect I’m not sure Apple made the wrong choice with the lightning connector. It’s a better connector in just about every way than micro-usb, which was the only standard alternative at the time. Apple’s experience with lightning was rolled into the design process for usb-c, which as I understand it they were heavily involved in. USB-c might not exist as it does today without Apple’s experiments with the lightning connector.
Even if we pretend you’re better at picking winners and losers in the tech world than Apple and Samsung, do you think regulators are going to be as canny as you are with this stuff? US politicians don’t seem to understand the difference between Facebook and the internet. Are you sure you want them making tech choices on your behalf?
If you ask me, I think regulators would make a dogs breakfast of it all. If they were involved we’d probably still have laws on the books mandating that all laptops still have parallel ports or PCMCIA slots or something. The free market can sure take its time figuring this stuff out. But competition does, usually, lead to progress.
You mean like forcing every phone maker to use the awesome connecter known as micro USB-B?
* https://en.wikipedia.org/wiki/Common_external_power_supply
That's hardly Apple's fault, but it is so annoying. Hardly any of my cables have proper reinforcement sleeves any more.
Apple started making awful cables that snapped at the plugs, and everybody else just copied them.
And maybe I'm wrong, but somehow it feels each improvement like that was actually pioneered by Apple. In the dreamworld of free-market enthusiasts this should have made Apple bankrupt or iPhone a very niche consumer device, but in the real world everything just became iPhone. There are some rare exceptions, but these are either outright experimental and gimmicky (because being different is their identity), or just bottom-of-the-line products that have these "intentional defects" that should make you chose the more expensive option.
But that 3.5mm port takes up a lot of room that could be used for more battery, backup antennas for when the user's hand is covering one of them, vibration motors etc.
What you say is also true: many people weren't ready to ditch the old when Apple decide to deprecate it.
(People look down on the move to usb-c which I don’t quite get, everyone seems to fawn over usb-c in other contexts but macbooks, amirite!?!?. Yes it’s nice to have a hdmi port but fundamentally if you buy into this vision that usb-c does everything but you also want to use a bunch of legacy ports (vs thunderbolt video, thunderbolt networking, etc) then obviously you’re going to have to have dongles, and people supposedly buy into that vision in other contexts. Apples implementation of that vision was fundamentally at least a decade ahead of the curve there, if you’re going to do that you want lots of ports and you want every port to do everything, not “this one is the only one that can charge fast”, “that one doesn’t have video output”, “if you run both ports they drop to some weird lower capability because you’re dividing the controller”. Those complaints are the things people don’t like about the base-tier M-series processors today, and apple’s previous models solved that problem long before anyone else did.)
Hell until very very recently a lot of the time the competition didn’t even have thunderbolt/Pcie tunnelling… you got 10gbps usb-c and a grab-bag of charging and display features, and you’re gonna like it. That’s still the case with motherboards and it’s literally only with this years’ release that we’re finally getting usb4/thunderbolt as standard on high-end boards. Literally more than a decade from when apple started putting thunderbolt on laptops, almost a decade from the era of 4x tb3 full-spec ports.
… and reminder that in classic usb fashion, usb4 still doesn’t even guarantee Pcie tunneling support. So really it can still be just a normal usb-c 10gbps connection in a silly mustache and trench coat. Even on the next-gen stuff. What’s the term for doing an ok, moderately competent but not even exceptionally good job while your competition repeatedly shoots themselves in the face, again? But it’s by design - the intent is deceiving and manipulating the customer into buying last year’s junk, it’s working as intended for USB-IF’s real customers and stakeholders.
Intel says hi.
> People look down on the move to usb-c which I don’t quite get
People loved Thunderbolt for replacing Firewire. They hated Apple's choice because these USB-C Macbooks shipped with precisely zero USB-A ports and relegated every user to carrying around a dongle.
The year is 2024, we're almost a decade out from Apple going all-in on USB-C and the predominant peripheral connector is still type-A. I don't like it either, but plugging our ears and pretending like it's not a problem is silly and only makes consumers mad.
> and reminder that in classic usb fashion, usb4 still doesn’t even guarantee Pcie tunneling support.
That is in fact the correct default to use. Ever heard of Thunderspy? https://thunderspy.io/
Intel helped make this move possible, but it doesn’t manufacture laptops. Apple took the heat for “donglegate”.
On the x86 desktop, usb-c is still surprisingly rare. I think my motherboard (that’s less than a year old) only has 2 usb-c ports and 8x usb-a.
And rightfully so. They took Intel's technology and told an unprepared and uninterested industry to switch or die. Naturally, very few manufacturers switched over and Apple's all-or-nothing strategy made more people mad than happy.
Having 4 lanes of Thunderbolt connectivity is awesome. It doesn't really fix the fact that none of them can easily connect to a wired keyboard or mouse.
> On the x86 desktop, usb-c is still surprisingly rare.
My motherboard only has one TB connector, everything else is type-A too. Most of the bandwidth is broken out over SATA or PCIe internally, and frankly I don't regret it one bit. 99% of my life, there is nothing plugged into that Thunderbolt port.
From https://en.wikipedia.org/wiki/AirPort:
"AirPort 802.11b card"
"The original model, known as simply AirPort card, was a re-branded Lucent WaveLAN/Orinoco Gold PC card, in a modified housing that lacked the integrated antenna."
That would be a very un-Apple thing to do. They really like to use their own marketing terms for technologies. It's not ARM, it's Apple Silicon. It wasn't Wi-Fi, it was AirPort. etc. etc.
FWIW the term “airport” predated the name “wifi” — in those days you had to otherwise call it IEEE 802.11.
And the name as great: people were buying them like crazy and hiding them in the drop ceiling to get around the corporate IT department. A nice echo of how analysts would buy their own apple II + visicalc to…get around corporate IT.
I’m OK with Apple using “apple silicon” as the ARM is only part of it.
Just commenting on your two examples; in general I agree with your point.
WECA reminds me of other "memorable" names of the era, my favorite being PCMCIA, though VESA is another fave.
Wikipedia: FireWire is Apple's name for the IEEE 1394 High Speed Serial Bus. Its development was initiated by Apple[1] in 1986,[3] and developed by the IEEE P1394 Working Group, largely driven by contributions from Sony (102 patents), Apple (58 patents), Panasonic (46 patents), and Philips (43 patents), in addition to contributions made by engineers from LG Electronics, Toshiba, Hitachi, Canon,[4] INMOS/SGS Thomson (now STMicroelectronics),[5] and Texas Instruments.
What might be interesting in this regard is that Sony was also using its own trademark for it: "i.LINK".
Except in Japan, where it's AirMac. And China, where it's WWAN not Wi-Fi.
There was such a time. Same as with Google. Interestingly, around 2015-2016 both companies significantly shifted to iterative products from big innovations. It's more visible with Google than Apple, but here's both.
Apple:
- Final Cut Pro
- 1998: iMac
- 1999: iBook G3 (father of all MacBooks)
- 2000: Power Mac G4 Cube (the early grandparent of the Mac Mini form factor), Mac OS X
- 2001: iPod, iTunes
- 2002: Xserve (rackable servers)
- 2003: Iterative products only
- 2004: iWork Suite, Garage Band
- 2005: iPod Nano, Mac mini
- 2006: Intel Macs, Boot Camp
- 2007: iPhone and Apple TV
- 2008: MacBook Air, iPhone 3G
- 2009: iPhone 3Gs, all-in-one iMac
- 2010: iPad, iPhone 4
- 2011: Final Cut Pro X
- 2012: Retina displays, iBooks Author
- 2013: iWork for iCloud
- 2014: Swift
- 2015: Apple Watch, Apple Music
- 2016: Iterative products only
- 2017: Iterative products mainly, plus ARKit
- 2018: Iterative products only
- 2019: Apple TV +, Apple Arcade
- 2020: M1
- 2021: Iterative products only
- 2022: Iterative products only
- 2023: Apple Vision Pro
Google:
- 1998: Google Search
- 2000: AdWords (this is where it all started going wrong, lol)
- 2001: Google Images Search
- 2002: Google News
- 2003: Google AdSense
- 2004: Gmail, Google Books, Google Scholar
- 2005: Google Maps, Google Earth, Google Talk, Google Reader
- 2006: Google Calendar, Google Docs, Google Sheets, YouTube bought this year
- 2007: Street View, G Suite
- 2008: Google Chrome, Android 1.0
- 2009: Google Voice, Google Wave (early Docs if I recall correctly)
- 2010: Google Nexus One, Google TV
- 2012: Google Drive
- 2013: Chromecast
- 2014: Android Wear, Android Auto, Google Cardboard, Nexus 6, Google Fit
- 2015: Google Photos
- 2016: Google Assistant, Google Home
- 2017: Mainly iterative products only, Google Lens announced but it never rolled out really
- 2018: Iterative products only
- 2019: Iterative products only
- 2020: Iterative products only, and some rebrands (Talk->Chat, etc)
- 2021: Iterative products only, and Tensor Chip
- 2022: Iterative products only
- 2023: Iterative products only, and Bard (half-baked).
Also small correction but iTunes (as Soundjam MP) was originally third-party software and Final Cut was acquired by Apple.
About iTunes: I did not know that! Thank you.
About iterative/innovative: I considered hardware and software that became household names or general knowledge to be significant innovations. It is not rigorous, I tried to include more rather than less. Still, on some years these companies mostly did version increases for their hardware and software, like new iOS and macOS versions, and that was it. Those years I marked as iterative.
I included a few too many iPhones, although when I wrote that, my thought process was that these phones were pivotal to how iPhones developed. I should have included the original iPhone, and iPhone 3G — the first iPhone developed around the concept of an app platform and with an App Store. This has undoubtedly been a big innovation. iPhone 4 and 3Gs, perhaps, should not have been included.
It's loose and just to illustrate a general trend, individual items are less important, we could all pick slightly different ones. But I believe the trend would remain.
If anything, that would be AMD. But I'd guess they're both more worried about the entire desktop & laptop market shrinking way more than anything Apple does.
Note that I'm not saying that there's anything wrong with their approach or that they didn't make real improvements. I'm just saying that Apple has never produced any successful product that would count as "new" to someone interested in cutting-edge research. They've always looked around at things that exist but aren't yet huge successes and given them the final push to the mainstream.
I used the definition of new somewhere between "new for most people", "newly popular", and "meaningfully advanced from the previous iteration". With such a definition, I think you can agree with me.
The Next Cube being a very obvious inspiration here.
Mac OS X was a needed step, and entire books have been written about its creation. While it has some innovative pieces, it was very much a do or die situation for Apple, not brought on by innovation so much as the need to survive. (I'm sure BeOS fans will argue that BeOS was the real innovative OS. ;) )
> - 2001: iPod, iTunes
iPod was a, very well done, refinement of the existing MP3 device category. iTunes' innovation was the licensing deal they got with record companies, that is what really surprised everyone.
> - 2002: Xserve (rackable servers)
Not sure how this is an innovation? Rack mount servers had been around a long time.
> - 2004: iWork Suite,
Microsoft literally had a product called Microsoft Works that was originally released in 1988 and came shipped on tons of home PCs.
> - 2005: iPod Nano, Mac mini
The iPod Nano was cool, the Mac mini was a wonderful feat of engineering and cost reductions.
> - 2006: Intel Macs, Boot Camp
Necessity brought this about.
> - 2007: iPhone and Apple TV
This is a perfect example of Apple entering an existing product category and doing an amazing job of execution. Palm, Blackberry, and Microsoft were already releasing very capable smart phones, but none of them bothered polishing the product (MS and Blackberry focused on corporate sales, end user experience was not the top priority) and while Apple did push a lot of technology forward to make the iPhone (notably screen tech and using capacitive touch screens), their main innovation was realizing they could get customers to pay for a cell phone. For those who don't remember, prior to the iPhone, most customers got their cellphone for "free" from their cellular provider in return for agreeing to a 2 year contract. Apple realized if they made a really nice product, that people would buy it.
Apple also did some really cool, and now largely forgotten about, positioning here involving the iPod Touch, where the iPod Touch had access to the full App Store and became entry level "kids toy" devices that got people into the ecosystem.
Heck arguably the App Store was a larger innovation than the phone.
Fun fact: Microsoft had an App Store ready to launch for Windows Mobile (pre Windows Phone 7) but it was scrapped at the last minute because an exec thought that "no way would phone users ever pay for apps".
(When I joined MS in 2006 the source code was still laying around in the Windows Mobile source tree!)
Apple TV was arguably too early at this point in time, I'd say it didn't really take off until later generations when more streaming media was available.
But innovative? Web TV was out in the late 90s (!!) and Microsoft tried to do Media Center PC's since 2002. Heck for awhile with Xbox 360, Microsoft basically owned the "TV smart device" market segment. (and they released the Xbox One as a media streaming device and sort of forgot that it was also a games console... oops)
As with most products, Apple just did a really good job of it, but Roku has dramatically outplayed everyone else in the market by getting embedded directly into cheap TVs sold at Costco.
> - 2010: iPad, iPhone 4
iPad is/was an amazing product, and it succeeded thanks to great apps.
It was also a refinement of Tablet PCs which have been around since the late 80s/early 90s.
Apple was willing to do what Microsoft wasn't, break all back-compat and make a really good single purpose device. Microsoft's tablets (some of which are really damn nice!) were always hamstrung because Microsoft never could go all in on abandoning existing x86 software. (The closest attempt being Windows 8 RT, which managed to make the perfect set of compromises to anger everyone!)
> - 2015: Apple Watch, Apple Music
The first generation Apple Watch was... meh. Now, I say this as someone who was working on a direct competitor - I am still not sure how it has such a miserable battery life and why such a massively overpowered CPU and GPU still dropped frames.
I am not sure what is innovative about Apple Music, vs every other streaming music service.
But this is exactly the type of marketing Apple is good at, though "retina" is probably not the most successful example.
One of the album covers being stretched had some kind of fine pattern on it that caused a clearly visible shifting/flashing Moiré pattern as it was being stretched.
Wish I could remember what album cover it was now.
Though really it's simple enough: As long as you can still spot a single dark pixel in the middle of an illuminated white screen, the pixels could benefit from being smaller. (Edit: swapped black and white)
20/10 is rare but can easily be corrected to with glasses or contacts.
You also left that "intended viewing distance" hanging there, without at all acknowledging what that is at a minimum?
Still remember the hard time using Apple newton in a conference vs the palm freely on loan in a Gartner group conference. Palm solved a problem, even though not very Apple … user can input on a small device. I kept it, on top of my newly bought newton.
It is the user …
The term "resolution" transitioned gradually to mean number-of-individual-lines-rendered horizontally and vertically for displays, but really, the idea is how many dots can you "resolve" (or "resolving power"): a "high resolution" screen or screen mode had a larger number of individual pixels being drawn, which meant that the density is higher. You never talk about a scanner having a resolution of 7200x3600 even if it can scan 12"x6" at 600dpi.
So really, in an informal conversation, I believe both are fine. If you want to be extremely precise, you missed the mark: width and height in pixels is the sanest way to call what you refer to as "resolution".
> For device displays such as phones, tablets, monitors and televisions, the use of the term display resolution as defined above is a misnomer, though common. The term display resolution is usually used to mean pixel dimensions, the maximum number of pixels in each dimension (e.g. 1920 × 1080), which does not tell anything about the pixel density of the display on which the image is actually formed: resolution properly refers to the pixel density, the number of pixels per unit distance or area, not the total number of pixels. In digital measurement, the display resolution would be given in pixels per inch (PPI).
Or Intel calling USB4 devices and cables which meet quality and feature requirements 'Thunderbolt 5'
Compared to, say, manufacturers who aren't willing to meet any certification requirements or to properly implement the standards at play saying they have "USB-A 3.2 2x2 ports" on their motherboards.
Retina doesn't carry the same weight as an industry certification effort like thunderbolt, but it still informs people that a screen actually meets some sort of bar without them having to evaluate pages of tech specs, and reviews saying whether the tech specs are accurate or have undocumented caveats.
Finally, establishing such certifications are difficult - look at the number of failed attempts at creating industry quality/feature marks in the television market.
Rather than just pre-baking static LoRAs to ship with the base model (e.g. one global "rewrite this in a friendly style" LoRA, etc), Apple seem to have chosen a bounded set of behaviors they want to implement as LoRAs — one for each "mode" they want their base model to operate in — and then set up a pipeline where each LoRA gets fine-tuned per user, and re-fine-tuned any time the data dependencies that go into the training dataset for the given LoRA (e.g. mail, contacts, browsing history, photos, etc) would change.
In other words, Apple are using their LoRAs as the state-keepers for what will end up feeling to the user like semi-online Direct Preference Optimization. (Compare/contrast: what Character.AI does with their chatbot response ratings.)
---
I'm not as sure, from what they've said here, whether they're also implying that these models are being trained in the background on-device.
It could very well be possible: training something that's only LoRA-sized, on a vertically-integrated platform optimized for low-energy ML, that sits around awake but doing nothing for 8 hours a day, might be practical. (Normally it'd require a non-quantized copy of the model, though. Maybe they'll waste even more of your iPhone's disk space by having both quantized and non-quantized copies of the model, one for fast inference and the other for dog-slow training?)
But I'm guessing they've chosen not to do this — as, even if it were practical, it would mean that any cloud-offloaded queries wouldn't have access to these models.
Instead, I'm guessing the LoRA training is triggered by the iCloud servers noticing you've pushed new data to them, and throwing a lifecycle notification into a message queue of which the LoRA training system is a consumer. The training system reduces over changes to bake out a new version of any affected training datasets; bakes out new LoRAs; and then basically dumps the resulting tensor files out into your iCloud Drive, where they end up synced to all your devices.
They didn't say that this is an "AI thing", but I can't honestly see how else you'd do it other than by fine-tuning a vision model on the user's own handwriting.
I can't picture any way to use a RAG to do that.
I can picture a way to do that that doesn't involve any model fine-tuning, but it'd be pretty ridiculous, and the results would probably not be very good either. (Load a static image2text LoRA tuned to describe the subjects of photos; run that once over each photo as it's imported/taken, and save the resulting descriptions. Later, whenever a photo is classified as a particular subject, load up a static LLM fine-tune that summarizes down all the descriptions of photos classified as subject X so far, into a single description of the platonic ideal of subject X's appearance. Finally, when asked for a "memoji", load up a static "memoji" diffusion LoRA, and prompt it with the that subject-platonic-appearance description.)
But really, isn't it easier to just fine-tune a regular diffusion base-model — one that's been pre-trained on photos of people — by feeding it your photos and their corresponding metadata (incl. the names of subjects in each photo); and then load up that LoRA and the (static) memoji-style LoRA, and prompt the model with those same people's names plus the "memoji" DreamBooth-keyword?
(Okay, admittedly, you don't need to do this with a locally-trained LoRA. You could also do it by activating the static memoji-style LoRA, and then training to produce a textual-inversion embedding that locates the subject in the memoji LoRA's latent space. But the "hard part" of that is still the training, and it's just as costly!)
> ...each LoRA gets fine-tuned per user...
Apple would not implement these sophisticated user specific LoRA training techniques without mentioning them anywhere. No big player has done anything like this and Apple would want the credit for this innovation.
> The adapter models can be dynamically loaded, temporarily cached in memory, and swapped — giving our foundation model the ability to specialize itself on the fly for the task at hand
This along with other statements in the article about keeping the base model weights unchanged says to me that they are simply swapping out adapters on a per app or per task basis. I highly doubt they will fine tune adapters on user data since they have taken a position against this. I wonder how successful this approach will be vs merging the adapters with the base model. I can see the benefits but there are also downsides.
And if Karpathy thinks so then I assume it's good enough for HN:
b) Apple only provides information to ChatGPT when the user consents to doing so and the information is only for that request i.e. it is not logged for future training.
Maybe it's my fault as a reader, but I think the writing could be clearer. Usually in a research paper you would link to the LoRA paper there too.
Besides I could do "named person on a beach in August" and get the correct thing in photos on Android photos, so I don't get it.
It's amazing for apple users if they didn't have it before. But from a tech stand point people could have had it for a while.
The current addition is "just" about adding a natural language interface on top of data they already have about your photos (on device, not in the cloud).
My iPhone 14 can, for example, detect the breed of my dog correctly from the pictures and it can search for a specific pet by name. Again on-device, not by sending my stuff to Google's cloud to be analysed.
It's a trade-off between getting the features I need and the price I have to pay. All else being equal I do prefer privacy as well. Unfortunately, all else is not equal.
>I prefer Apple's privacy focused option myself.
It's only an option if it works.
Ref: https://worldpopulationreview.com/country-rankings/iphone-ma...
Poland, Greece, Hungary and Bosnia-Herzegovina are the only ones under 20% (and maybe a few others).
OTOH Britain is over 50% as is Sweden. Finland, the land of Nokia is over 35%.
Yes. Americans are THE most valuable customer base, y'all use insane amounts of money on mobile crap.
Perhaps there is still hope of a relaunch of xserve; with the widespread use of Apple computers amongst developers Apple has a real chance of challenging NVIDIA's CUDA moat.
Quadro and Tesla cards might be a different story. I would still like to see concrete FLOPS/$ numbers.
Even their high-end prosumer hardware could be interesting as an AI workstation given the VRAM available if the software support were better.
Idk every business I’ve worked and all the places my friends work seem to be 90% Apple hardware, with a few Lenovo issued for special case roles in finance or something.
Also that a significant proportion (majority?) of them will have just 8 GB of memory which is not exactly sufficient to run any complex AI/ML workloads.
I expect OS's will expose an API which, when queried, will indicate the level of AI inference available.
Similar to video decoding/encoding where clients can check if hardware acceleration is available.
4090 peaks out at around 550w which means they can run 5+ of their Max chips in the same power budget.
A 4090 is $2000. Apple can probably get 5 chips on a custom motherboard for that cost. They'll use the same amount of power, but get a lot more raw compute.
Part of the advantage of using "one 4090" is that the max TDP is only 450w, as opposed to 5 M2 Ultras running at ~150w each. When you scale up to Nvidia's latest Blackwell architecture, I genuinely don't know how Apple could beat them on performance-per-watt. Buying M2 Ultras wholesale is probably cheaper than an NVL72 cluster, but certainly not what you'd want to use for Linux or maximizing AI-based performance-per-watt.
The Max TDP is not actual peak power consumption. Gamer's nexus recorded 500w peak and almost 670w overclocked. Most reviews I've looked at seem to put peak power consumption around 550w.
M2 Ultra wasn't even mentioned and it uses more than 150w. The correct question would be about M3 Max as we have solid numbers on it. M3 Max uses around 100w when both the GPU and CPU are heavily utilized and less than that when only the GPU is used.
This means that Apple could run 5 of their M3 Max chips in the same peak power as the 4090. But wait, there's more. 4090 doesn't run in a vacuum. It requires a separate CPU setup and a couple hundred more watts.
That means we could power 7 or so M3 Max chips with that same amount of power.
Of course, this isn't the whole story. 4090 isn't a professional chip either (while Apple can bin and certify their own CPUs and know they're getting a server-grade chip) and the 4090 also doesn't have nearly enough RAM. H100 starts at $25,000 and goes up. Apple could buy 75-100 M3 Max chips for that kind of money. That's certainly a load more compute than H100 would offer. Blackwell will be even more expensive in comparison.
but you also need the bandwidth to feed those units. If LLMs are your game, you need a lot of RAM and a lot of bandwidth. The 4090 can't really stretch their tensor core muscle in this scenario. That's why real ML datacenter Nvidia tech uses much wider RAM interfaces and a considerably different price point. While the M2 Ultra does not have as much
That's true. I was talking about end user pricing.
> ...each chip can do more than a 4090 because it also has a CPU onboard.
That's a strange thing to say. It has a CPU, correct. It makes the chip more versatile but for data center ML tasks it doesn't really matter. A 4090 chip also has much more ML relevant compute per chip. So apple's chips can't really "do more than a 4090" in any relevant way.
Of course apple pays less for their in house made chips vs external products. That comparison doesn't seem relevant to the context, e.g. they're not going to challenging CUDA with internal chips.
They might get more compute per watt though. My guess is that nvidias datacenter chips are competitive in that space, but that's another story.
Should we keep arguing like on school playground?
Crazy times.
Just want to point out I call this launch huge, didn’t say “huge development” as quoted, and didn’t imply what was interesting was the ML research. No one in this thread used the quoted words, at least that I can see.
My comment was about dev experience, memory swapping, potential for tuning base models to each HW release, fine tune deployment, and app size. Those things do have the potential to be huge for developers, as mentioned. They are the things that will make a local+private ML developer ecosystem work.
I think the article and comment make sense in their context: a developer conference for Mac and iOS devs.
Apple also explicitly says it’s LoRA.
This is the important part.
My advisor said new means old method applied to new data or new method on old data.
Commercially, that means price points, i.e., discrete points where something becomes viable.
Maybe that's iterative, but maybe not. Either way, once the opportunity presents, time is of the essence.
(So there is room left if you're limited by memory or budget.)
* Clearly outlining their intent/policies for training/data use. Committing to no using user data or interactions for training their base models is IMO actually a pretty big deal and a differentiator from everyone else.
* There's a never-ending stream of new RL variants ofc, but that's how technology advances, and I'm pretty interested to see how these compare with the rest: "We have developed two novel algorithms in post-training: (1) a rejection sampling fine-tuning algorithm with teacher committee, and (2) a reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator. We find that these two algorithms lead to significant improvement in the model’s instruction-following quality."
* I'm interested to see how their custom quantization compares with the current SoTA (probably AQLM atm)
* It looks like they've done some interesting optimizations to lower TTFT, this includes the use of some sort of self-speculation. It looks like they also have a new KV-cache update mechanism and looking forward to reading about that as well. 0.6ms/token means that for your average I dunno, 20 token query you might only wait 12ms for TTFT (I have my doubts, maybe they're getting their numbers from much larger prompts, again, I'm interested to see for myself)
* Yes, it looks like they're using pretty standard LoRAs, the more interesting part is their (automated) training/re-training infrastructure but I doubt that's something that will be shared. The actual training pipeline (feedback collection, refinement, automated deployment) is where the real meat and potatoes of being able to deploy AI for prod/at scale lies. Still, what they shared about their tuning procedures is still pretty interesting, as well as seeing which models they're comparing against.
As this article doesn't claim to be a technical report or a paper, while citations would be nice, I can also understand why they were elided. OpenAI has done the same (and sometimes gotten heat for it, like w/ Matroyshka embeddings). For all we know, maybe the original author had references, or maybe since PEFT isn't new to those in the field, that describing it is just being done as a service to the reader - at the end of the day, it's up to the reader to make their own judgements on what's new or not, or a huge development or not. From my reading of the article, your conclusion, which funnily enough is now the new top-rated comment on this thread isn't actually much more accurate the the one old one you're criticizing.
Quick straw poll survey around the office, many think their data will be sent off to OpenAI by default for these new features which is not the case.
They seem to have a good model for adding value to their products without the hold my beer, conquer the world bullshit that you get from OpenAI, et al.
If they try to market it with a seemingly unique or yet-unheard of name, then yeah. It is nice knowing what the "real world" name of an Apple-ized technology is.
Just ignoring it and marketing the technology under some new name is adjacent to lying to your audience through omission.
They don't market technology, they market solutions. E.g. afib detection on Apple Watch, rather than calling it a BNNS using a custom-built sensor for one-wire EKG.
This is the document where they describe how the solution works, and they clearly state adapters work based on LoRA.
IMO they should do a better job of referencing existing papers and techniques. The way they wrote about "adaptors" can make it seem like it's something novel, but it's actually just re-iterating vanilla LoRA. It was enough to convince one of the top-voted HackerNews comments that this was a "huge development".
Benchmarks are nice though.