News for hackers. Stuff that matters.
Re:Ferret: A Multimodal Large Language Model by Apple by tambourine_man on Saturday December 23, 2023 @12:49PM (#38746256)
Apple doesn’t even sell NVidia cards on their Mac Pros. Are they training it on Linux?

I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried.

AI/LLM is the new hot thing. If people are using Windows or Linux, you’re loosing momentum, hearts and minds… and sales, obviously.

Re:Ferret: A Multimodal Large Language Model by Apple by hermannj314 on Saturday December 23, 2023 @12:59PM (#38746365)
If a train with a GM diesel engine delivers raw materials to a Ford factory for making F150s, you would conclude that consumers whould start driving trains to work?

Is that your argument?

Re:Ferret: A Multimodal Large Language Model by Apple by tambourine_man on Saturday December 23, 2023 @01:05PM (#38746427)
Not at all, just that the engineers at Ford would be more proud if the train used their own diesel engine. And that this kind of thing affects public perception. “Ford is not for heavy duty”
Re:Ferret: A Multimodal Large Language Model by Apple by shepherdjerred on Saturday December 23, 2023 @03:49PM (#38748084)
> I think Apple would strive to be great at all computing related tasks. “Oh, Macs are not good for that, you should get a PC” should make them sad and worried.

What percent of Apple's customers train models? Does it even crack 1%?

Apple already fails for many types of computing, e.g. any workflow that requires Windows or AAA gaming.

Re:Ferret: A Multimodal Large Language Model by Apple by tambourine_man on Saturday December 23, 2023 @08:50PM (#38750311)
Right, and they are trying to attract games and gamers. “Macs are bad at gaming” shouldn’t be something they settle with.
Re:Ferret: A Multimodal Large Language Model by Apple by nicolas_17 on Saturday December 23, 2023 @01:01PM (#38746382)
Apple doesn't even support NVidia cards on their Mac Pros. The technical details are above my head, but the way Apple M* chips handle PCIe make them incompatible with GPUs and other accelerator cards. Whether you use macOS or Linux.
Re:Ferret: A Multimodal Large Language Model by Apple by causal on Saturday December 23, 2023 @12:59PM (#38746368)
Don't think that follows. I doubt all their cloud services run on Apple hardware. They make consumer devices.
Re:Ferret: A Multimodal Large Language Model by Apple by nicolas_17 on Saturday December 23, 2023 @01:03PM (#38746411)
AFAIK all their cloud services run on x86 hardware with Linux, including Xcode Cloud (which runs macOS in a QEMU VM, in a way that only Apple can technically and legally do).
Re:Ferret: A Multimodal Large Language Model by Apple by threeseed on Saturday December 23, 2023 @02:31PM (#38747306)
It has always been x86 Linux going back to the NeXT days.

Today they have large scale Kubernetes ands some legacy Mesos clusters.

Re:Ferret: A Multimodal Large Language Model by Apple by bjtitus on Saturday December 23, 2023 @02:28PM (#38747278)
This

They ceded the data center environment years ago.

Re:Ferret: A Multimodal Large Language Model by Apple by Gorgor on Saturday December 23, 2023 @12:54PM (#38746317)
But no one is training these kinds of models on their personal device. You need compute clusters for that. And they will probably run Linux. I'd be surprised if Microsoft trains their large models in anything else than Linux clusters.
Re:Ferret: A Multimodal Large Language Model by Apple by tambourine_man on Saturday December 23, 2023 @12:57PM (#38746339)
Apple used to sell servers. I don’t thing they should settle for “just use Linux” in such and important field.
Re:Ferret: A Multimodal Large Language Model by Apple by MBCook on Saturday December 23, 2023 @03:09PM (#38747682)
Why does the OS matter for training models?

Apple would want to train models as fast as they could. Nvidia provides an off the shelf solution they can just buy and use for a very reasonable price and sell on the second hand market.

If they wanted to use their own hardware they would either need more of it, which would cost a lot and divert production from sellable devices; or they would need to make special chips with much bigger neural engines, which would cost even more.

Also Apple uses public clouds for service stuff. They may not even own any hardware and just be renting it from AWS/Azure/GCP for training.

Re:Ferret: A Multimodal Large Language Model by Apple by tambourine_man on Saturday December 23, 2023 @08:52PM (#38750325)
I feel like I’ve answered the issues you raised in this thread already.
Re:Ferret: A Multimodal Large Language Model by Apple by hhh on Sunday December 24, 2023 @10:01AM (#38753868)
> Used to

Exactly, over a decade ago...

Re:Ferret: A Multimodal Large Language Model by Apple by SpaceManNabs on Saturday December 23, 2023 @01:10PM (#38746475)
> But no one is training these kinds of models on their personal device

on-device transfer learning/fine tuning is def a thing for privacy and data federation reasons. Part of the reason why model distillation was so hot a few years ago.

Re:Ferret: A Multimodal Large Language Model by Apple by threeseed on Saturday December 23, 2023 @02:29PM (#38747286)
> Are they training it on Linux

Yes. Apple has always run their servers on Linux e.g. App Store, iTunes.

And training isn't right now an end user activity but something reserved for server farms.

Re:Ferret: A Multimodal Large Language Model by Apple by tjohns on Saturday December 23, 2023 @01:44PM (#38746817)
Apple doesn't even make rack-mount server hardware. It's not that surprising.

Apple makes very capable, efficient devices for end users and content producers. End users do not normally need to train new models.

Re:Ferret: A Multimodal Large Language Model by Apple by danieldk on Saturday December 23, 2023 @04:39PM (#38748495)
They do make rack mount hardware:

https://www.apple.com/shop/buy-mac/mac-pro/rack

Don't know if that qualifies as server :).

Re:Ferret: A Multimodal Large Language Model by Apple by tambourine_man on Saturday December 23, 2023 @08:59PM (#38750363)
They used to sell the best developer’s machines. Before Docker, at least. Don’t developers need to train models? And if they are doing so on Linux or Windows, wouldn’t that make it easier for them to better or exclusively support the end product on the one they’re more used to?

You want to be the platform where the excitement is happening.