News for hackers. Stuff that matters.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @08:27AM (#42622080)
Yes, but people already had their Macs for others reasons.

No one goes to an Apple store thinking "I'll get a laptop to do AI inference".

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by JohnBooty on Tuesday January 07, 2025 @09:06AM (#42622421)
They have, because until now Apple Silicon was the only practical way for many to work with larger models at home because they can be configured with 64-192GB of unified memory. Even the laptops can be configured with up to 128GB of unified memory.

Performance is not amazing (roughly 4060 level, I think?) but in many ways it was the only game in town unless you were willing and able to build a multi-3090/4090 rig.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @11:14AM (#42624005)
I would bet that people running LLMs on their Macs, today, is <0.1% of their user base.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by sroussey on Tuesday January 07, 2025 @01:02PM (#42625314)
People buying Macs for LLMs—sure I agree.

Since the current MacOS comes built in with small LLMs, that number might be closer to 50% not 0.1%.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @03:59PM (#42627383)
I'm not arguing whether or not Macs are capable of doing it, but whether is a material force that drives people to buy Macs because of it; it's not.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by justincormack on Tuesday January 07, 2025 @03:05PM (#42626764)
Higher than that buying the top end machines though, which are very high margin
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by throwaway48476 on Tuesday January 07, 2025 @04:25PM (#42627711)
All macs? Yes. But of 192GB mac configs? Probably >50%
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by the_other on Tuesday January 07, 2025 @08:55AM (#42622296)
I'm currently wondering how likely it is I'll get into deeper LLM usage, and therefore how much Apple Silicon I need (because I'm addicted to macOS). So I'm some way closer to your steel man than you'd expect. But I'm probably a niche within a niche.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by com2kid on Tuesday January 07, 2025 @10:33AM (#42623427)
Tons of people do, my next machine will likely be a Mac for 60% this reason and 40% Windows being so user hostile now.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by kelsey98765431 on Tuesday January 07, 2025 @09:27AM (#42622639)
my $5k m3 max 128gb disagrees
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @11:12AM (#42623970)
Doubt it, a year ago useful local LLMs on a Mac (via something like ollama) was barely taking off.

If what you say it's true you were among the first 100 people on the planet who were doing this; which btw, further supports my argument on how extremely rare is that use case for Mac users.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by sroussey on Tuesday January 07, 2025 @01:04PM (#42625331)
No, I got a MacBook Pro 14”with M2 Max and 64GB for LLMs, and that was two generations back.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by kgwgk on Tuesday January 07, 2025 @05:26PM (#42628423)
People were running llama.cpp on Mac laptops in March 2023 and Llama2 was released in July 2023. People were buying Macs to run LLMs months before M3 machines became available in November 2023.