News for hackers. Stuff that matters.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by dagmx on Monday January 06, 2025 @11:45PM (#42619339)
I think the enthusiast side of things is a negligible part of the market.

That said, enthusiasts do help drive a lot of the improvements to the tech stack so if they start using this, it’ll entrench NVIDIA even more.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by qwertox on Tuesday January 07, 2025 @12:11AM (#42619479)
You could have said the same about gamers buying expensive hardware in the 00's. It's what made Nvidia big.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Cumpiler69 on Tuesday January 07, 2025 @01:51AM (#42620002)
There's a lot more gamers than people wanting to play with LLms at home.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by estebarb on Tuesday January 07, 2025 @02:06AM (#42620091)
Sure, but those developers will create functionality that will require advanced GPUs and people will want that functionality. Eventually OS will expect it and it will became default everywhere. So, it is an important step that will push nvidia growing in the following years.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by anonylizard on Tuesday January 07, 2025 @02:04AM (#42620074)
There's a titanic market with people wanting some uncensored local LLM/image/video generation model. This market extremely overlaps with gamers today, but will grow exponentially every year.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Cumpiler69 on Tuesday January 07, 2025 @02:05AM (#42620079)
How big is that market you claim? Local LLM image generation already exists out off the box on latest Samsung flagship phones and it's mostly a Gimmick that gets old pretty quickly. Hardly comparable to gaming in terms of market size and profitablity.

Plus, YouTube and the Google images is already full of AI generated slop and people are already tired of it. "AI fatigue" amongst majority of general consumers is a documented thing. Gaming fatigues is not.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by madwolf on Tuesday January 07, 2025 @02:46AM (#42620282)
I think he implied AI generated porn. Perhaps also other kind of images that are at odds with morality and/or the law. I'm not sure but probably Samsung phones don't let you do that.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by TeMPOraL on Tuesday January 07, 2025 @09:29AM (#42622670)
> Gaming fatigues is not.

It is. You may know it as the "I prefer to play board games (and feel smugly superior about it) because they're ${more social, require imagination, $whatever}" crowd.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Cumpiler69 on Tuesday January 07, 2025 @10:17AM (#42623228)
The market heavily disagrees with you.

"The global gaming market size was valued at approximately USD 221.24 billion in 2024. It is forecasted to reach USD 424.23 billion by 2033, growing at a CAGR of around 6.50% during the forecast period (2025-2033)"

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by com2kid on Tuesday January 07, 2025 @10:42AM (#42623573)
Farmville style games underwent similar explosive estimates of growth, up until they collapsed.

Much of the growth in gaming of late has come from exploitive dark patterns, and those dark patterns eventually stop working because users become immune to them.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by mrguyorama on Tuesday January 07, 2025 @01:28PM (#42625629)
>Farmville style games underwent similar explosive estimates of growth, up until they collapsed.

They did not collapse, they moved to smartphones. The "free"-to-play gacha portion of the gaming market is so successful it is most of the market. "Live service" games are literally traditional game makers trying to grab a tiny slice of that market, because it's infinitely more profitable than making actual games.

>those dark patterns eventually stop working because users become immune to them.

Really? Slot machines have been around for generations and have not become any less effective. Gambling of all forms has relied on the exact same physiological response for millennia. None of this is going away without legislation.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by com2kid on Tuesday January 07, 2025 @03:05PM (#42626766)
> Slot machines have been around for generations and have not become any less effective.

Slot machines are not a growth market. The majority of people wised to them literal generations ago, although enough people remain susceptible to maintain a handful of city economies.

> They did not collapse, they moved to smartphones

Agreed, but the dark patterns being used are different. The previous dark patterns became ineffective. The level of sophistication of psychological trickery in modern f2p games is far beyond anything Farmville ever attempted.

The rise of live service games also does not bode well for infinite growth in the industry as there's only so many hours to go around each day for playing games and even the evilest of player manipulation techniques can only squeeze so much blood from a stone.

The industry is already seeing the failure of new live service games to launch, possibly analogous to what happened in the MMO market when there was a rush of releases after WoW. With the exception of addicts, most people can only spend so many hours a day playing games.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by stuaxo on Tuesday January 07, 2025 @04:38AM (#42620868)
Apart from the uncensored bit, I'm in this small market.

Do I buy a Macbook with silly amount of RAM when I only want to mess with images occasionally.

Do I get a big Nvidia card, topping out at 24gb - still small for some LLMs, but I could occasionally play games using it at least.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by otabdeveloper4 on Tuesday January 07, 2025 @05:59AM (#42621277)
> There's a titanic market with people wanting some uncensored local LLM/image/video generation model.

No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by ceejayoz on Tuesday January 07, 2025 @09:03AM (#42622387)
AI porn is currently cringe, just like Eliza for conversations was cringe.

The cutting edge will advance, and convincing bespoke porn of people's crushes/coworkers/bosses/enemies/toddlers will become a thing. With all the mayhem that results.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by otabdeveloper4 on Tuesday January 07, 2025 @11:29AM (#42624219)
It will always be cringe due to how so-called "AI" works. Since it's fundamentally just log-likelihood optimization under the hood, it will always be a statistically most average image. Which means it will always have that characteristic "plastic" and overdone look.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by ceejayoz on Tuesday January 07, 2025 @02:17PM (#42626174)
The current state of the art in AI image generation was unimaginable a few years back. The idea that it'll stay as-is for the next century seems... silly.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by otabdeveloper4 on Wednesday January 08, 2025 @04:07AM (#42632388)
If you're talking about some sort of non-existent sci-fi future "AI" that isn't just log-likelihood optimization, then most likely such a fantastical thing wouldn't be using NVidia's GPU with CUDA.

This hardware is only good for current-generation "AI".

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by JohnBooty on Tuesday January 07, 2025 @09:15AM (#42622503)
I think there are a lot of non-porn uses. I see a lot of YouTube thumbnails that seem AI generated, but feature copyrighted stuff.

(example: a thumbnail for a YT video about a video game, featuring AI-generated art based on that game. because copyright reasons, in my very limited experience Dall-E won't let you do that)

I agree that AI porn doesn't seem a real market driver. With 8 billion people on Earth I know it has its fans I guess, but people barely pay for porn in the first place so I reallllly dunno how many people are paying for AI porn either directly or indirectly.

It's unclear to me if AI generated video will ever really cross the "uncanny valley." Of course, people betting against AI have lost those bets again and again but I don't know.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Filligree on Tuesday January 07, 2025 @10:33AM (#42623432)
> No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast.

I needed an uncensored model in order to, guess what, make an AI draw my niece snowboarding down a waterfall. All the online services refuse on basis that the picture contains -- oh horrors -- a child.

"Uncensored" absolutely does not imply NSFW.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by otabdeveloper4 on Tuesday January 07, 2025 @11:10AM (#42623952)
Yeah, and there's that story about "private window" mode in browsers because you were shopping for birthday gifts that one time. You know what I mean though.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Filligree on Tuesday January 07, 2025 @12:12PM (#42624746)
I really don't. Censored models are so censored they're practically useless for anything but landscapes. Half of them refuse to put humans in the pictures at all.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Paradigma11 on Tuesday January 07, 2025 @11:04AM (#42623850)
I think scams will create a far more demand. Spear Phishing targets by creating persistent elaborate online environments is going to be big.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by itsoktocry on Tuesday January 07, 2025 @07:13AM (#42621615)
>There's a titantic market

How so?

Only 40% of gamers use a PC, a portion of those use AI in any meaningful way, and a fraction of those want to set up a local AI instance.

Then someone releases an uncensored, cloud based AI and takes your market?

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by weregiraffe on Tuesday January 07, 2025 @08:16AM (#42622007)
>There's a titanic market

Titanic - so about to hit an iceberg and sink?

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by JohnBooty on Tuesday January 07, 2025 @09:11AM (#42622468)
I'm sure a lot of people see "uncensored" and think "porn" but there's a lot of stuff that e.g. Dall-E won't let you do.

Suppose you're a content creator and you need an image of a real person or something copyrighted like a lot of sports logos for your latest YouTube video's thumbnail. That kind of thing.

I'm not getting into how good or bad that is; I'm just saying I think it's a pretty common use case.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by spaceman_2020 on Tuesday January 07, 2025 @04:48AM (#42620925)
I keep thinking about stocks that have 100xd, and most seemed like obscure names to me as a layman. But man, Nvidia was a household name to anyone that ever played any game. And still so many of us never bothered buying the stock

Incredible fumble for me personally as an investor

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by bloomingkales on Tuesday January 07, 2025 @07:18AM (#42621650)
I think we don’t fully understand the power of some technologies because society is just lame. Graphics cards were always space age technology. It was just associated with this nerdy gamer culture, so it’s kinda lumped into this non-serious space.

But the truth is GPUs have been so advanced for so long, that we really should not be blindsided when the tech begins doing remarkable things (it was already doing remarkable things, it’s just been hidden away in “niche” forums, when in reality it was world changing tech that was underestimated since it was just pimple face teens buying it).

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by motoxpro on Tuesday January 07, 2025 @06:29AM (#42621403)
Unless you predicted AI and Crypto then it was just really good, not 100x. It 20x from 2005-2020 but ~500x from 2005-2025

And if you truly did predict that Nvidia would own those markets and those markets would be massive, you could have also bought Amazon, Google or heck even Bitcoin. Anything you touched in tech really would have made you a millionaire really.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by fragmede on Tuesday January 07, 2025 @10:41AM (#42623557)
Survivors bias though. It's hard to name all the companies that failed in the dot com bust, but even among the ones that made it through, because they're not around any more, they're harder to remember than the winners. But MCI, Palm, RIM, Nortel, Compaq, Pets.com, Webvan all failed and went to zero. There's an uncountable number of ICOs and NFTs that ended up nowhere. SVB isn't exactly an tech stock but they were strongly connected to it and they failed.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by adolph on Tuesday January 07, 2025 @12:18PM (#42624833)
It is interesting to think about crypto as a stairstep that Nvidia used to get to its current position in AI. It wasn't games > ai, but games > crypto > ai.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by robohoe on Tuesday January 07, 2025 @11:01AM (#42623817)
Nvidia joined S&P500 in 2001 so if you've been doing passive index fund investing, you probably got a little bit of it in your funds. So there was some upside to it.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by computably on Monday January 06, 2025 @11:58PM (#42619404)
Yeah, it's more about preempting competitors from attracting any ecosystem development than the revenue itself.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by VikingCoder on Tuesday January 07, 2025 @12:01AM (#42619430)
If I were NVidia, I would be throwing everything I could at making entertainment experiences that need one of these to run...

I mean, this is awfully close to being "Her" in a box, right?

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by dagmx on Tuesday January 07, 2025 @12:06AM (#42619453)
I feel like a lot of people miss that Her was a dystopian future, not an ideal to hit.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by tacticus on Tuesday January 07, 2025 @12:15AM (#42619500)
they don't miss that part. they just want to be the evil character.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by t0lo on Tuesday January 07, 2025 @12:24AM (#42619529)
The dystopian overton window has shifted, didn't you know, moral ambiguity is a win now? :) Tesla was right.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by VikingCoder on Tuesday January 07, 2025 @12:41AM (#42619627)
We already live in dystopian hell and I'd like to have Scarlett Johansen whispering in my ear, thanks.

Also, I don't particularly want my data to be processed by anyone else.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by croes on Tuesday January 07, 2025 @12:44AM (#42619642)
OpenAI doesn’t make any profit. So either it dies or prices go up. Not to mention the privacy aspect of your own machine and the freedom of choice which models to run
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by blackoil on Tuesday January 07, 2025 @03:05AM (#42620369)
> So either it dies or prices go up.

Or efficiency gains in hardware and software catchup making current price point profitable.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by croes on Tuesday January 07, 2025 @03:34PM (#42627105)
Training data gets mired in expensive and they need constant input otherwise the AI‘s knowledge is outdated
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by com2kid on Tuesday January 07, 2025 @10:46AM (#42623621)
OpenAI built a 3 billion dollar business in less than 3 years of a commercial offering.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by croes on Tuesday January 07, 2025 @03:33PM (#42627088)
3 billion revenue and 5 billion loss doesn’t sound like a sustainable business model.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by VikingCoder on Tuesday January 07, 2025 @08:59PM (#42630094)
The real question is what the next 3 years look like. If it's another 5 billion burned for 3 billion or less in revenue, that's one thing... But...
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by croes on Wednesday January 08, 2025 @02:21AM (#42631889)
How...
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by menaerus on Wednesday January 08, 2025 @03:18AM (#42632175)
Recent report says there are 1M paying customers. At ~30USD for 12 months this is ~3.6B of revenue which kinda matches their reported figures. So to break even at their ~5B costs assuming that they need no further major investment in infrastructure they only need to increase the paying subscriptions from 1M to 2M. Since there are ~250M people who engaged with OpenAI free tier service 2x projection doesn't sound too surreal.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by com2kid on Wednesday January 08, 2025 @04:24AM (#42632463)
Rumor has it they run queries at a profit, and most of the cost is in training and staff.

If they is true their path to profitability isn't super rocky. Their path to achieving their current valuation may end up being trickier though!

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by int_19h on Tuesday January 07, 2025 @03:12AM (#42620406)
This is exactly the scenario where you don't want "the cloud" anywhere.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by nostromo on Tuesday January 07, 2025 @03:34AM (#42620519)
Fun fact: Her was set in the year 2025.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by swat535 on Tuesday January 07, 2025 @07:57AM (#42621870)
Boring fact: The underlying theme of the movie Her is actually divorce and the destructive impact it has on people, the futuristic AI stuff is just for stuffing!
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by AnonymousPlanet on Tuesday January 07, 2025 @09:01AM (#42622366)
The overall theme of Her was human relationships. It was not about AI and not just about divorce in particular.The AI was just a plot device to include a bodyless person into the equation. Watch it again with this in mind and you will see what I mean.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by adolph on Tuesday January 07, 2025 @12:57PM (#42625252)
The universal theme of Her was the set of harmonics that define what is something and the thresholds, boundaries, windows onto what is not thatthing but someotherthing, even if the thing perceived is a mirror, not just about human relationships in particular. The relationship was just a plot device to make a work of deep philosophy into a marketable romantic comedy.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by smt88 on Tuesday January 07, 2025 @06:18AM (#42621356)
If Silicon Valley could tell the difference between utopias and dystopias, we wouldn't have companies named Soylent or iRobot, and the recently announced Anduril/Palantir/OpenAI partnership to hasten the creation of either SkyNet or Big Brother wouldn't have happened at all.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by VikingCoder on Tuesday January 07, 2025 @09:01PM (#42630110)
I mean, we still act like a "wild goose chase" is a bad thing.

We still schedule "bi-weekly" meetings.

We can't agree on which way charge goes in a wire.

Have you seen the y-axis on an economists chart?

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by dnissley on Tuesday January 07, 2025 @01:39PM (#42625745)
Please name the dystopian elements of Her.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by vasco on Tuesday January 07, 2025 @04:10PM (#42627519)
One man's dystopia is another man's dream. There's no "missing" in the moral of a movie, you make whatever you want out of it.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by int_19h on Tuesday January 07, 2025 @03:13AM (#42620410)
The real interesting stuff will happen when we get multimodal LMs that can do VR output.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by option on Monday January 06, 2025 @11:57PM (#42619397)
today’s enthusiast, grad student, hacker is tomorrow’s startup founder, CEO, CTO or 10x contributor in large tech company
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Mistletoe on Tuesday January 07, 2025 @02:31AM (#42620210)
> tomorrow’s startup founder, CEO, CTO or 10x contributor in large tech company

Do we need more of those? We need plumbers and people that know how to build houses. We are completely full on founders and executives.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by davrosthedalek on Tuesday January 07, 2025 @07:15AM (#42621629)
We might not, but Nvidia would certainly like it.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by hatboat on Wednesday January 08, 2025 @07:00AM (#42633300)
If they're already an "enthusiast, grad student, hacker", are they likely to choose the "plumbers and people that know how to build houses" career track?

True passion for one's career is rare, despite the clichéd platitudes ecouraging otherwise. That's something we should encourage and invest in regardless of the field.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by Karupan on Tuesday January 07, 2025 @12:20AM (#42619510)
I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs.

Surely a smaller market than gamers or datacenters for sure.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by dagmx on Tuesday January 07, 2025 @12:43AM (#42619637)
I mean negligible to their bottom line. There may be tons of units bought or not, but the margin on a single datacenter system would buy tens of these.

It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by htrp on Tuesday January 07, 2025 @01:23AM (#42619863)
>It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

100%

The people who prototype on a 3k workstation will also be the people who decide how to architect for a 3k GPU buildout for model training.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by mrlongroots on Tuesday January 07, 2025 @06:41AM (#42621450)
> It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

It will be massive for research labs. Most academics have to jump through a lot of hoops to get to play with not just CUDA, but also GPUDirect/RDMA/Infiniband etc. If you get older/donated hardware, you may have a large cluster but not newer features.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by ckemere on Tuesday January 07, 2025 @09:17AM (#42622526)
Academic minimal-bureaucracy purchasing card limit is about $4k, so pricing is convenient*2.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by bwfan123 on Tuesday January 07, 2025 @10:46AM (#42623624)
Devalapers developers developers - balmer monkey dance - the key to be entrenched is the platform ecosystem.

Also why aws is giving trainium credits for free

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by stuaxo on Tuesday January 07, 2025 @04:36AM (#42620854)
It's annoying I do LLMs for work and have a bit of an interest in them and doing stuff with GANS etc.

I have a bit of an interest in games too.

If I could get one platform for both, I could justify 2k maybe a bit more.

I can't justify that for just one half: running games on Mac, right now via Linux: no thanks.

And on the PC side, nvidia consumer cards only go to 24gb which is a bit limiting for LLMs, while being very expensive - I only play games every few months.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by bloomingkales on Tuesday January 07, 2025 @07:08AM (#42621585)
My $800 computer from two years ago playes modern games at 144hz at 1440p and runs 8gb models locally really fast. I’m about a $500 you upgrade away from running 16gb models and playing at 4k.

You are so late to this party it’s not even funny. It’s affordable and available.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by wkat4242 on Tuesday January 07, 2025 @09:12AM (#42622473)
32gb as of last night :)
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by WaxProlix on Tuesday January 07, 2025 @09:15AM (#42622505)
The new $2k card from Nvidia will be 32GB but your point stands. AMD is planning a unified chiplet based GPU architecture (AI/data center/workstation/gaming) called UDNA, which might alleviate some of these issues. It's been delayed and delayed though - hence the lackluster GPU offerings from team Red this cycle - so I haven't been getting my hopes up.

Maybe (LP)CAMM2 memory will make model usage just cheap enough that I can have a hosting server for it and do my usual midrange gaming GPU thing before then.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by FuriouslyAdrift on Tuesday January 07, 2025 @02:46PM (#42626549)
Unified architecture is still on track for 2026-ish.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by sliken on Tuesday January 07, 2025 @03:57PM (#42627370)
Grace + Hopper, Grace + blackwell, and discussed GB10 are much like the currently shipping AMD MI300A.

I do hope that a AMD Strix Halo ships with 2 LPCAMM2 slots for a total width of 256 bits.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @08:27AM (#42622080)
Yes, but people already had their Macs for others reasons.

No one goes to an Apple store thinking "I'll get a laptop to do AI inference".

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by JohnBooty on Tuesday January 07, 2025 @09:06AM (#42622421)
They have, because until now Apple Silicon was the only practical way for many to work with larger models at home because they can be configured with 64-192GB of unified memory. Even the laptops can be configured with up to 128GB of unified memory.

Performance is not amazing (roughly 4060 level, I think?) but in many ways it was the only game in town unless you were willing and able to build a multi-3090/4090 rig.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @11:14AM (#42624005)
I would bet that people running LLMs on their Macs, today, is <0.1% of their user base.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by sroussey on Tuesday January 07, 2025 @01:02PM (#42625314)
People buying Macs for LLMs—sure I agree.

Since the current MacOS comes built in with small LLMs, that number might be closer to 50% not 0.1%.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @03:59PM (#42627383)
I'm not arguing whether or not Macs are capable of doing it, but whether is a material force that drives people to buy Macs because of it; it's not.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by justincormack on Tuesday January 07, 2025 @03:05PM (#42626764)
Higher than that buying the top end machines though, which are very high margin
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by throwaway48476 on Tuesday January 07, 2025 @04:25PM (#42627711)
All macs? Yes. But of 192GB mac configs? Probably >50%
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by the_other on Tuesday January 07, 2025 @08:55AM (#42622296)
I'm currently wondering how likely it is I'll get into deeper LLM usage, and therefore how much Apple Silicon I need (because I'm addicted to macOS). So I'm some way closer to your steel man than you'd expect. But I'm probably a niche within a niche.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by com2kid on Tuesday January 07, 2025 @10:33AM (#42623427)
Tons of people do, my next machine will likely be a Mac for 60% this reason and 40% Windows being so user hostile now.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by kelsey98765431 on Tuesday January 07, 2025 @09:27AM (#42622639)
my $5k m3 max 128gb disagrees
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by moralestapia on Tuesday January 07, 2025 @11:12AM (#42623970)
Doubt it, a year ago useful local LLMs on a Mac (via something like ollama) was barely taking off.

If what you say it's true you were among the first 100 people on the planet who were doing this; which btw, further supports my argument on how extremely rare is that use case for Mac users.

Re:Nvidia's Project Digits is a 'personal AI supercomputer' by sroussey on Tuesday January 07, 2025 @01:04PM (#42625331)
No, I got a MacBook Pro 14”with M2 Max and 64GB for LLMs, and that was two generations back.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by kgwgk on Tuesday January 07, 2025 @05:26PM (#42628423)
People were running llama.cpp on Mac laptops in March 2023 and Llama2 was released in July 2023. People were buying Macs to run LLMs months before M3 machines became available in November 2023.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by gr3ml1n on Tuesday January 07, 2025 @01:28AM (#42619885)
AMD thought the enthusiast side of things was a negligible side of the market.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by dagmx on Tuesday January 07, 2025 @02:45AM (#42620274)
That’s not what I’m saying. I’m saying that the people buying this aren’t going to shift their bottom line in any kind of noticeable way. They’re already sold out of their money makers. This is just an entrenchment opportunity.
Re:Nvidia's Project Digits is a 'personal AI supercomputer' by epolanski on Tuesday January 07, 2025 @07:18AM (#42621646)
If this is gonna be widely used by ML engineers, in biopharma, etc and they land 1000$ margins at half a million sales that's half a billion in revenue, with potential to grow.