News for hackers. Stuff that matters.
Re:Nvidia might do for desktop AI what it did for desktop ga... by richardw on Wednesday January 15, 2025 @10:15PM (#42720644)
I don’t get the need. With gaming there’s a real benefit to having the card close to a display. There’s enough benefit that you don’t mind it being unused 20 hours a day. There’s relatively little benefit to having training happen a few feet away rather than a data center. Solid chance it sits unused most of the time, and when you really need it you run into capacity issues, so you’d need to predict your future needs carefully or be happy waiting for a job to finish.

AI training feels like transport. You rent the capacity/vehicle you need on demand, benefit from yearly upgrades. Very few people are doing so much training that they need a local powerhouse, upgraded every year or so.

Even sharing the hardware in a pool seems more rational. Pay 200/month for access to a semi private cluster rather than having it sit on your desk.

Re:Nvidia might do for desktop AI what it did for desktop ga... by ClemFandango22 on Wednesday January 15, 2025 @10:35PM (#42720809)
I'm not too familiar with the AI space but I wonder if this is an effort from NVIDIA to combine their AI and Gaming markets. Did this come from a conversational question stating, 'How do we sell discrete cards through our existing manufacturing partnerships to both gaming enthusiasts and AI enthusiasts?'. I do wonder how comfortable they are pivoting back to being a consumer hardware company if AI becomes a more competitive space, or if the 'hype' subsides. Pure speculation and I'm probably off the mark.
Re:Nvidia might do for desktop AI what it did for desktop ga... by richardw on Wednesday January 15, 2025 @11:19PM (#42721141)
Yeah quite possible. They have distribution, brand, customers used to paying a lot of money. Can go far just selling to the portion of existing customers who would like "the best local setup for AI".
Re:Nvidia might do for desktop AI what it did for desktop ga... by gunian on Wednesday January 15, 2025 @11:45PM (#42721329)
privacy for machine learning to be locally usable besides a chat interface in a meaningful way I would have to give it access to pretty much all my digital data

sending that over the network feels very idk icky because it's not just photos or emails

Re:Nvidia might do for desktop AI what it did for desktop ga... by kadushka on Wednesday January 15, 2025 @10:41PM (#42720858)
I see the DIGITs box as mainly for inference, not training. It allows me to load a fairly large model (e.g. 70B llama or 12B flux) and run it locally at decent speeds.
Re:Nvidia might do for desktop AI what it did for desktop ga... by richardw on Wednesday January 15, 2025 @11:16PM (#42721131)
Then surely far simpler custom chips are the eventual model, like happened with crypto? Groq, Etched etc. In that universe, Nvidia has absolutely no moat and a thousand chips are coming.
Re:Nvidia might do for desktop AI what it did for desktop ga... by kadushka on Friday January 17, 2025 @07:38AM (#42736895)
Maybe. For me to consider buying some unknown hardware it needs be a lot better than Nvidia. Like double the speed, half the power, and half the price. And the software better be rock solid and tested on all popular models.

Many startups have been trying for several years now, and eventually someone will succeed, but it's not easy. Even AMD hasn't been able to pull it off.

Re:Nvidia might do for desktop AI what it did for desktop ga... by echelon on Wednesday January 15, 2025 @10:33PM (#42720800)
A video editor wants the tool sitting on their desk. Not pay-per-gen SaaS where the results are garbage nine out of ten times.

The comfy ecosystem is rife with people that want local tools.

Re:Nvidia might do for desktop AI what it did for desktop ga... by returnInfinity on Thursday January 16, 2025 @12:08AM (#42721514)
6000 / 24 = 250 usd

If you replace your GPU every 2 years, its 250 usd per month

If the price halves, still its 125 usd.

Even if price halves and use for 5 years, its 50 usd per month.

Re:Nvidia might do for desktop AI what it did for desktop ga... by echelon on Thursday January 16, 2025 @02:35AM (#42722434)
People are already spending far more than that on Runway and Kling:

> I spent $745 in Kling credits to bring the Princess Mononoke trailer to life.

https://www.reddit.com/r/aivideo/comments/1fvchbf/i_spent_74...

Most of that is in failed generations.

Re:Nvidia might do for desktop AI what it did for desktop ga... by sliken on Thursday January 16, 2025 @12:29AM (#42721658)
A local NAS makes sense for some use cases. Privacy, local control, and network bandwidth adds up to having a NAS for some use cases.

I see similar coming for AI. Tagging local photos, reading/summarizing private documents, helping you code (without uploaded it to 3rd parties), using uncensored LLMs, maybe even playing NPCs once games support API for LLMs.

It's going to take awhile, but I wouldn't be surprised if NASs start supporting AI accelerators to provide local endpoints for AI.