Pay up for license to unlock full capacity, pay up for license to use encrypted models from AI moguls, paid per billion tokens processed or just operations in GPU
I wonder if this space is explored fully enough at the moment.
Aside from some implementation details, it's conceptually much easier than doing it for a SQL database.
The consumer has access to free and open models for local use.
The supplier has better control and wider market to serve running the models in data centers.
Only way I can see encrypted models being widespread is if they are bundled as part of a software package (OS/Browser/etc) and not sold as the product.
And, the same GPUs are run in datacenter, where there's a big insider risk that vendors want to mitigate with encrypted models (if you're Anthropic and you want AWS to host your models, you don't want 5000 or even 500 or even 50 AWS employees with access to your highly valuable/desirable model weights).
NVIDIA is holding to its pearls like a sloth holding to its branch, because their all business model and livelihood is dependent on that control. I still commend AMD's decision to allow full card enablement with any driver which talks the correct language.
There will be ways for them to segment the market and make their money. Maybe not 100x stock in 5 years but money all the same.
So it is something that has been done for a long time now. Still feels like unethical rent-seeking that should not be tolerated by the customer.
See Also: car companies that charge additional fees for hardware already on the car. Additional services, I sort of understand(remote start via cell network), but stuff like heated seats, that is just wrong.
It saves the car manufacturer on logistics cost. The per-unit cost of a seat heater is (much) smaller than the cost related to keeping two variants of seat in stock and to provide spare parts. Say the heating mat costs 100€ per seat including materials and installation, and can be sold to the customer for 400€ permanent, that means as long as 1 in 4 customers buys the seat heating option, the books are balanced, and all money you save on logistics is pure profit.
Adding subscription on top of that is financially even better, if there is one thing companies LOVE thanks to stonk market incentives it is monthly recurring revenue.
I think more than that, it is probably a lie they tell us that you are not paying for the hardware/equipment.
Yeah, Nothing morally wrong there.
A license has zero marginal cost in the first place. Software distribution over networks have a negligible marginal cost. An ebook as zero margin cost. We are constantly paying lots of things with zero or negligible marginal cost.
AI is a software just like operating systems and executables where open source absolutely dominates the industry. Remember when the compiler engineers optimized the shit out of llama.cpp.
Right now the bottleneck for AI software is compute and data. Moore's law is give us compute, StackOverflow, arXiv (and it's children like bioRxiv), gutenberg and others (3lib, anna, internet.org) will give us data.
It's a waiting game. Open source always wins.
Tons of stuff happenning behind closed doors today though. No more stackoverflow for the next decade, that could create a problem if all new context only exists in chatgpt conversations only openAI has access to (or whoever).
Based on... what? Thoughts and prayers? Feelings? Vibes?
I don't know if you looked around but open source is basically unusable for the largest platform in the world, mobile.
The next platform, whatever that is, will also be closed source (maybe VR, once it finally matures...).
LLMs are closed source, is there even 1 (ONE) LLM which is both open source and open weights and publishes its training data and methodology and is also remotely competitive with the frontier lab LLMs?
Consoles are closed source. Appliances are closed source. Embedded devices are closed source.
Even the web is basically closed source. Blobs of encrypted, obfuscated JS used to serve news pages (!).
Maybe I'm looking in the wrong places, but closed source has won.
And regarding servers, who cares? The platforms users interact with are their clients and the actual server software on top of the Linux OS is closed source 99% of the time.
Don't post lazy comments.
Nothing stops you from giving Nvidia half a million for their latest and greatest AI GPUs, unless you are in China I guess, but China does very well despite the export control.
It is just classic market segmentation, the thing Nvidia and just about every company that can benefit from it has done for decades. Cloud providers are ready to pay a lot of money, consumers wanting local AI have a lower budget, and gamers even lower. But a low budget is greater than zero, and there is still money to be made on these markets. But they also don't want to compete against themselves, so they are going to find ways of crippling enterprise-class workloads on consumer hardware.
It used to be that they crippled high precision arithmetic on consumer hardware, because it was a differentiating factor, but AI doesn't care about high precision, so now, it is something else.
These are multi-million dollar datacenter systems, not consumer. If you mean the consumer market, it's coming but it's going to be 2028-29 most likely (if the bubbled hasn't popped by then).
Strix Point is quite good but not the same thing as a DGX, though. Same idea, different methodology.
I miss the old days when they released enthusiast products like the R9 295X2. Though that’s just the gaming enthusiast market in general declining (not that this happened organically though, it feels like a deliberate choice).
I have many pleasant memories of keeping myself warm just by opening Witcher 3 on my R9 390.
AMDs revenue this quarter just passed Intels for the first time in history and they still are dominating in the HPC market which is fairly immune to the AI bubble (while also selling everything they can build at a high premium to AI datacenters).
AMDs margins on their Instinct datacenter GPUs is nearly as good as nVidia's. They just can't get contract volume at TSMC to make more because nVidia buys all of it.
https://d1.awsstatic.com/onedam/marketing-channels/website/a...
They're still getting plenty of Nvidia GPUs. Someone in Jensen Huang's inner circle apparently runs a number of companies in Singapore that have been found to source GPUs at scale. The catch is that she has close ties to Beijing and is suspected of working closely with their government, potentially even spying for them.
This was reported just a week or two ago and is just one example of how China is still getting their hands on resources they're supposedly unable to get.
Social processes that pull towards freedom tend to be low investment, chaotic, individualistic processes; usually just working within the voids between enclosed spaces. There are projects of significant complexity working in these spaces, but they tend to be focused on their individual goals rather than the preservation of those voids.
Social processes that pull towards structural control, in contrast, tend to be well organised, funded, and focused mostly on exerting that control as a means to further some other goal.
With a high diversity of strategic actors and their highly divergent goals, almost any movement in a space of technological change can be seen as an opportunity to implement a ratchet that encloses more ground in the service of commercial or political vision. The end result is creeping enclosure of every active commons, while the expansion of opportunities occurs only at the frontiers of technology or in the abandoned areas with minimal commercial value (until they become revitalised, at least)
I do not understand this play. This means that only hyperscalers or someone who purchased a licence can use Spark?
I stopped using my laptop for dev and switched to talking to claude code on my phone and as you can imagine I am faster at producing code.
100% cloud with a single entry point device to manage dedicated servers + agents
[Hanlon's Razor] This is such a contrived performance drop scenario that I find it hard to believe it was done with the purpose you're inferring.
If anything I'd be more inclined to interpret this as Nvidia not caring that much about Linux once again, rather than anything to do with local vs cloud.
Even that argument is weak - the DGX Spark is Linux, and the RTX Spark is practically the same exact chip.
Sorry to remind people of HN, but most of the consumer market still runs Windows. This was a joint project to have a consumer-level Windows-on-ARM machine, likely requiring lots of development efforts from both NVIDIA and Microsoft.
So, for these specific laptops, they put 100% of their attention for it to be on Windows for launch to capture as much market mindshare as they can. Local AI, Graphics and creative work, and some gaming.
I doubt Microsoft would have been happy about NVIDIA having Linux on their laptop right at launch.
Market Share by OS for consumer laptops & desktops:
• Windows: 76.65%
• OS X / macOS (Combined): ~15.67% (OS X at 9.86% + macOS at 5.81% due to user-agent reporting quirks)
• Linux: 5.63%
• Chrome OS: 2.04%
As in, make your own inference and training hardware?
Sovereign AI is less about being able to build it from first principles and more about being able to tinker and modify what already exists. I can’t train a 30B model, but I can modify the weights of one. I can’t fab my own chips, but I can putter with the software that interfaces with them.
Sovereignty is lost when encryption hides your toys and attestation tells you “you may not”.
Estonian only has sovereign AI as long as they’re allowed to buy from Nvidia. Oh wait, they’re already restricted in what they can get.
But those servers most likely use CPUs not made in China, if I were to hazard a guess, doesn't that mean that that AI isn't "truly sovereign" in your mind then?
Who exactly?
Beyond him, I would bet most of his bilderberg group friends agree. E.g. Peter Thiel, the owner of JD Vance.
This is a linux driver proble. On laptops designed for windows.
Read the article and stop sensationalizing.
This is a problem on the MSI Prestige laptop using the RTX Spark platform. Lets assume it also applies to the Asus and Dell machines using RTX spark. It isn't an issue with RTX spark desktops.
RTX Spark is a new platform and all of these laptops ship with Windows. You cant get Linux on any of these machines from the factory. There is also precedent for NVIDIA shipping linux support well after Windows (2 years in the case of Dynamic Boost, a similar power sharing feature).
Kindly remove this from your list of "Evidence that NVIDIA is Colluding to Centralize Inference" and everyone else take note of this confirmation bias in action.
That them trying to protect IP around something very profitable isn't crazy. That not being fully supported in Linux, when the product was literally designed around Windows, also isn't crazy or really evidence of anything, including evil (see the whole fucking history of Linux compatibility of made-for-Windows hardware).
Maybe let them cook.
It obviously can't be because of ulterior motives!
No. It's 'the NVidia way or the highway' problem. NVidia is so full of themselves they refuse to work under anyone's rules (i.e. the Linux kernel where they're constantly trying to work around them).
They're egomaniacs who are so full of themselves, riding high on the AI bubble, they think they're above everyone else in the ecosystem, while utilising all kinds of other people's work for their own benefit.
Look at the way they develop their 'open' Linux drivers where it's just a shim that loads a huge binary blob off the GPU rather than a proper driver like amdgpu has.
Meanwhile they wouldn't be a $5T company without Linux and the wider FLOSS ecosystem.
It's kind of like when Secure Boot debuted and the plan was to allow only Windows with it enabled, then no doubt someone would claim the fact that only Windows can boot with it enabled as a 'Linux driver problem'.
Claude, make a driver that runs at full capacity on my laptop. (make no mistakes).
nvidia has always been crap like this. Look at amd and intel, they let u write proper drivers (even if its real difficult to do so). No encrypted blobs or voodoo needed.
the nvidia drivers u need to use a card on Linux are unintelligible binary blobs. thats sad as it is.
Is the problem one that could be easily countered by assuming that laptops are generally better when they're not on fire?
Sovereign AI has never been more important.
Controlling AI is so much more powerful and impactful than controlling search or social algorithms.
Centralized AI a 10x on social control over everything we have seen before, and there is no way that a free and open society survives contact with centralised, high capability AI if the system is manipulated towards guiding outcomes. It’s essentially mind control, as people will offload an even greater share of their cognition to AI than they do with search and social.
Build and maintain sovereign AI while you still can.