Underutilization was indeed a problem before. The anecdotes you've presented are not broadly applicable.

The entire reason SETI@Home started was because people had unused hardware sitting idle at home. Every company I've ever been in has had slides on resource utilization for procured hardware.

In fact not only could the two statements be true, in fact both are true. People want to ignore otherwise because doing so will make local LLM inference sound like the only option when only one is considered.

Nobody put "your computer isn't idle sometimes" as a problem, or shamed users for it.

What they did was to say "hey, we can leverage your idle computer to help process this".

Different perspective.

Nobody has shamed users for having unused compute, only stating that it is not economically optimal when there are global restrictions on a resource.

It can still be optimal because some activity A might provide more utility at 10% utilization rate than activity B at 100% utilization rate . Otherwise shared cars would be vastly superior to individual owned cars and same for houses in which 2 or 3 families live in different shifts ... we actually had that before in some areas in early stages of the industrialisation.

Users apparently don't need to be thought about at all if a few companies can secure loans large enough to purchase all dram capacity, and in the process close that market for consumers.

It seems to me it's economically optimal for inference providers to intentionally close markets that would allow local inference at all, even if that's bad for users as a whole.

If those companies can pay for those loans from the DRAM facilitated revenue then it will have been an efficient use of resources.

And if they're wrong, you'll be able to purchase second hand cheap data centre grade hardware in a couple of years and DRAM prices will return to normal.

If you, or I, or a company want hardware we can choose to pay market prices, but 4+ years of students will be robbed of the opportunity to tinker with a class of hardware that would have been beneficial unless they have extraordinary resources or an extraordinary support network.

There is opportunity theft here.

>If those companies can pay for those loans from the DRAM facilitated revenue then it will have been an efficient use of resources.

"Efficient use of resources" is not some absolute based on mere energy or economics calculations, it depends on the goal.

If the goal is "reduce reliance on big tech", "privacy", "autonomy", and so on, then the most efficient use of resources if to say fuck you to Big AI and not let them hoard ram and gpus.

> ... blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.

A hard rule of life is, if there are resources available, something will consume them. It's interesting that this also ended up being true for compute!

This is true, but I think building an entire technology on top of the question of who can consume the resource as inefficiently as possible is still something else.

This is why fusion power is scary. Once invented, we WILL irreversibly convert all seawater to computation as quickly as we can.

99.67% of seawater can't be used for fusion. Some very poor napkin math, that's around 1.3 million cubic miles, with around 2200 cubic miles per inch of sea level, which is around 600 inches, or 50 feet drop in sea level if fully depleted (not including any sea level rise from global warming).

>Weirdly, underutilization of hardware was never a problem before. In fact, wastefulness is so ingrained in programming culture that the previous hype - blockchain - was about an explicit race over who could waste the most computing power on nonsensical calculations.

Same with AI. AI use is a colossal waste, for all kinds of frivolous purposes (e.g. some person's ai girfriend chat, or someone asking about K-pop lore, or slop image generation, or slop "dark internet posts"), but also to facilitate "serious" business that we'd be better if it wasn't done at all.

> Weirdly, underutilization of hardware was never a problem before.

Fixing exactly this has been the constant drumbeat of IT since the early 00's at least. It started with virtualization, to get capacity factor of hardware up in the enterprise. The average server back before VMWare (and HyperV/etc.) became a major thing was probably sitting at 10% utilization or less 95% of the time. Hypervisors were invented to fix this as their primary use case. You'd take a 5-10% performance hit on the raw server performance to gain 90% efficiency back.

Next came the cloud. Which was virtualization at scale. The larger the scale, the better the efficiency.

It's basically almost all anyone has actually accomplished in IT over the past decade or two. Everything else has been largely stagnant compared to the 80's and 90's.

AI inference is in it's infancy, but I imagine we will see more or less the exact same thing play out yet again. Lots of initial on-prem stuff due to cost and freedom, and eventually it becomes virtualized, commoditized, customer segregated, etc. Just like average cloud workloads are today and all the various "small" megawatt-scale builds moved back into the hyperscaler type providers within the next 5-10 years.

> How many companies are building datacenter commons and sharing with other companies to optimize resource usage?

Quite a few that I've seen. Rented inference/token factories are the current new hotness. Plenty of speculative builds being done by hundreds of smalltime providers. These will consolidate per usual and everyone but the last few top percent will remain as the winners. Just like the first hosting providers. Virtualization and fancy schedulers to maximize hardware utilization across a number of end-user customers are a huge topic in this space at the moment. Heck, a lot of these small time players are providing raw compute for the giant AI labs and even hyperscalers.

I don't like that Altman is right on this btw. I much preferred the highly inefficient days of every midsize corporate office having a 200-500kw datacenter in the basement. AWS and their ilk are quite obviously orders of magnitude more hardware efficient though.

OK, but Altman is complaining about individual users. Nobody said that maxing out my PC for gaming purposes or just to keep Windows from getting sluggish was a waste, pretty much ever.

And, "waste" has an overtone of moral wrong, of "you shouldn't". When before did anyone say that I shouldn't get a nice PC? Never.

So, yeah, moralizing to individual users that they shouldn't buy DRAM so someone else can have it is new.

He doesn't say anything about individual users. It's left unclear exactly what he means when he talks about low utilization inference using DRAM. I figured he was talking about companies building clusters. I see way more people using old GPUs for local inference than buying 15-20k Mac Studios, or RTX Sparks. I've seen plenty of stories about how companies are building clusters because they want to run open source models, partly for privacy purposes, partly because they're afraid of a rug pull, partly because they want to control when they move to new models, etc, etc.

> He doesn't say anything about individual users. It's left unclear exactly what he means when he talks about low utilization inference using DRAM. I figured he was talking about companies building clusters.

No, he absolutely does:

"From a global allocation perspective, actually, it's quite terrible to have everyone have a bunch of the world's limited supply of DRAM sitting in their own computer. And be used 5% of the time," Altman said.

> "How many companies are building datacenter commons and sharing with other companies to optimize resource usage?"

They're all sharing with each other. Anthropic's S-1, for instance, shows $100 billion+ commitments each to renting datacenter space from AWS, xAI, and Google (two of whom compete with Anthropic on AI). None of theme are idly sitting on unused depreciating hardware for no reason.

I've never heard anyone accuse FAANG's of not trying hard enough to optimize their own resource usage.

There was indeed a lot of blockchain bullshit, but Bitcoin had and still has its usefulness. Though not necessarily for you and many others.