(People actually came by my desk to see "the AI", the physical thing. It blows people's minds to realize AI doesn't need a data center, that it is just some computer parts sitting on a desk.)

The big hitter AI companies see the writing on the wall: open source models and local AI rigs will decimate the market for big AI in the next few years.

It, of course, runs on linux. I went with stock ubuntu so I could attach a screen and/or remote in to a full GUI. Having a full GUI also makes the rig more relatable to people who's eye glaze over when they see a terminal interface. "It's just a funny-looking desktop computer."

At 48GB of VRAM it can run basically any LLM I would want to download. Speeds are fine for a single user (Tok/s numbers change based on model/mode). But if I were using it for something like ComyfUI I would ditch the multiple 5060s and go with a 5070ti or 5080 as ComfyUI does better with a single powerful GPU.

I'm sure cloud models will always have a slight edge but most tasks, most of the time, won't need that few extra percentage points of performance.

Llama 3.1 8B baked into an ASIC. Expect 1/10th the power requirements, and other savings.

This is one of the reasons I don't believe the "it'll never be profitable" narrative. Just stop training when you think your model's "good enough" and move to custom silicon.

Yes and no. I bought most of the hardware before the rampocalypse. It is just an AM4 chip on a nothing motherboard. Even today, 16GB 5060s are maybe 500 each. 64GB of DDR4 is another 500$. Add a power supply and pci risers and the basic parts could be purchased today for around 3000$... call it 4000$ all-in including my labor to get everything connected and talking to each other.

4000$ is money but is also almost nothing in the world of commercial IT purchases.

MS don't train your data

Here every person gets $500 of copilot "credits" per month and that is a lot of use. $500 in Anthropic API gets used in 3 days.

The only disadvantage is that copilot cli has a very crappy and limited plugin system. For example a plugin can register a hook when a model calls a tool, but it's impossible to show a question to a user from such plugin.

Also it's "Autopilot" mode is designed to waste tokens. It injects things like "did you do absolutely everything you can about this? If not continue" and it's impossible to disable this without reverse engineering.

So only mildly surprised on reading the article to read they are intentionally forcing them onto a specific worse model.

You say that but that's not reflected in the law, and historically the us government has repeatedly created such sweeping surveillance systems. It's highly unlikely they only made it legal, and aren't using it sweepingly considering theyre on record doing so before it was legalized through the act before too...

And as that's international law... It's always kinda a coin toss whenever anyone actually pays it any attention. No real recourse if violated after all.

Yeah, and probably also forcing Zero Data Retention contracts on those service, instead of the consumer variants.

Of course something like what you describe could happen. But under the central case, ensuring some privacy (with the ability to seek damages if breached) whilst still providing access to frontier models seems like a reasonable trade-off with a lot of value.

I’m not saying it won’t happen, I’m saying it’s outweighed for most by other concerns.

Pfff, that’s fairy land territory. There is no “could”. It’s happening as we speak. There is no fine to be paid as everything is behind a gag order. And if it’s not secret enough they do it in public anyway.i.e:

https://www.gadgetreview.com/microsoft-accused-of-sharing-du...

That is called democracy. Where the good of citizens is valued above what big corporations want. To push up pollution and power prices so big American tech companies can play with AI seems something that only a mad king would do.

Governments are always at least a decade behind when it comes to technology. Just don't complain when they have massive data breaches weekly, because they decided their principals were more important.

Big corporations are only big because they provide what customers want. They don't get to extract money from people's work, the way politicians can.

This is, however, one of the reasons Germany is hit so hard by the current spike, I'll give you that.

Big corporations are big because they purchased all competitors. I recommend that you read the book Enshitification.

It's also so easy to astroturf to manipulate public opinion. If I were China, I'd post so much anti-datacenter content everywhere in the West to gain a leg up in the race to superintelligence, because we are dumb enough to let our citizenry influence these decisions.

Imagine if we did the same thing for the nuclear race. ASI is winner-take-all, the first one to ASI can just lock everyone else out.

The recent growth of LLMs is a black swan. Any future prediction you attempt to make has massive error bars.

Europe is in a very precarious position. If LLMs actually play out and lead to economic booms (I'm not saying for individuals, note that), the EU as a whole will get absolutely trounced economically and likely experience a shrinking economy for the next 20+ years along with a shrinking population in most of the member countries. They are so far behind on power and compute of the US and China there is not any current path to catching up.

The US itself is in a precarious position with the amount of debt it has accrued, it could be right that LLMs are what leads to future economic booms, but be so far in the hole it takes decades to recover.

China is really in the best place at the moment. Trying to build up power and compute as fast as it can, copying American innovation at the cheap, and having a mountain of US and EU money to spend on it.

It’s time to decide your stand. But if you want to make an informed decision, you probably want to:

- Read up on the possible dangers until you get it,

- Learn about the advantages and breakthroughs already happening, and,

- Do some extensive work on a real project with at least an Opus-level LLM, so you really understand what an astounding new thing it is in our world.

So our technoverlords should decide for us? All these arguments to go back to the middle ages and feudal systems are based on the idea that we should be oppressed and our opinion should not count. That thinking destroys enlighten societies and bring inequality and poverty.

https://gcping.com/

Now, do we have DCs with "temporary gas generators" operating 24/7 that were thrown together in the last 2 years? Probably not.

Just because power used to be even cheaper in the recent past, by doing non-sustainable shit and pushing all the resulting costs onto others/the future does not mean that was ever appropriate, necessary or desirable, and everyone advocating for chasing that model is a greedy, selfish asshole in my view.

If cloud based AI is any good, they will sell versions that can be hosted locally just like every other service since the beginning of the internet.

Ooops, too late.

I think one of the AI 2027 videos covered this pretty well. EU cannot make nor really afford the compute they need to buy. All they really have the ability to control is UV machines for making masks which is going to get politically hot very soon.

The US and China have claimed all production for years to come.

IIRC OpenAI do offer data residency controls but only to enterprise customers and subject to minimum spend.