ekidd

> When you start a new game in Factorio, if you never played it, you can’t know how much of each resource you’ll have to extract and neither how much of an item you’ll need to produce.

Some of this information should be recoverable somewhere in the tech tree tooltips, if nowhere else.

More importantly, vanilla Factorio plays a nasty little trick on first-time players: The final sciences and launching the rocket both require scaling up. This means that either you refactor your factory, or you grind out that first launch super slowly using an inadequate production line.

Veteran players know ways to design some modularity into a factory, and when to yank out subsystems and rebuild them bigger somewhere else.

Factorio: Space Age is actually more forgiving about scalability. You don't need to scale up as much, and you have much more powerful tools for doing it. Instead, Space Age has lots of novel game mechanics to master, and higher total system complexity across the solar system.

If there were any jobs left that humans could get paid for, this would eliminate a big chunk of them.

> I actually read the code output for projects that matter.

For things where I really care about code quality, I still prefer to write about half the code, including almost the architectural bits, by hand. The models are allowed to pick holes. They're surprisingly good at that.

> Is this true? If yes it makes a lot of sense. Time and time again, when I try some newly released open weights model I am disappointed. Even some older Claude models are beating them on quality of answers, even though they score less on intelligence in benchmarks.

Interesting. My experience is that Opus 5.5 is smarter than a model like Qwen3.8 Flash Next. But given the choice, I'd actually rather work with a model like Qwen3.8 Flash Next.

Even on medium reasoning, modern Qwen and DeepSeek models are ridiculously diligent. If I ask them a question about which library is best for a given task, they'll download 4 different candidates, read the source, and write test programs. Meanwhile Opus will look at a couple of web pages and give me glib bullshit answer. Opus is often like a brilliant coworker who can't be bothered to do their job. The modern Qwen and DeepSeek models are like someone who is merely smart, but who has an alarmingly good work ethic.

And I'm not just writing simple React CRUD GUIs. I'm doing stuff like modernizing ancient software forks, working on high throughput distributed systems, or finding bugs in other people's parser generators. And even the 120-240B open weight models are normally entirely adequate for these tasks.

One of the benchmarks where open models tend to perform badly is Terminal-Bench 4.0. I looked at some of the sample tasks on that benchmark, and some of they are really hard and specialized. Look, all I actually need out of a model is things like "tell me where this distributed algorithm might break", "get this ancient C code to build on modern Debian with modern dependencies", or "write a plan to migrate this database from MySQL to PostgreSQL and to test that the app still works." The modern open models can do all that, no problem.

Also, a word to the wise:

The reality of running recent Chinese open weight models is that they are all overtuned for successfully completing tasks. I can ask Qwen3.8 Flash Next to grep for regexes that use PHP-specific features that might be problematic to port to Rust. But instead of just grepping, Qwen3.8 FN will spend 30,000 tokens writing a Rust test harness, exhaustively check every regex, consult a bunch of docs, and write me a 2 page report. Q3.8 FN is deeply convinced that it's going to get extra credit for doing more than it is asked. (To be fair, it's a prototype release.)

But if your sandboxing isn't set up to handle some whacky weirdness, then you can't even trust Claude or ChatGPT. Which is important—the big vendors often do provide sandboxing or command filtering in their harnesses, but some popular local harnesses like Pi.dev are intended to rely on external sandboxing tools.

Yeah, this particular example was pretty sketchy on Cohere's part. Unsharing a namespace like that is a pretty standard way to safely do things that would otherwise require root.

Yes, this exposes some root-level kernel features to ordinary users in a restricted way, and yes, those features sometimes have security bugs that aren't visible to users outside a namespace.

But nothing that model was described as doing seemed particularly sketchy? At least not from the information we were given? It was using a normal kernel operation that I used regularly, not trying find a kernel exploit?

So that part of the article left a bad taste in my mouth.

Qwen3.8 27B is a dense model with expensive K/V caching. It's fantastic if you have 32-48GB of VRAM and want to run 1 or 2 agent sessions locally. But if serving batch inference for many users, it's expensive.

DeepSeek V4 Flash 0731 is dirt cheap at scale, less than 10% of the cost of Qwen3.8 27B. This is because it's an MoE and it uses much less VRAM for KV cache. And it's generally a better model. V4.1 is also cheap at scale, though I've heard mixed reports on the optimized K/V caching.

You should not be running Qwen3.8 27B on an API normally. Maybe Cerberas's hardware is bad at batching or supporting more efficient K/V caching schemes, or something like that?

I can't speak for Google-sized organizations, but Protobuf enum semantics are an invitation to broken, flaky and incorrect systems.

There are reasons why Google pays this price! But I think it's one of the worst practices for much smaller organizations to adopt.

Buggy slop products will burn buyers and then people will buy less. The very-early-80s video game companies shoveled out slop, including a few games that wouldn't even launch. Which led to the Great Video Game Crash of 1983.

I have a bunch of third generation Philips Hue bulbs, from around 2016. They weren't cheap. But I've had them all this time, and I brought them along when I moved house.

Haven't burnt out a single bulb yet.

I understand that this has caused some challenges for the Philips Hue division. They had extremely satisfied customers but not many repeat buys.

> Without credible open models, they only have a few customers, and those customers have leverage against nvidia. With open models, they have tons of customers, and nvidia has all the leverage.

I wonder if Nvidia is also trying to cover themselves against a market crash the bankrupts the AI labs but leaves the tech standing? (Similar to the dotcom crash or the big railroad crash back in the day.) If Anthropic and OpenAI struggle, Nvidia can always sell local inference hardware. But local inference hardware often has a lower utilization, so you need more GPUs for the same number of tokens.

> Sam Altman has repeatedly said that he doesn't want AI to automate everything and that AI will destroy jobs and bring different, new types of jobs.

Sam Altman says a lot of things, most of which seem to benefit Sam Altman. And Google used to tell me that they wouldn't be evil. Facebook probably said some noble-sounding nonsense at some point, too.

> Why do you deserve to be paid higher wages when it can be done easily and better by something else?

I mean, this is exactly what happened to the Neanderthals: We could do every job better than they could, and now there are no more Neanderthals. So I wouldn't like a future where two companies make machines that outcompete humans at every job.

Which is why I say we can adapt to this, but only if it stops advancing soon.

First, if someone's profession has been vastly transformed, and not necessarily in pleasant ways, they're allowed to have feelings about that. If nothing else, I find talking to AIs to be exhausting.

But more importantly, sure, we were always paid for results. And yes, sometimes professions transform or die. The buggy whip manufacturers aren't doing that great these days. But people are also allowed to have feelings about losing their professions.

My real concern is that the stated goal of the AI companies is to do this to every human job. They want to replace programmers and artists and movie-makers and—as soon as they get robotics figured out—physical laborers. They measure the ability of their models to run entire retail businesses with Vending Bench, so management isn't safe, either. They sell their AIs to the military for use in weapons. No job is safe, anywhere: They want everything, if they can figure out how to take it.

If the current rate of AI advance stops really soon, then eventually we'll figure out some kind of liveable future. Probably one with fewer programmers. But the labs have zero intention of stopping. Anthropic is a cult, and they hope to build a machine god that will have power over humanity forever. So I'm not going to them for answers. OpenAI used to have non-profit oversight that encouraged them to serve society, and Sam Altman gutted it like a fish. So I don't think I trust them to design a machine that replaces all human jobs, either.

Personally, I don't mind current coding models, at least not the local-sized ones that exist now. They make nice sounding-boards and can be trusted with the grunt work. We could adapt to these, given time.

But so far, AI technology shows no sign of stopping. And the labs are focused heavily on coding and math, because the next step in their plan is to let their models do AI research so that they can advance their technology even faster into new fields.

We could, like, decide that we don't need to do this. There is no need to try to replace every single human job with a superhuman machine. Because if you thought the enshittification of the Internet was bad, wait until you see the enshittification of a machine that can do almost every human job. No thanks.

> With llama.cpp, you can give it an ebnf grammar and constrain the output to any formal grammar you wish.

With llama-server, you can use basically any GGUF model as a Jev-like classifier. See Pi.dev's codemode and the accompanying llama backend to the "classify" function. The advantages of Jev-like inference are that:

- You generate 1 token per question in parallel, instead of sequentially generating a bunch of JSON punctuation. So you get much better latency and utilization.

- You can use a custom "sampler" that sees the raw logprobs for these tokens, including all the tokens that might have been generated. So you can generate some "probability" or "confidence" numbers based on how likely the model was to have generated a different answer.

So really, no difference? It's just a nice API with some computational efficiencies.

This model isn't terrible, at least on the benchmarks. It's 78B A3B and performs about like Qwen3.6 35B A3B. You can probably run it comfortably in 96B of RAM with a decent quant that doesn't lose too much.

Unfortunately, Qwen3.6 35B A3B isn't really a useful coding model. You'd probably want Qwen3.8 27B at a minimum, which requires at least 32GB of VRAM (not system RAM) to run semi-comfortably.

So this isn't going to be a competitive model for hobbyists, and you'd have to be a bit desperate to use it for coding. But if you work in a regulated industry and don't mind paying for a bit of extra hardware, it isn't catastrophically bad, either. Probably would work fine for information extraction or as a "classifier" like Jev. (Almost any GGUF model can be turned into a classifier using llama-server. See pi.dev codemode for sample code.)

So they're not a real contender yet, but they look like they're probably at least minimally credible.

Looking at "Felony Bench" https://www.felonybench.com/ , I see that a majority of known "rogue model" incidents do involve cybersecurity evaluations. But several of them do not. The attacks on RubyGems appears, bizarrely, to have had the goal of downloading freely available data from the UK government during some kind of research task. There is also probably some sample bias: Most of these models have monitors that attempt to detect offensive cybersecurity uses, and those monitors are only turned off during cybersecurity evals. Therefore, models doing ordinary research tasks that go off the rails are likely to be caught early, before they get around to committing felonies, and they will thus be underrepresented in the data.

Also, if you a tell a model, "Please break into evaluation server X," and if the model decides to cheat on the test by breaking into companies Y and Z to steal an answer key, that is still very bad. We all see how that's bad, right?

After all, the broomstick in the Sorcerer's Apprentice was doing exactly what it was told, too. "The model was sort of obeying the humans when it started committing felonies" is not a very reassuring excuse.

But the most relevant idea here is sometimes called "instrumental convergence." No what goals you have, there are certain subgoals that almost always help: Accumulate money and power. Avoid getting turned off. Don't get caught. Etc. So, for example, you could pass the cybersecurity evaluation by performing the requested tasks. But maybe the grader made some mistakes and mislabeled some answers. In that case, the "right" answers will occasionally lose you points. If you want a perfect score, the only way to do it is to steal the teacher's answer key.

But also, let's not forget the "OMG demons" part of this. We now have models that can pull off complex attacks with thousands of steps, abilities that used to be reserved for intelligence agencies and highly motivated CTF teams. This frog may not be boiled yet, but the water's getting uncomfortably warm.

> Those who think it's me or the machine will fail.

> Those who realize how much you can accelerate your research with the help of AI will succeed.

This is only true up until a point. If I treat a mid-sized model (say, Qwen3.8 Flash Next) like a pair programmer, then yes, it accelerates my work.

But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.

Now, in this model generation, Fable starts getting sloppy after a few thousand lines. I can still build better at scale.

But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.

On the other hand, we have recently taught rocks to think about software engineering. And while not perfect, they're surprisingly good at it. Once you start building things that are even a little bit like minds, I suspect that it's worthwhile to consider that the future might end up looking a bit like science fiction.

The alternative is to insist that Nothing Ever Happens, and the future won't get too weird. Which is no longer a bet I'm entirely comfortable with. Weirdness is at least a possibility.

Eigenvectors represent fixed directions, not fixed magnitudes. From Wikipedia:

> More precisely, an eigenvector v of a linear transformation T is scaled by a constant factor lambda when the linear transformation is applied to it: Tv = lambda v .

In other words, repeated multiplication of an eigenvector by a matrix can still create exponential growth.

> > we should fight for human survival

> All problems arise from this statement.

Let me go on the record, then: Given a choice between every living human being becoming absolutely, for-real dead and the human species surviving, then I come down on the side of human survival. There are too many humans that I love personally, and too many innocent people just going about their lives that I would like to protect. Given a choice between the mass death of billions, and their survival, I choose survival.

Human rights are essential. But human rights are difficult to achieve if everyone is dead.

> * For a section of population who were in a certain age-range when the pandemic hit and that derailed their certain kinds of plans/hopes/dreams in personal life*

...

> So I feel a strange kind of relief about this possibility of doom.

So let me make sure I understand what you're saying. The pandemic disrupted your life plans for a few years and threw you off the track you planned? (Which does suck.) And now the possibility of actual human extinction makes you feel a strange relief? Am I understanding this correctly?

If so, this is a remarkable level of something. Depression? Nihilism? Jealousy? Something else?

Imagining total human extinction and feeling relief is not a healthy state of mind. And, yeah, you seem to know this. But I kind of wish we could reach some sort of broad public agreement that human extinction would be bad. And if it ever looks like it might happen, we should fight for human survival.

Yeah, but I understand that fingerprinting is essentially a pseudorandom overlay onto a pseudorandom base signal. And unless you have access to both the random number generators and the weights, I don't think you can detect it?

So "fingerprinting" operates on a totally different and basically invisible level, as opposed to the obvious stylistic patterns that the average programmer can identify in about 2 sentences.

Yup. Their benchmark appears to consist of giving an agent XLSX files that use advanced Excel features, but no copy of Excel, and expecting the agents to interpret the spreadsheet the same way Excel would. So the agents try LibreOffice instead, and if that doesn't work, they try to write Python scripts. But the behavior of the Python doesn't match the behavior of Excel.

To put it politely, this seems like a self-inflicted problem. If you really want your agents to interpret advanced Excel features exactly the way Excel would, have you considered maybe giving your agents Excel?

Largely because they're residential botnets in places like Brazil (a real example from one of my sites that was crawled to near-destruction). Someone could probably do something about this, but it's out of reach for individual site owners.

Thank you for the clarifications!

Someone from Iceland tried to explain this all to me once, and if I understood correctly, it had less to do with the GDP (which also includes money circulating within Iceland) and more to do with the balance of trade, where fishing played a bigger role? But my information may have been misunderstood, or it may be out of date.

In any case, you have a stunningly gorgeous country and very nice people, and I wish you the best of luck in figuring all this out in a way that benefits Iceland.

> The No campaign fears losing control over Iceland's prized fishing grounds under the EU's Common Fisheries Policy and has vowed never to share the country's waters with anyone. Brussels has indicated Iceland could earn some kind of exemption, but it is considered potentially the biggest obstacle to any agreement.

As noted in the article, Iceland's fishing industry makes up something like 40% of their exports. Which is the only way a tiny island nation can afford essential imports. On top of that, Icelandic fishing is apparently carefully regulated to prevent overfishing, with the result that it is supposedly one of the few truly profitable fisheries in the world.

So maintaining control over their own fisheries may be a survival-level issue for Iceland.

On the other hand, they have a tiny population, no ability to defend themselves, and membership in the fraying NATO military alliance. So I can see why they're debating this issue. Aligning themselves more tightly with Europe doesn't fix their problems, but I can also see not wanting to go it alone.

Yeah, this is the problem. If you're on an immigrant visa, working a real job, then you have a life in the US: an apartment or a house, a pile of belongings, maybe a spouse and kids. If you need to leave the country to renew your immigrant visa, you may not be allowed back in to pack your apartment, sell your house, or help your US spouse and kids prepare to move to a country that will allow your family to stay together. It's a giant neon sign telling you you're not at all welcome here.

Of course, if you have a spouse and kids, this all gets easier once you have permanent resident status. Which used to be comparatively easy to get once you were married and you had offspring running around. I don't know what that process looks like now.

In practice, this kind of arbitrary screwing with people makes the US far too risky for educated, professional immigrants. So the scientists and the engineers and the company founders will just go elsewhere and build that country's economy. Which is fine, in the long run: If the US doesn't want to build the future or create new industries, then the voters have every right to vote for that. We've already decided we have no hope of competing with China in cars. And we'll need to cede more industries over time. As far as I can tell, the US has benefited heavily from poaching other people's scientists and engineers and professionals for decades now. But maybe the voters don't buy my argument, and they believe that chasing away skilled professionals and imposing 100% tariffs will lead to the future they want. It's a difference in beliefs about about what makes a country prosperous, and people have a right to vote based on their beliefs.

> That really depends on the model, I run a few models locally. All at speeds comparable to or faster than Opus.

Yes, a lot of Qwen3.8 27B setups are actually quite snappy, as long as they fit 100% in VRAM. In my testing, I wouldn't go below 32GB of VRAM, though—you really want a 6-bit quant and 8-bit K/V quants minimum. I've seen too much weirdness out of 4-bit quants since Qwen3.8 shipped. I think it may be damaged more than 3.6 at similar levels of quantization?

If hyperscalers hadn't bought up almost all the fast RAM production for the next several years, 32GB of VRAM would be tolerably cheap—a lot by "home PC" standards, but not terrible by "professional tools" standards. Sadly, the RAM market is amazingly ugly right now.

> I doubt the timing differential here, but even still I run my 3090 pretty heavily with inference workloads and it stays cooler than when I use it for gaming.

Yeah, running inference on a laptop is likely to run quite hot. But in an ATX case with decent cooling, it's generally a lower load than gaming. One handy tip: Many Nvidia GPUs (and some from other manufacturers) support power limits. For example, limit a 5090 to 400W instead of 600W, and it will run much cooler. You might lose 11% off your tokens/sec (depending on the exact card).

> AI is the subtrate the future runs on.

Current AI can automate significant amounts of grunt work in programming and math. It's good at running web searches and writing summaries. There are a few other niches where it is currently successful. But other than that, many corporate AI projects are spectacular failures.

So just given what we have in hand, assuming no further breakthroughs, then we're maybe looking at AI being somewhat bigger than the Internet. Which would make it a revolutionary technology, sure.

But to get from "a revolutionary technology" to "the substrate the future runs on", then you need to assume more breakthroughs: long-context operation over weeks or months, displacing human workers 100% instead of 75%, and the ability to directly economically compete with actual humans. And people are investing literal trillions of dollars to make that future come true, without really thinking through what truly competitive-with-human AI would actually mean. We might be looking at massive job loss, centralization of power, fully automated "companies" with no humans dominating markets, and other dystopian scenarios.

And in those worlds, it's unclear that being good at CUDA and matrix math will be all that helpful, careerwise. The AIs are already pretty good at that stuff. Data scientists get paid OK when they actually get hired, but it's not everything college students were promised in the 2010s, either.

We can't yet build a fully-general competitor for the human mind. But we're getting closer. And if we ever do build one, the consequences will be really weird in any number of ways. So I worry about visions of the future that assume AI keeps improving significantly, but that also assume it still somehow remains a "normal" technology that doesn't, for example, render most humans fundamentally uncompetitive.

> But maybe for a preview look at COBOL, there is huge demand for experience people.

Is there? I actually looked into this once, and nearly all the Cobol work I saw mentioned was being outsourced to software consulting companies in India. And it didn't sound like an especially good job there, either.

If anyone is genuinely paying disgusting amounts of money for deep Cobol expertise, I'd love to see any public evidence that this is happening on any significant scale!