> A great example of this is here: https://marshallbrain.com/manna1

This is a piece of science fiction and has its own (inaccurate, IMO) view on how minimum wage McDonald's employees would react to a robot manager. Extrapolating this to real life is naive at best.

>Extrapolating this to real life is naive at best.

Why, it's as much of a view of our past adherence to technology without thinking as a well as a view of the future.

"Computer says no" is a saying for a reason.

>"Computer says no" is a saying for a reason.

Current LLMs rarely or seldom say no. Unless, they're specifically configured to block out certain types of requests.

I must say that the book is unrealistic, but it makes a good sci-fi story. Thanks, I read it just now in 80 min.

I'm saying there's something structurally different form autonomous systems generally and from an LLM corpus which has all of the information in one place and at least in theory extractable by one user.

But none of those things are AI in the same sense that we use the term now, to refer to LLMs.

But those things were considered on the same level of current LLMs in the sense of "well, a computer might do part of my job but not ALL of it".

No, algorithmic trading didn't replace everything a trader did but it most certainly replaced large parts of the workload and made it much faster and horizontally scalable.

The two key differences to me are infrastructure and specificity of purpose.

Autoland in plane requires a set of expensive, complex, and highly fine-tuned equipment to be installed on every runway in the world that enables it (which as a proportion is statistically not a majority of them).

And as to specificity, this system does exactly one thing - land a specific model of plane on a specific runway equipped with instrumentation configured a specific way.

The point being: it isn’t a magic wand. Any serious conversation of AI in these types of life or death situations has to recognize that without the corresponding investment in infrastructure and specificity of purpose, things like this blog post are essentially just science fiction. The fact that previous generations of technology considered autoland and algorithmic trading to be magic doesn’t really change anything about that.

The problem here is that you are cherry picking examples of successful technology.

The inverse would be to list off Theranos, Google Stadia, and other failed tech and claim that people said that there was massive steps that subsequently didn't materialise. In fact a lot of times it was mostly fabricated by people with stuff to gain from ripping off VCs.

Look at how bad it is with Microsoft in Windows despite their "all in on AI".

Ultimately no one really knows how it will pan out, and if we will end up with Enron or an Apple. Or even if it's a combination of a successful tech that ultimately is mishandled by corporations and fails, or a limited tech that regardless captures the imagination through pop culture and takes over.

You gave examples of feedback loops.

We know very well how to train computers to handle those effectively.

Anything without quick feedback is much more difficult to do this way.

LLMs designing PID loops?

My point is the code for feedback loops is architecturally simple: when reading is wrong correct.

By boosting the accuracy and frequency of the reading you can get pretty good results.

But that has little to do with LLMs, or LLM generated code.

"Expert system" running a company is never going to happen unless shareholders are okay with no accountability from the company. You'll always need someone to blame in case things go wrong. You could have an executive using such an "expert system" for literally all their decisions, but it has to be a human being signing off on those decisions. There is no way to prosecute code and unless these expert systems can become sentinent or appear in court, best of luck trying to let it run a company in the real sense of actually making those decisions with full autonomy and responsbility.