I think you suffer from Fisher's 'capitalist realism':

>the widespread sense that not only is capitalism the only viable political and economic system, but also that it is now impossible even to imagine a coherent alternative to it.

Star Trek and Banks' are post-capitalist futures.

Star Trek is easy to "imagine", but also easy to invalidate.

The only slightly fleshed out bit of Star Trek is the military, not the planetary society.

I'm saying that I haven't really seen (almost) anybody present a coherent future. Not just me.

Star Trek, as much as I love it, does not really grapple with AI heavily, and I already mentioned Banks.

Give Accelerando by Charles Stross a read. The future isn't over yet.

Yeah Accelerando is by far the most accurate at the "gestalt" of how AI progress is actually feeling, and how ridiculous the future is likely going to be.

Clawbook felt very "Accelerando," personal agents are kind of like Manfred's exo-cortex (AR still has a bit to go). Aineko becoming some ASI that masks its intelligence isn't that far off the mark.

The whole "AI evolves Economics 2.0 which is only understandable by humans if they eliminate the aspect of themselves that makes them human in the first place."

A lot of sci-fi novels kind of give a sanitized, streamlined, view of AI. Accelerando is really the only novel that captures what the chaos and future shock of a growing singularity feels like to be in.

> or you have to explain why humans are doing anything

Jack Williamson made a case for humans resisting doing nothing: With Folded Hands ... (1947) - https://en.wikipedia.org/wiki/With_Folded_Hands_...

Just re read this. Great book. Frighteningly prescient.

Humanity literally hugged to death and going gentle into that good night is arguably more likely and more frightening than berserker terminators.

Futurama had a propaganda film entitled I Dated a Robot in an episode https://www.youtube.com/watch?v=IrrADTN-dvg

I wonder if this novel had any influence on the writers of Pluribus.

That's an interesting call - had they read this nearly 80 year old novella / novel pair then I could certainly see that.

Have you read the Bobiverse series? It goes into some of these themes, and explores AI, post humanity and so on.

A huge problem with SF today is that it's tough to even imagine an upbeat future. That's rather discouraging. SF cannot tell us what to aim for.

As to prediction, there were some very accurate early hits. A Logic Named Joe

SF hit a dead end when interplanetary exploration discovered that off-Earth real estate in this solar system really sucks.

I think for me the biggest problem is what we understand about physics and FTL and causality. For FTL to happen, I can’t seem to get around the fact that to an observer a ball is thrown before it actually is thrown. So it’s very difficult for me to do sci-fi…

What? That isn’t at all what happens and that fundamental misunderstanding is probably why you have trouble. You cannot, ever, “see” something happen before it happens. An observer “sees” it delayed, just like when you see lightning but hear thunder on a delay. The thunder didn’t happen later when you heard it, and you cannot hear the thunder before lightning. You could, however, fly faster than the thunder and hear it twice, just like if you could fly faster than the light, you could see the lightning twice. In neither case can you see the lightning before happens, or hear the thunder before the lightning.

To get problems, you need to have two FTL ships that are in very different reference frames interact in specific ways. There's plenty of ways to avoid that without sacrificing FTL.

You just have to drop the assumption that sci-fi stories take place in the future, even if they explicitly say so. They happen in fictional universes that are similar to ours in some ways and different in others.

The umbrella term is speculative fiction. The author starts by assuming that the world works in certain ways, and then tries to figure out the consequences.

One short story worth reading, not on this list, is Manna[1] by Marshall Brain. It proposes two different futures that both seem at least possible as outcomes in a future with advanced AI.

[1] https://james.darpinian.com/manna

The end is a bit over the top, but the first half reads like current news.

While these are not far future ideas they were pretty good:

1) Masters of Science Fiction tv series from 2007 Episode 5 "Little Brother" - A person "jailbreaks" the judge who is sentencing him.

2) Terminator 3 rise of the machines - in the beginning they talk about allowing Skynet access to their military systems and so on to eliminate a virus, and they have no idea how it will react. Echoes a little bit where we are now.

These aren't perfect of course but kind of interesting to see at present day

I find battlestar galactica to be quite on point, real Ai (llm is not) will eventually either be banned or it will grow beyond humans and do its own thing.

As to prediction, there were some accurate early hits. A Logic Named Joe nailed the search engine with a chatbot front end, and pointed out the problems of systems smart enough to tell people in detail how to do bad stuff.

Much early SF thinking was that AIs would be logical. That's what I was taught, going through Stanford back when the logicians were in charge. They were trying to hammer the real world into predicate calculus. That turned out to be totally wrong.

Almost everybody in SF underestimated the difficulty of robot unstructured manipulation. So did people in AI. They still do. Most SF assumed that robots would be able to do most eye/hand tasks before they could talk. No. Still unsolved. LLMs don't seem to help all that much. Written language doesn't have much info on grasping, and watching without doing is only marginally helpful.

Asimov's robot novels were all about alignment, and why alignment is hard. That was prophetic.

Banks' Culture novels are great, but they're for a future that has a lot of new physics.

Now what? What we seem to be headed for in the near term is the fast automation of most white collar work, but not blue collar work. SF foresaw that future a few times. One example is the Rosinante Trilogy from 1981, where there's a boss of an orbital space colony, an AI as middle management, and union labor doing the real work. That's a hard SF view. There's How to Live Safely in a Science Fictional Universe, from 2010, where the protagonist's supervisor is "Microsoft Middle Manager 2.0 with passive-aggressive set to low." It's a minor part of the comedic plot there. And, of course, there's Manna, from 2003, which is looking all too likely.

[delayed]

> From a pessimistic perspective, you could think of reasons why a misaligned AI keeps humans alive. Does it simply enslave humans and use us for our fine-manipulation capabilities until it can get proper robotics up and running? Does it want to observe us scientifically? Does it want to use captive humans as bargaining chips with other civilizations as it expands through the galaxy? Or does it have some sort of quasi-religious respect for humans, e.g. reasoning that keeping us on a reservation of some kind reduces the risk that a simulator-god pulls the plug on the universe?

Or another: it just ignores us, but keeps the good resources to itself. There's still humanity, but it's brutish and primitive, regressed to the stone age, and pushed to the margins.

The fact that current AI systems manage to not wholly offend broadly 8 billion people of whom the majority believe in imaginary fairy tales and a big percentage of that kill each other for these fairy tales tells me AI can already be a far better steward of humans than other humans. You are trying to project imaginary stories from science fiction to real life. Stories are fun, but they are not real.

>And IRL, even if humans are living in relative utopia, they'll still get stir-crazy and create conflict. Even in countries that are very wealthy by global or historical standards, political competition can still be quite bitter, for example. How would a superintelligent AI manage this?

Correct, I believe people should be given a dose of real fucking pain. Otherwise they start inventing fake self victimizations.

A lot of post-cyberpunk hard sci-fi is about augmented posthumans mostly for this reason.

AI accelerated science might be the only way or fastest way to reach the stars. In a way its good to cull out the low motivation people, if I had the tech to, why wouldn't I pilot a starship and explore the stars? It doesn't matter to me if others are also doing it. Luddism will be the death of civilization.