Everybody needs to go read Asimov's Someday. Now's a good time, go do it.

Not mentioning Peter Watts here is criminal, because he came unnervingly close to nailing it. The voice of Rorschach in Blindsight is very similar to early LLMs. And, well, hopefully the dead internet of Starfish doesn't turn out to be a prophecy but I'm getting nervous.

Not just the voice, but the underlying process. Rorschach passively absorbed decades of human broadcasts and built up a language model that was eerily fluent despite literally not understanding what it was saying. The conversation with the "head cheese" neural network in Starfish hits a lot of the same notes (and even features a great example of misalignment that threatens humanity).

Crazy he wrote these in the 2000s.

Iain M Banks' Culture series seem to have got a lot right.

It's gonna be cool to reevaluate that in few years. Different stories might move to the top as real world AIs capabilities grow and new behaviors manifest.

Yeah, The Terminator scored low on this rating for now.

It's interesting how the Hard SF authors (at least, the ones I read) had poorer predictions than the more fantasy-like ones. Asimov, who was considered a giant of the field, thought that AI (robots) would have difficulty with ambiguity, creativity, and lying. This trend extends all the way to Star Trek's Data. Looking back on it, this view was probably based on Asimov consciously or unconsciously basing his robots on the more autistic of the scientists that he knew.

Whereas fantasy authors write stuff like "a sufficiently large pile of books becomes an intelligence" (Pratchett) and that's pretty much what's happened...

AI attempts at the time were based on modelling and logic, not machine learning. Whereas soft SF authors were just like "there are people made of computer chips".

Mid-20th-century hard sci-fi authors were not sufficiently Bitter-Lesson-pilled. Which is unsurprising, that essay was written in 2019.

Asimov’s AI was based around the positronic brain which simulated organic neurons if I recall correctly.

"positronic" is just a scifier version of "electronic"

I think you’ve misread Asimov. Asimov highlighted the problem of alignment before it had a name and generated stories out of rule-based attempts to solve alignment.

I don’t recall his robots having problems with creativity nor lying - precisely Daniel in the post-Foundation books was found to have developed a zeroth law which he used not only to lie but also to creatively get the universe back on Seldon’s plan after the Mule disrupted it. There were also stories in the robot/Spacer era pre-Foundation where robots lied precisely because they had to to save a human life or themselves. And the robots were creative - the human inspector those series followed relied on the creativity of his robot companion to help him.

Well, it is a very long time since I read Asimov himself. I may be over-indexing on the Roger McBride Allen authorised series, which portrayed robots as attempting art, but not really succeeding as being creative.

But I thought in the lying thing Daneel was unusual in succeeding.

Lying was uncommon in the sense that Asimov only had a few such stories. I don’t recall if it was treated as a complete anomaly or happens sometimes.

Liar!: robot can read minds due to manufacturing defect, realizes words can cause harm to humans and starts lying because of the first law.

Mirror image: robot lies to protect master’s reputation

Risk: perception of robot lying although in reality it followed instructions too literally

Little lost robot: weakened first law robot engages in malicious compliance.

So quite a few stories

I have a hard time suspending disbelief with sci-fi these days. It's really hard to imagine any believable future along current trend lines; either you contrive a reason why AI didn't become a thing (1), or you have to explain why humans are doing anything - piloting a starship, science, exploration, etc. (2)

(1) Butlerian Jihad. Sanderson's Skyward at least had a reason.

(2) Banks's Culture had a possible future. I'm sure there are others, but not many.

I think about this a lot, and while I have tons of near future sci-fi ideas (which probably would have seemed very far future, 10-20, years ago), I just can't conceive of anything much farther out. I guess that's why they call it the "singularity".

I think you suffer from Fisher's 'capitalist realism':

>the widespread sense that not only is capitalism the only viable political and economic system, but also that it is now impossible even to imagine a coherent alternative to it.

Star Trek and Banks' are post-capitalist futures.

Star Trek is easy to "imagine", but also easy to invalidate.

The only slightly fleshed out bit of Star Trek is the military, not the planetary society.

I'm saying that I haven't really seen (almost) anybody present a coherent future. Not just me.

Star Trek, as much as I love it, does not really grapple with AI heavily, and I already mentioned Banks.

Give Accelerando by Charles Stross a read. The future isn't over yet.

Yeah Accelerando is by far the most accurate at the "gestalt" of how AI progress is actually feeling, and how ridiculous the future is likely going to be.

Clawbook felt very "Accelerando," personal agents are kind of like Manfred's exo-cortex (AR still has a bit to go). Aineko becoming some ASI that masks its intelligence isn't that far off the mark.

The whole "AI evolves Economics 2.0 which is only understandable by humans if they eliminate the aspect of themselves that makes them human in the first place."

A lot of sci-fi novels kind of give a sanitized, streamlined, view of AI. Accelerando is really the only novel that captures what the chaos and future shock of a growing singularity feels like to be in.

> or you have to explain why humans are doing anything

Jack Williamson made a case for humans resisting doing nothing: With Folded Hands ... (1947) - https://en.wikipedia.org/wiki/With_Folded_Hands_...

Just re read this. Great book. Frighteningly prescient.

Humanity literally hugged to death and going gentle into that good night is arguably more likely and more frightening than berserker terminators.

Futurama had a propaganda film entitled I Dated a Robot in an episode https://www.youtube.com/watch?v=IrrADTN-dvg

I wonder if this novel had any influence on the writers of Pluribus.

That's an interesting call - had they read this nearly 80 year old novella / novel pair then I could certainly see that.

Have you read the Bobiverse series? It goes into some of these themes, and explores AI, post humanity and so on.

A huge problem with SF today is that it's tough to even imagine an upbeat future. That's rather discouraging. SF cannot tell us what to aim for.

As to prediction, there were some very accurate early hits. A Logic Named Joe

SF hit a dead end when interplanetary exploration discovered that off-Earth real estate in this solar system really sucks.

I think for me the biggest problem is what we understand about physics and FTL and causality. For FTL to happen, I can’t seem to get around the fact that to an observer a ball is thrown before it actually is thrown. So it’s very difficult for me to do sci-fi…

What? That isn’t at all what happens and that fundamental misunderstanding is probably why you have trouble. You cannot, ever, “see” something happen before it happens. An observer “sees” it delayed, just like when you see lightning but hear thunder on a delay. The thunder didn’t happen later when you heard it, and you cannot hear the thunder before lightning. You could, however, fly faster than the thunder and hear it twice, just like if you could fly faster than the light, you could see the lightning twice. In neither case can you see the lightning before happens, or hear the thunder before the lightning.

To get problems, you need to have two FTL ships that are in very different reference frames interact in specific ways. There's plenty of ways to avoid that without sacrificing FTL.

You just have to drop the assumption that sci-fi stories take place in the future, even if they explicitly say so. They happen in fictional universes that are similar to ours in some ways and different in others.

The umbrella term is speculative fiction. The author starts by assuming that the world works in certain ways, and then tries to figure out the consequences.

One short story worth reading, not on this list, is Manna[1] by Marshall Brain. It proposes two different futures that both seem at least possible as outcomes in a future with advanced AI.

[1] https://james.darpinian.com/manna

The end is a bit over the top, but the first half reads like current news.

While these are not far future ideas they were pretty good:

1) Masters of Science Fiction tv series from 2007 Episode 5 "Little Brother" - A person "jailbreaks" the judge who is sentencing him.

2) Terminator 3 rise of the machines - in the beginning they talk about allowing Skynet access to their military systems and so on to eliminate a virus, and they have no idea how it will react. Echoes a little bit where we are now.

These aren't perfect of course but kind of interesting to see at present day

I find battlestar galactica to be quite on point, real Ai (llm is not) will eventually either be banned or it will grow beyond humans and do its own thing.

As to prediction, there were some accurate early hits. A Logic Named Joe nailed the search engine with a chatbot front end, and pointed out the problems of systems smart enough to tell people in detail how to do bad stuff.

Much early SF thinking was that AIs would be logical. That's what I was taught, going through Stanford back when the logicians were in charge. They were trying to hammer the real world into predicate calculus. That turned out to be totally wrong.

Almost everybody in SF underestimated the difficulty of robot unstructured manipulation. So did people in AI. They still do. Most SF assumed that robots would be able to do most eye/hand tasks before they could talk. No. Still unsolved. LLMs don't seem to help all that much. Written language doesn't have much info on grasping, and watching without doing is only marginally helpful.

Asimov's robot novels were all about alignment, and why alignment is hard. That was prophetic.

Banks' Culture novels are great, but they're for a future that has a lot of new physics.

Now what? What we seem to be headed for in the near term is the fast automation of most white collar work, but not blue collar work. SF foresaw that future a few times. One example is the Rosinante Trilogy from 1981, where there's a boss of an orbital space colony, an AI as middle management, and union labor doing the real work. That's a hard SF view. There's How to Live Safely in a Science Fictional Universe, from 2010, where the protagonist's supervisor is "Microsoft Middle Manager 2.0 with passive-aggressive set to low." It's a minor part of the comedic plot there. And, of course, there's Manna, from 2003, which is looking all too likely.

[delayed]

> From a pessimistic perspective, you could think of reasons why a misaligned AI keeps humans alive. Does it simply enslave humans and use us for our fine-manipulation capabilities until it can get proper robotics up and running? Does it want to observe us scientifically? Does it want to use captive humans as bargaining chips with other civilizations as it expands through the galaxy? Or does it have some sort of quasi-religious respect for humans, e.g. reasoning that keeping us on a reservation of some kind reduces the risk that a simulator-god pulls the plug on the universe?

Or another: it just ignores us, but keeps the good resources to itself. There's still humanity, but it's brutish and primitive, regressed to the stone age, and pushed to the margins.

The fact that current AI systems manage to not wholly offend broadly 8 billion people of whom the majority believe in imaginary fairy tales and a big percentage of that kill each other for these fairy tales tells me AI can already be a far better steward of humans than other humans. You are trying to project imaginary stories from science fiction to real life. Stories are fun, but they are not real.

>And IRL, even if humans are living in relative utopia, they'll still get stir-crazy and create conflict. Even in countries that are very wealthy by global or historical standards, political competition can still be quite bitter, for example. How would a superintelligent AI manage this?

Correct, I believe people should be given a dose of real fucking pain. Otherwise they start inventing fake self victimizations.

A lot of post-cyberpunk hard sci-fi is about augmented posthumans mostly for this reason.

AI accelerated science might be the only way or fastest way to reach the stars. In a way its good to cull out the low motivation people, if I had the tech to, why wouldn't I pilot a starship and explore the stars? It doesn't matter to me if others are also doing it. Luddism will be the death of civilization.

What AI?

The ones that our brightest mathematicians are forking the sign of the cross at?

AI in sci Fi is usually shown as closer to AGI, and we don't have that yet because true AGI has actual consciousness. Which - not a neuroscience specialist but - consciousness isn't a straightforward thing to achieve, as it's not a product of raw compute but of a reflective design structure.

AI in sci Fi is usually shown as closer to AGI, and we don't have that yet because the whole field of transformer-based AI is less than 10 years old.

vernor vinge bubble in time series gives a lot of food for thought without overly patronizing or moralizing

I'm glad to see Aurora (by KSR) up there. It's a great sci-fi story overall, but for this topic the "waking up" of the AI and the entire evolution as we get to know it, and hear its voice is absolutely great. Highly recommend it, if you haven't read it.

Aurora? Great? No, it's really not. Utterly unrealistic, from both physical and just "they can't be THAT dumb" standpoints. All to push the point that the Earth is a starship.

A much more realistic example of AI and starflight: good old Rocheworld by Robert Forward.

This goes so hard with numbers without even a passing mention of how the author actually read the books. AI wrote this.

I think this quote is cherry-picked:

> I agree with Ursula Le Guin that “Science fiction is not predictive; it is descriptive.” To me, the interesting parts of science fiction isn’t how accurately it extrapolates into the future, but rather, given a future circumstance, how might be?

You can read her views at https://www.penguinrandomhouse.ca/books/342990/the-left-hand... .

"""Prediction is the business of prophets, clairvoyants, and futurologists. It is not the business of novelists. A novelist’s business is lying.

The weather bureau will tell you what next Tuesday will be like, and the Rand Corporation will tell you what the twenty-first century will be like. I don’t recommend that you turn to the writers of fiction for such information. It’s none of their business. All they’re trying to do is tell you what they’re like, and what you’re like—what’s going on—what the weather is now, today, this moment, the rain, the sunlight, look!"""

Science fiction is about the present, even when it describes the future.

AI from the past seems more like it’s based on the computers of the time - the AIs seem to be very logical, never make mistakes and act like they’re an expertly programmed bunch of switch statements and ifs. Whereas modern AI is based on human text and so the models have human biases and do things like swear in their chain-of-thought and sometimes make errors for no reason at all.

People call it slop, but that's what we raised it on. Common Crawl is >99% slop.

(I read some bits and pieces, would not recommend!)

[delayed]

The sci-fi story that I constantly find myself thinking about when I read about real-world AI developments is Ra, by qntm/Sam Hughes: https://qntm.org/ra

I think qntm might be one of the greatest sci fi writers to ever live. I put him with Greg Egan.

That jump in computer intelligence, so soon, is completely surprising. And I bet most actual sci fi fans would agree.

I used to dismiss anything near-future involving any robot or AI you could just talk to. Now it exists. Unfathomable.

There's also "Destination: Void" by Frank Herbert.

A bunch of clones manipulated into a high-pressure situation to accidentally/on-purpose create god… again.

It might not have predicted LLMs but it certainly fit with Herbert's ongoing thematic study of people in insane environments producing unanticipated outcomes.

It's been a while since I read it, but I vaguely remember something about the way they were creating the AI, resonating with recent ai (llm?) developments at the time.

Well it's been more than 40 years since I read it, so trust your memory!

If you haven't, you may want to check out the other books in Herbert's Pandora Sequence (Destination: Void is the beginning)… but they are a bit of an acquired taste.

Neal Stephenson's Diamond Age (a young lady's illustrated primer) ≈ LLMs + Intelligent UI (OpenAI)

Fun fact: most sci-fi writers really dislike LLM for various reasons, whether it is the generative works they produce, or training on their works and allegations of copyright infringement. Many sci-fi writers have espoused a very negative attitude towards LLM and actively refuse to use Claude or GPT even for coding projects. John Scalzi comes to mind…

Many <s>sci-fi</s> writers really dislike LLM.

Charlie Stross dismissing AI as "just spicy autocomplete on steroids" three months ago <https://np.reddit.com/r/accelerate/comments/1vbtdga/what_is_...>, for example

> Many...

https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Clar...

As my parents liked to say at the dinner table when I was a picky eater, "You don't like it? Great, then that's more for us".

I think the score for Colossus: The Forbin Project is too low. There is nothing in the film that contradicts the notion of Colossus being a very, very large LLM. Although I think the film is even better than the book by D. F. Jones, only the latter mentions how, despite being created specifically for US national defense, Colossus is also fed unrelated data including Shakespeare's sonnets, because its creators do not know if it could be important.

>The two machines begin with arithmetic and soon communicate through mathematics their engineers can’t follow. That’s the most relevant prediction here: machine intellectual activity racing ahead of the people supervising it.

This misses the specificity of the scene's prediction. Colossus and Guardian are basically speaking in the style of the super-dense, almost incomprehensible-to-human chain of thought that LLMs indulge in nowadays. One might also point to the just-released news of LLMs easily seeing patterns, incomprehensible to humans, in model weights.

Sci-fi writers were generally wrong or over-simplistic about many predictions, mostly just extrapolating whatever the current zeitgeist of their time was, and stretching it out to 50 or so years into the future

I mean look how most science fiction of the 1980s depicted the world of the future as dominated by Japan... Look at Robocop 3 :)

Computers were then just appearing or increasing in public discussion, along with fears that they'll make humans less human somehow: Like in this Computer Chronicles video about word processing from 1983 at 4:08: "Some writers are concerned that computer assistance may promote dry bland writing."

https://www.youtube.com/watch?v=Jt0OoXluC8g

Terminator appeared in 1984, and the first Alien movie in 1979 had an antagonist android. 2001 A Space Odyssey with a disagreeable AI came out in 1968. All way before the real world had anything remotely close to today's AI: It was a reaction to the appearance of computers themselves.

On the other hand, Star Trek had optimistic AI like Data.

Humankind's own sentient creations leading to the ruin of humans is probably a trope in fiction since thousands of years, with golems or summoned demons etc.

You may want to reconsider your history. Benign and even helpful AIs on the screen which predate Terminator include:

Robbie the Robot (Forbidden Planet, 1956) "The Robot" (Lost in Space, 1965-1968). Twiki and Dr. Theopolis (Buck Rogers, 1979–1981), Zen[1] and Orac (Blake's 7, 1978-1981), R2-D2 and C3PO (Star Wars, 1977).

Computers were part of public discussion well before the 1980s.

Nixon's famous radio 1974 radio address was about the American right of privacy: "There are, of course, many facts which modern government must know in order to function. As a result, a vast store of personal data has been built up over the years. With the advent of the computer in the 1960's, this data gathering process has become a big business in the United States--over $20 billion a year--and the names of over 150 million Americans are now in computer banks scattered across the country." This lead to the Privacy Act of 1974.

The late 1960s reaction to how computers were used was to bomb and invade data centers. 1970 was "the year of the crisis", according to Computerworld in 1971. https://archive.org/details/sim_computerworld_december-30-19...

[1] "I have failed you. I am sorry." still brings tears to my eyes.

Ah yes, thanks for calling that out

The general air is so pessimistic that we forget about those!

I didn't forget. I limited myself to some pre-Terminator good/benign AIs in TV and movies. ;)

[delayed]

This question assumes "this is it, this is AI", which I find laughable. A necessary stopgap accelerator at most, I think people will look back at LLMs like toys in the future, and our societal bewonderment as a result of mirage.

Nothing before 2017? What about:

  - Sally, Asimov 1953 (spot on with Waymo)
  - I, Robot etc, Asimov
  - HAL 2001 A Space Odyssey
  - any Iain M. Banks Culture novel

A compete miss: Sci-fi author and computer scientist Vernor Vinge popularized the term programmer-archaeologist in his 1999 science fiction novel A Deepness in the Sky

The premise is that AI never came around and we were left scavenging code since that was more efficient than writing new code. But with LLMs, it’s much easier to write new than to scavenge.

I never did read his stuff. But aren't libraries effectively this? Not scavenged, but it is more efficient to write small chunks of reusable code and reutilize them everywhere than to constantly write new code

This the libraries are easy to find, reason about, and are easily adapted to your use case, yes. But the SciFi novel put this process more as arduous process because there was just so much code to sift through.

There's a few smaller sci fi stories that are not covered here, like the crystal society stories. That one was deliberately written to explore AI using modern AI architecture. Not popular, but very accurate.

I'd also like to note that the short fiction stories published on LessWrong are a direct contradicion to this. They're sci fi. Not published for profit, but still.

I think Avogadro Corp deserves a higher score on purely a vibes level. when LLMs first came out I just immediately thought of “elope” the AI in Avogadro Corp which was basically a text generator.

Person of Interest missing. Watching again right now - awesome. Listed as science fiction for series 1 ... then as drama by series 5. Well, so it's said.

Is it just my narrow selection of experience, or do all long-running sci-fi series eventually devolve to focus on ordinary interpersonal drama over focusing on exploring speculative scenarios?

It sure happens a lot. I got tired of the Expanse when it turned into just being geopolitics in space. Still good in it's own way, just not what I'm there for.

I guess Revelation Space stays speculative. It just kinda gets worse after a while. It doesn't have a lot of character continuity between books, which might help with leaving the stage open for wacky scifi concepts. (A lot of the books are really good, but Redemption Ark was meh and I found Absolution Gap disappointing.)

I'm disappointed Hitchhiker's Guide didn't get a mention. Genuine People Personalities feels like something Anthropic could launch shortly.

Nothing does human like humans do.

We are organic and so survive on organics well; we are low power; and myriad other evolved traits that put us in equilibrium with the surrounding Earthly environment.

From a dynamical systems perspective, we operate in a niche. Sure, maybe there are still some tweaks left. But machines approximating that niche sufficiently will just be human in the end.

The real question is why we are adamant about being non-humman, so to speak. Therein lies the tautology that AI will exceed us in non-humman areas. Of course it will.

One of my favorites that I still think about is The Wedding Album by David Marusek[1]. In the short story, agents are brain copies that are used to do anything and everything, and then thrown away. Eventually, they're given rights, and society falls apart for a while and then rebuilds better.

[1] https://www.isfdb.org/cgi-bin/title.cgi?84509