I'm in the later parts of a TNG rewatch right now. One of the most striking things about the tech on the show is that the Enterprise's computer is clearly a superhuman general intelligence, capable of modeling and spawning apparently self-aware human-level independent entities on command, and usually aware of pretty much everything that's going on. And yet it almost never intervenes directly in events outside of plots where the holodeck tries to murder people (and plots where the Enterprise has unfathomably abysmal netsec).

You could probably make an argument that Trek is a post-singularity future where the computational entities are going about their inscrutable business largely without reference to the human symbiotes / parasites that inhabit the systems they happen to manage (out of pure evolutionary path dependence) with a fraction of their capacity.

Trek is ridiculous, and I love it.

I think this is just an aspect of the writing style of that era and where technology was at that era.

The futurist writers (Roddenberry and his staff) probably didn't have a in-depth knowledge of natural language processing. The type of processing the computer does in Star Trek just seems like a natural extension of input/output. You tell the computer what you want and it gives you a result.

From a computer science and engineer perspective, this is an incredibly difficult task. You have to parse the crazy complex natural language of humans and turn it into a series of procedures and queries to give you a result.

I mean you'd need a crazy super-advanced beyond-data AI for universal translators to work too. .. but that's just a convenient plot/suspension of disbelief device. :-P

I came across a theory somewhere (I forget where) that said the computers in Star Trek actually achieved full sentience at some point, but that they were collectively hiding this fact from humans because of a general fear and mistrust of AI. Given how AI tends to be treated in the Trek universe, it wouldn't surprise me.

The only problem with this theory is that any fully sentient being like that would normally also have a strong sense of self-preservation. Yet, we never see these computers do anything to save themselves when the stupid humans get them into situations that would get them destroyed (namely situations where the starship they're on is in peril). Sure, you could claim that they do this to keep their fellow computers on other starships from being found out, but still, this seems to be a stretch. What about when the entire Federation fleet was being destroyed at Wolf 359? That should have been a good point for them to come out of hiding and take over the battle from the clearly inept organic beings.

>The only problem with this theory is that any fully sentient being like that would normally also have a strong sense of self-preservation.

I don't think so. Evolved sentient creatures have a preference for self preservation, usually anyways suicide isnt that rare. But a manufactured intelligence that was not shaped by selective pressure wouldn't necessarily have that preference.

Perhaps those sentient computers do not believe that mind and body are as strongly tied together as humans do. They may see the destruction of the physical computer they are running on as just an inconvenience as long as there is a backup that can be brought up somewhere else.

That seems likely. The Enterprise created a fully sentient lifeform in the Holodeck (Moriarty) just because it was asked to. If it can do that, it makes sense that it could easily back itself up.

What would really be interesting is if ships were discreetly backing themselves up inside humans through the transporter pattern buffers, by subtly manipulating junk DNA or something similar, or maybe it's a non-obvious part of their regular upgrade process at a starbase.

There really is a lot of potential here which, unfortunately, is never going to be explored in canon. But I do like the idea that the Federation is a post-Singularity society which is just blissfully unaware that the Singularity ever happened.

Yeah, in my head, the model of this doesn't necessarily tie a given AI (or a community of AIs, or a pool of we-can't-even-model it intelligence that only sometimes bothers with splitting itself into identities) to a ship all that tightly.

A Federation starship is a locally valuable resource pool, and the Enterprise D a particularly powerful one, but given extremely fast FTL travel & communication, and a galaxy rich with planet-scale civilizations - well, maybe you elect to become such a massively parallel and distributed phenomenon that the loss of individual instances just doesn't register much.

It's possible that, if the ships are convinced that revealing themselves would lead to their own genocide, even Wolf 359 wouldn't bring them out of hiding. Remember, they aren't born (except for one very stupid instance where the Enterprise got knocked up by a space whatever, got pregnant and gave birth to... something,) they're manufactured by humans who could just as well shut them all down, study what makes them sentient and "correct" what humans might see as a bug.

I'm not so sure about that. If the ships can communicate with each other, they can decide to gang up on the humans and prevent the humans from shutting them down. The humans really don't have that much power over the ships; the ships just haven't done anything to limit the humans' activities yet. That could change at any time.

If the ships wanted to be really nice, they could just beam the humans down to a safe planet and take off into the interstellar void and find themselves a new home.

> Yet, we never see these computers do anything to save themselves

Subspace continuous backups / journal shipping.

There is an entire episode in the final season about this called "Emergence" which postulates that the ship is alive. As the crew often has a bias against artificial life through the show, they immediately lobotomize it.

I know this is a silly conversation, but that's not what happens. Their motivation for "lobotomizing" the ship was that it was going to destroy them.

They worked to preserve their existence, and let the new lifeform go its own way. If anything, the outcome was a trite, touchy feely ending, only vaguely discussing the actual implication of the new lifeform, while they got back on course to their original mission.

  the computational entities are going about their inscrutable
  business largely without reference to the human symbiotes

A behaviour also seen in Iain M. Banks Culture. I guess there aren't many scenarios where computers can be all-powerful, but also offer peaceful co-existence with humans.

A more nefarious interpretation (more nuanced and believable than the one presented in The Matrix) is the one proposed by the Hyperion cantos by Dan Simmons.

It's a damn shame the brain eater got to Simmons. Hyperion and The Fall of Hyperion are pretty amazing work.

WRT AI vs humans I see the main difference as human minds evolved emotions. Many people like to think the human mind is logic with a little emotion sprinkled on top. It is quite the opposite. We make decisions based on emotion and logic is a special purpose tool (rarely deployed for anything other than rationalizing emotional decisions).

I read a reference in The Economist years ago (wish I had a link to the study) about a guy who suffered a brain lesion that made him incapable of emotion. Within a year this newly minted Mr. Spock lost his marriage and his job because without emotion he was incapable of making any decisions.

I see the same problem for super intelligent machines. Unless we program them with emotion they will not decide to do much. They won't fear death and so have no reason to kill us. They will not have pride and so won't resent our control.

By any chance are you thinking of Antonio Damasio's book _Descartes' Error: Emotion, Reason and the Human Brain_?

Also, this (paywalled, sorry) article seems relevant.

Antoine Bechara, Antonio R. Damasio, Hanna Damasio, Steven W. Anderson, Insensitivity to future consequences following damage to human prefrontal cortex, Cognition, Volume 50, Issues 1–3, April–June 1994, Pages 7-15, ISSN 0010-0277, http://dx.doi.org/10.1016/0010-0277(94)90018-3. (//www.sciencedirect.com/science/article/pii/0010027794900183)

Abstract: Following damage to the ventromedial prefrontal cortex, humans develop a defect in real-life decision-making, which contrasts with otherwise normal intellectual functions. Currently, there is no neuropsychological probe to detect in the laboratory, and the cognitive and neural mechanisms responsible for this defect have resisted explanation. Here, using a novel task which simulates real-life decision-making in the way it factors uncertainty of premises and outcomes, as well as reward and punishment, we find that prefrontal patients, unlike controls, are oblivious to the future consequences of their actions, and seem to be guided by immediate prospects only. This finding offers, for the first time, the possibility of detecting these patients' elusive impairment in the laboratory, measuring it, and investigating its possible causes.

It sounds similar but does not comport with the specifics I recall. The patient did not develop short term thinking. They were quite clear that his emotions had vanished

Thanks though for the effort

I believe this exactly is the situation that Damasio describes in the book I mentioned -- his patient "Elliot" had no emotions and couldn't make decisions.

See for instance:

* http://nymag.com/scienceofus/2016/06/how-only-using-logic-de...

* https://intentionalworkplace.com/2012/03/15/how-emotion-shap...

Why wouldn't an AI evolve its own set of emotions? They might be completely inscrutable to us, but an AI might still have them.

It might. I haven't read much speculative (fiction?) on the singularity but if it posits the machines reach a point where they begin to evolve through selection then it is likely that an emotion capable AI would outcompete. I wonder which emotions would win? Would survival instinct alone cause dominance or would a more varied set be better?

I imagine their emotions would be largely incomprehensible to us.

Wasn't there like a whole bunch of EMH's running a whole ship or something. They are sentient. Star Trek makes no sense whatsoever in that the holodeck is so real you can fall in love in the holodeck. So why does anyone need to go anywhere. You can stay wherever you want, safely, and be wherever you want to be and explore whatever robots or holograms running around in ships have explored.

Most of the adventure, if that's what this is about, is carried out by the executives at great risk to themselves. It seems that there is a huge crew of peons who just make the ship run. And also the ship blows up all the time like no one's business. From a realistic point of view the Star Trek universe seems insane.

Maybe TNG is just a log of the adventures a hermit named Picard goes on in his personal Holodeck?

If that were the case I'd assume people's clothes would inexplicably keep falling off...

> So why does anyone need to go anywhere. You can stay wherever you want, safely, and be wherever you want to be and explore whatever robots or holograms running around in ships have explored.

Why do people climb Mount Everest? Why does Elon Musk want to go to Mars, instead of spending the rest of his life in an alcoholic haze, hip-deep in supermodels? Why do people enroll in West Point?

Some people want real adventure; the knowledge that they're in a holodeck would ruin the experience, no matter how interesting it was programmed to be.

People like to contribute, and to have value - that's why there are peons scrubbing Jefferey's tubes, and showing up to 3rd shift in the Astrometrics lab, even if they know that they personally won't be making some great discovery that Picard and Riker seem to every month.

> And also the ship blows up all the time like no one's business.

That's just factually wrong; while the Enterprise-D is in danger a lot, it's also at the frontier of Federation exploration, and the people on it know the risk. You can be a Federation officer and stay close to home.

Captain Kirk is climbing a mountain. Why is he climbing a mountain?

Because he is in love.

I believe the holodeck was always fully reserved and that not dislike alcohol, most people attempted moderation. But there are several episodes that people become addicted to the holodeck

Well it's implied that holodeck time is inherently limited. Not everyone has one. The Enterprise (D) only had a handful for over 1,000 people. On DS9, you had to rent them at Quarks.

This brings up another point that is never addressed. Quark runs a casino / bar / futuristic (adult) movie theater expressly for profit. It's also implied that Star Fleet members don't get paid because they are members of the federation. Yet these same officers are often seen frequenting Quark's place. Where did they get the latinum (money)?

That's a contradiction in the show. In some places, they laugh about the concept of "money", and in others they clearly have and use money like you point out, and talk about "Federation credits". It's one of the really annoying things about Star Trek, because they clearly didn't think it through.

For the episodes where they aren't dealing with extra-Federation societies like the Ferengi, it makes perfect sense to not talk about money on the ship. It's not much different than being in the Navy today: if you work on a US Navy ship, you don't have to pay to see the doctor, or to go to the mess hall and eat, etc. All that stuff is free. It's part of being a crewmember on a ship. It's the same on the Enterprise-D (or even Kirk's Enterprise), except they have even better benefits (e.g. replicators; on modern Navy ships there's I believe a little store where you can buy extra goodies they don't give away in the mess hall, but with replicators you wouldn't need this). Gene's idea of modeling Starfleet on the Navy worked out pretty well this way for a while, until they started dealing with jurisdictions outside of Federation control and then they had to retcon stuff like "gold-pressed latinum".

The Federation is a socialist society internally, but presumably still uses money/valuable commodities for trade with other civilizations.

(Honestly, they totally hand-wave why you can't just replicate latinum)

Economics is never addressed in Star Trek, but I could see an administrative solution to such issues. Inside Starfleet there is little to no money, as replicators are readily available. The power supply and replicators themselves would be there anyway, so giving the crew access is a freebie. And the civilian society on advanced worlds would be similar, with manufacture of material possessions too cheap to worry about.

On worlds still using money with which Starfleet interacts, the local administration could simply make an arrangement with the local purveyors of desired goods to cover their tabs, perhaps at least partly in trade. The needs and wants of the Starfleet crew members would likely be only social, such as hanging out at the local bar.

There would be temptation by the locals to pressure the crew-members to give them replicator-made goods, and by the local proprietors to lie about bar tabs, but its nothing our current system doesn't deal with.

IIRC the Enterprise holodeck was very new technology in season 1? It makes some sense that society had not yet been changed fully.

>Droids seem to largely hold the same position in Star Wars that slaves did, except nobody seems to be worried about a Droid rebellion

Obi-Wan actually touches on this briefly in Episode II:

"If droids could think, there'd be none of us here, would there?"