It looks comfortable in the Star Trek universe because everyone trusts Starfleet and the ship's computer.

I mean, if you don't trust the computer to look after your interests with regards to your voice interactions, why are you trusting it when it also controls life support and airlocks? ;)

Another classic SF example of a voice interface: "I'm sorry, Dave..."

But what's an AI, what's 'artificial', when does something become so if it's made by/of 'natural' things? Which reminds me of BSG, they have airlocks too.

It isn't. In fact, people there are uncomfortable about computers giving them orders (TNG S03E06). It works because in Star Trek, the society works. For instance, a ship's captain could easily access plenty of surveillance records about their whereabouts and activities, but as a matter of course, they don't. Cases where someone abuses technology for ulterior reasons are presented as negative, and protagonists do their best to thwart such misuse.

In TNG they ask the computer for someone's location every other episode, usually without hesitation. Didn't go so well for Barclay, when they walked into his cringy holodeck fantasy program.

> In TNG they ask the computer for someone's location every other episode, usually without hesitation.

Which is appropriate. There's nothing wrong about locating a crew member on a _military vessel_.

They do, but they never abuse this capability.

I'd disagree with this. Frequently they look through peoples records. Whenever someone dies it is the first thing they do. They also on one hand talk about holo decks being a private space but how often do we see them walking in on peoples' sims. You could totally just shut down the sim from the outside if the person inside is not responding and that would more readily respect that person's privacy (but isn't good for entertainment nor shows that Star Fleet is also composed of "humans").

When someone dies or goes missing, not even an hour goes by before someone like Geordi or Riker is sifting through their personal log entries. :)

QUICK! Let's look at their personal logs!

Like come on guys, there's more than personal logs. There should be more barriers before you go there. But to also be fair, it is stuff they know is going to be used in that way and it isn't just a log of everything they've said in their room. So there is a little more consent involved. I don't think you can really give consent to always listening devices.

I disagree. Given it takes place hundreds of years into our future, I'd assume speech recognition is just a "solved problem" and not something to worry about. Plus, there's also the Federation -> navy comparison. If speech recognition is mandated in Federation vessels, why would the crew worry about who's listening?

>a society essentially controlled by the AI, as it will be above us.

This is the highest probability of the realistic outcome of AGI as it will advance in capabilities so much and so rapidly that we won't be able to keep up. I personally think it will be benevolent and care for us like we do for our pets which won't necessarily be a bad thing.

If you're interested in reading about AI goal theory further, I suggest Superintelligence, orthogonality and CEV as books/topics you could explore.

These are fantastic recommendations, thanks!

I think it looks comfortable in the Star Trek universe because Star Trek portrays a society that is not run by mobsters, warlords, delusional ideologues, and sociopaths.

We are still very much savages. Clinical psychopaths and delusional narcissists run our institutions and major decisions are made on the basis of fanatical superstitious ideologies and mindless herd behavior.

In short I won't put an Internet connected microphone in my house and I'm very wary of privacy leakage in general because I don't trust the people who run our world (including both governments and corporations). I don't think this is irrational. Glancing around it seems like a decent number of world government and corporate leaders belong in prisons and mental institutions, not powerful offices.

> Star Trek portrays a society that is not run by mobsters

But also portrays societies run by and indeed composed entirely of mobsters (see "A Piece of the Action" TOS S2E17).

It does, but the protagonists are part of the utopian society; in that context, technology actually works to benefit people.

> Star Trek portrays a society that is not run by mobsters, warlords, delusional ideologues, and sociopaths.

Interestingly, while this is certainly the background assumption of Star Trek, it is much less valid in episodes where the crew must interact with their own government.