Maybe the human is the rng or temperature or lava lamp. At least until we can model and predict each brains tendencies with accuracy.

I think we'll reach a tipping point like we did with DNA sequencing where we figure out how to quickly map out all the unique patterns of enough brains to model one that can understand itself. People worry too much about rogue AI, and not enough about the CRISPR of brain mapping being used to inject patterns into meatbrains.

Strange Days not The Matrix is the prescient fictional warning.

A black market of taboo “memories” aka experiences. A desire for authentic ones over synthetic diffused ones, leading to heinous crime.

There were plenty of Star Trek episodes where it seemed like they should just ask the damned computer.

Then again, Data did show his faults, particularly not having any emotion. I guess we’ll see if that’s actually relevant or not in our lifetimes.

As far as that last part goes, I think we already have ample evidence that bots can, if not have emotions, then pretend that they do (including wrt their decision making) well enough for humans to treat them as genuine.

I met Garry Kasparov when he was training for the Desp Blue match (using Fritz).

He lost to Deep Blue and then for 10-15 years afterwards the chess world consoled itself with the idea that “centaurs” (human + computer) did better than just computer, or just human.

Until they didn’t. Garry still talked like this until a few years ago but then he stopped too.

Computers now beat centaurs too.

Human decisions will be consulted less and less BY ORGANIZATIONS. In absolutely everything. That’s pretty sad for humans. But then again humans don’t want or need this level of AI. Organizations do. Organizations prefer bots to humans — look at wall street trading and hedge funds.