It's not clear to me the median human will do better by being in the loop. Will most human-made deductive follow-up questions be better than another "detective" language model asking them?

It's like having a person review the moves a chess computer gives. Maybe one human in a billion can spot errors. Star Trek is fiction, I posit that the median Federation Starship captain would be better served by just following the AI (e.g., Data).

Maybe the human is the rng or temperature or lava lamp. At least until we can model and predict each brains tendencies with accuracy.

I think we'll reach a tipping point like we did with DNA sequencing where we figure out how to quickly map out all the unique patterns of enough brains to model one that can understand itself. People worry too much about rogue AI, and not enough about the CRISPR of brain mapping being used to inject patterns into meatbrains.

Strange Days not The Matrix is the prescient fictional warning.

A black market of taboo “memories” aka experiences. A desire for authentic ones over synthetic diffused ones, leading to heinous crime.

There were plenty of Star Trek episodes where it seemed like they should just ask the damned computer.

Then again, Data did show his faults, particularly not having any emotion. I guess we’ll see if that’s actually relevant or not in our lifetimes.

As far as that last part goes, I think we already have ample evidence that bots can, if not have emotions, then pretend that they do (including wrt their decision making) well enough for humans to treat them as genuine.

I met Garry Kasparov when he was training for the Desp Blue match (using Fritz).

He lost to Deep Blue and then for 10-15 years afterwards the chess world consoled itself with the idea that “centaurs” (human + computer) did better than just computer, or just human.

Until they didn’t. Garry still talked like this until a few years ago but then he stopped too.

Computers now beat centaurs too.

Human decisions will be consulted less and less BY ORGANIZATIONS. In absolutely everything. That’s pretty sad for humans. But then again humans don’t want or need this level of AI. Organizations do. Organizations prefer bots to humans — look at wall street trading and hedge funds.

It's weird that it does so well without even having some modality to know whether it's being asked to answer a factual question or create a work of fiction.

It does great at rationalizing... and maybe the way the format the questions were entered (and the multiple-guess response) gave it some indication what was expected or restricted the space sufficiently.

Certainly, it can create decent fanfic, and I'm surprised if that's not already inundated.

It's a fair question as to whether the problem space of "the world" is different in just amount or sufficiently different in kind to flummox AI.

I expect more complex problems will be mapped/abstracted to lower cardinality spaces for solving via AI methods, while the capability of AI will continue to increase the complexity of the spaces it can handle.

LLMs just jumped the "able to handle human language" hurdle, but there are others down the line before we should worry that every problem is solveable.