This sounds very close to AGI capabilities. Maybe I've just been thinking too much about Westworld, but I'm starting to get very concerned that people can be gradually replaced by AI agents with their own agendas (focus on this last part).

At some point they probably will be, but it seems unlikely to happen within the next 7 years at least. Deep Learning still has the same deep-seated issues it has had since its inception.

I don't think it's a question of incremental progress, with AGI behind some ghost line. I think the current progress is, despite all the excitement, actually orthogonal to true AGI development. We're developing ever more sophisticated narrow AI, not even taking baby steps to proper true AI. I believe that the progress to the latter would be evidenced by the development of artificial intelligence first with animal capabilities, then with child-level capabilities and ultimately adult human tier capabilities. But I could be wrong - perhaps our current navigation of the problem space will reveal some strange pathway to true AI.

> But I could be wrong - perhaps our current navigation of the problem space will reveal some strange pathway to true AI.

Most likely wrong if you compare to how most other development in engineering happened.

From my point of view, the most likely path towards some form of weakly general intelligence at this point is emergence. We keep working on these narrow problems and from the broadening networks at some point we end up with something indistinguishable from a general ai inadvertently.

What engineering development are you referring to? Do you have any examples?

Engines, planes, PVs. They all gradually became better in a way which doesn’t really map to the organic function they augment, replace or take inspiration from.

Planes don’t fly like birds. There is very little reason for what you would call a “general intelligence” ai to develop in a way which mimics our own intelligence. It might but I would find that more notable that if it did not.

I think the biggest difference between building a general intelligence and other engineering problems is that in the latter, the target function is well-defined. With planes, anything that is able to fly safely goes. Nuclear reactors, anything that generates electricity. But with intelligence... What does it need to do, exactly? The 'moving goalpost' is moving because we have no idea where it should be. We're building AI blindfolded. I don't think we're currently solving the problem, simply because we don't know what the problem is.

It's just a pattern matching glorified search engine! He screamed as his legs were converted into paperclips.

> We're developing ever more sophisticated narrow AI, not even taking baby steps to proper true AI.

I'm fairly certain you cannot rigourously define "proper true AI", and I'm also fairly certain you don't mechanistically understand how humans think. This raises the question: where does your confidence that our current path is not pretty close to "proper true AI" and/or how humans think come from?

The only intelligence with human capabilities is, so far, human intelligence. We can see how nature incrementally improved 'the algorithm' when we look at other animals, from simpler ones to primates. We don't see the capabilities of even lower animals in modern deep learning based networks. We know these networks don't match in neural function what we see in biology. Whether deep learning will lead to the North Star of human intelligence is guesswork, and guesswork rarely works well with complex systems.

> We don't see the capabilities of even lower animals in modern deep learning based networks

Machine learning models have written multiple scholarly papers that have been accepted to prestigious journals. Find me a rabbit that did that.

But, but, but that is just tricks unlike human intelligence which is based on no trickery soul fuel.

/s

You're in it!

FYI you are quite literally describing dungeons and dragons, just virtualized and automated. But on that topic, I've had a sort of similar idea kicking around in my head, but on a bigger scale.

I want to run a game of DnD using a heavily modded Crusader Kings or Civ to simulate the world they are playing in. Just set up the starting conditions, like the various kingdoms, their methods of succession, their relationships and blood feuds with other kingdoms, the internal treacherous machinations and possible rebellions brewing, wars between kingdoms, etc, and then just let it run and use that to generate the overarching plot events of the setting while somehow translating the players actions as some kind of input into the simulated world so they can affect it as well.

Check out https://character.ai

This is what a next generation Skyrim should be.

Funny you say that. For Elder Scrolls 4 (Oblivion), the devs claimed to have built something like this. Where all the NPCs had their own wants and needs and schedules, like The Sims. Everyone needed to eat, but not everyone had a job. So some NPCs resorted to theft. But since crime and witnessing crime was already a system programmed in to the game, this often meant that by the time a player arrived at a town, everyone was dead. Some beggar took a bread roll from a shop keeper, who alerted the guards, who confronted the beggar, who responded with violence, who started a scuffle, which had collateral damage (many people have fireball spells), which lead to a free for all of death. It ruined the game, and they had to scrap most of the system.

maybe the guards should't default to violence?

We already have this, it's just a matter of cost now.

Cost and consumer hardware. These models take a lot of resource to run inference and most consumers don't have top hardware to run this stuff on.