The way this is done in every domain where the real is a bottle neck is with simulation. So the actual argument is: What domains do we think are not practical to simulate even with extreme intelligence making the simulation and the ability to bring immense amounts of computing power to bear while running.

I don't think there are actually that many domains where you can't simulate them with enough knowledge and compute. So then the bottleneck becomes your ability to sample the world and refine your models. For some domains you can have the models run against each other in your sim which greatly reduces the need for real world feedback (see AlphaZero for example).

So in the case of human persuasion, like asking a boss for a raise, the model could of failed many times a second while trying to persuade a simulation of a typical middle manager with the same parameters as your boss.

That's a very "Spherical cow in a vacuum" approach. You would need to accurately model the boss, as well as all your interactions with the boss up until your raise request, as well as the boss' environment when you make your request.

You could be on the wrong side of a reasonable boss, or in his good graces but during a bad day ...

That's the point the writer was making when he mentioned role playing games. In isolation, you could spend hours drafting the perfect plan, that wouldn't survive two minutes of contact with reality. You just don't know if your model is correct until you try.

But how do you source the observations used that would be the basis of your simulation? Or are you assuming the real world won't change? At least in my mind I don't see many domains that remain static enough where continuous sampling of the real world will not be required. Go and Chess are artificially stable domains, the rules are fixed.

Any social domain where there's competition of resources (salary, funding) will definitely not be stable, it would require constant testing against the real world.

> I don't think there are actually that many domains where you can't simulate them with enough knowledge and compute.

The problem is that you have to simulate them accurately enough and fast enough. A perfect simulation of physics will necessarily be orders of magnitude slower than physics. To accelerate it, you need to figure out which parts can be cut or approximated without derailing the model, and just that part will require a lot of back and forth with reality.

Success isn't guaranteed either. If a physical system does whatever it does optimally enough, fast simulation is just physically impossible. Furthermore, even simple chaotic systems require unrealistic precision in the initial parameters to be useful. Natural entities are fairly chaotic and also pretty darn efficient at what they do, so it's plausible that even the best possible simulation of a middle manager is a slower bottleneck than a real middle manager.

Ok, but this just means that model predictive control won and you're not modeling just individual concerns but rather, you're modeling the entire world.