Agree with these comments. Like maybe the "AI" found some "Hand of God" move for the economy....

Like many solutions that AIs come up with—such as some of AlphaZero’s game-winning moves—the result appears counterintuitive and not something that a human might have devised.

...or maybe their model doesn't include the fact that when you raise taxes on the poor too high, in eras of tremendous inequality, you get bread riots, political violence, and eventually, revolutions.

Not to say this kind of modeling isn't potentially useful!

Just modern monetary policy already uses algorithms etc (Taylor rule optimization etc), so the question is how can ML/AI supplement those humans with better tools, rather than thinking a computer is going to be able to deal with the complexity of the real world.

Put another way, what does the machine do when interest rates hit 0?

The so called A.I. was not working with anything that resembled a real economy. Standard economic models also have this problem, they also have other problems, but this particular example is just doubling down and then some. What is particularly unfortunate is that there are people out there trying to create real agent based models based on actual simulations of the Economy that are far more advanced than this (realistic markets, accurate simulations of the Banking System) but are not getting any of this kind of exposure.

Salesforce obviously is backing its own product, but the MIT Technology Review should really know better.

If the AI workers can't get sick, die, starve, suffer reduced productivity from chronic health issues, or care for children, then I don't see what lessons they can really expect to learn from this model. If their poor AI workers can just keep working hard gathering wood and stone forever, being fully productive without ever eating any food so they can afford to pay their taxes, then it's disconnected from reality beyond anything I'd even consider a "toy model".

I wonder if it's not punishing the poor, but trying to encourage them to earn more. Of course, this might not take into account the nuances of the real world where some people just can't because of constraints like children, location, disability, etc. That's where AI simulation might break down.

The more I think about it, the more I think the reason is that the policy network simply doesn't have training data for incomes that low so the shape is basically random.