If they didn't fully understand how it worked, but fraudulently represented that they did, they'd be on the hook. It all hinges on what he was promised when he bought it. If they had a good disclaimer, then he might be out of luck.

There are places where adopting AI is going to be difficult for this reason - notably anywhere that has a legal requirement to explain why a decision went a particular way. That includes things like insurance risk models, pension investments, etc.

The losses made by software training may be the least problem for the institution.If case goes to court,they may be required to reveal the reasoning behind software's decision and that means going through the code,as all that marketing bs called AI is just a trading algorithm trained on some datasets. No company would want to reveal the code.

In these cases, I wonder if it would cause a movement toward more interpretative models with more mathematical backing.

For example, if not being able to explain a model leaves an organization open to litigation, they may instead rely on statistical based learning methods even if they performed less well.

What if some other person really understands how the robot worked and half of the money you lost ended up directly in their accounts?

Is cheesing an algorithm illegal?