We don't rely on the brain though, at least not on any single one. Any system that relies on human brains alone without cross-checking or, ideally, much simpler automatic systems, will eventually malfunction terribly. A large organization never wants to rely on a single person's judgment for anything, a programmer wants automated systems checking their work, etc.

We rely on a single human to drive a car.

In medicine, machine learning systems will work alongside other brains.

What are the instances you’re imagining where a group of brains running an important system are replaced by a single machine learning algorithm running in isolation?

Yeah, and around 800 people a day die in auto accidents in the US. Bringing up cars bolsters my point, which is that "we rely on the brain without understanding it and that works out fine" is not a good argument. So let's be careful. I'm mystified as to why that seems to be controversial.

Are you saying that we should require 2 drivers per car? It seems like at the moment the risk/reward ratio of allowing people to drive a car after getting a license seems to work out well enough. 1 death per 100 million miles driven, and falling.

No. I'm saying

> that "we rely on the brain without understanding it and that works out fine" is not a good argument.

It wasn't my idea to bring cars into it particularly.