Distributed systems as a whole can still suffer from the same kind of issues that you would expect to see in a monolithic system. Look at how a single bad software update can take down Google or AWS.

At a basic level this kind of sytem gives you lots of low latency bandwidth connecting CPUs to each other and CPUs to storage. That is a useful quality to have and not neccessarily easily repoducible using network and internet connections between servers and data centers.