It has nothing to do with responsibility. A tower is not magic. There are well defined protocols for using an airport with a tower that are perfectly safe.

A tower only allows more efficient usage of the runway without a bunch of distributed coordination over a general radio frequency.