The modular approach doesn't need to be enforced by strict boundaries. One network that starts with various components but can be remodeled and even merged together as it goes could adapt.

What we're discussing here is merely a starting state of something that'd have to rebuild itself.

Or maybe evolving modalities within one network is easier than putting it together.