Connectors inherently need to reach the outside of the device, and they're all different shapes and sizes. At a minimum you need to reserve as much external space, per module, as you expect the largest module will need to expose.

For devices that don't have external connectors, modular devices are already common, like M.2.