Or buy the integrated phone that includes all of those features and doesn't make you swap them out and risk losing or damaging the blocks.
Modularity often ends up being a _cost_ to the unsophisticated majority of end users. People don't like it in their software (preferring a "complete solution"), nor in their hardware (e.g. cheap integrated hifi versus expensive minority separates).
But these kind of things are thought up by a minority of technically minded people who;
- like to change/tinker with their hardware - like to change/tinker with their software - like products which do one thing and do it well
Edit; not saying this is a good idea for a phone, but to some extend it would be for a laptop (not ultrabook). Some rugged ones are quite like that.