Simple layman's explanation (from a layman): lenses spread light out a bit as the light passes through, so the red wavelengths hit the sensor in a slightly different spot than the blue wavelengths. You need more expensive lenses to reduce this effect, but you're not going to get those in consumer-grade mass-market 1/8"-across lenses.

As a result, higher pixel densities will cause light from a single point to be spread across multiple pixels, with different wavelengths going to different pixels. Software can probably reconstruct the image somewhat, but I'm sure you'll hit a limit where there is too much interference to get the image any sharper no matter how many pixels you squeeze in.