Diffraction and airy disks have an impact as mentioned by a sibling comment, but the simplest explanation I've used is that ultimately you're capturing photons, and if you have a larger sensor you can capture more photons. More photons are better. Of course you need a larger lens with a longer focal length to match the same field of view, but that's the basic idea.