It makes normal photographs look weird.

Full-spectrum mods are something you want in an astrophotography camera, but they're not something you want for regular photography.

So to use the camera for normal things like portraits, wildlife, landscapes, product shots, or basically anything other than taking pictures of the sky you need to get a set of (expensive) filters to go on the lens to block the excess light and restore normal color rendering.

If you're using the camera for regular photography, you need to attach an IR/UV blocking filter to all of your lenses, or else you'll be taking pictures of IR/UV in addition to visible light, and color reproduction may be distorted.

Sensitivity to colours outside the visible spectrum can cause colours to appear incorrect relative to what the human perceives. Imagine two objects whose visible spectrums are the same, but whose IR or UV spectra are different. They’d look the same to a human, but different to such a modified camera.

Indeed. Cameras are built to replicate human eyes. For instance, we’re more sensitive to green. Thus the Bayer filter.