Cryptographic is all about privacy of data-in-transit and data-at-rest. What you're discussing is peoples lack of understanding of (or appreciation for) threat models, or taking a "countermeasure-first" approach to security/privacy.

To my understanding, FHE is designed for the threat model where you don't trust the machine you're calculating on to know the data you're calculating.

As for the argument that it can spew out garbage results or false data and no one would be the wiser, when the encrypted data is provided by you, it's pretty easy to validate when something isn't correct. 100,000 passes on predictable data outcomes for example.