Trusting the server to perform the operation I asked it to is an entirely different domain than trusting the server with the knowing the input and output data of the operation.
Well put.
Gnull just pointed out that FHE by itself does not provide authentication so you still need some additional means to achieve that.
With the use cases of FHE though that just is not a major downside. Instead if you rely on some consensus protocol you can get a reasonable guarantee of correctness. If a random N out of M available operators are selected and all produce the same result or hash of a result, you can be reasonably sure that the expected operation was performed.
Additionally, as the "master" who can actually read the FHE data, for most non-trivial tasks I suspect there is often a proof or heuristic you can use to reasonably judge whether the result is correct or at the very least reasonable which should stave off a significant portion of the avenues for attack.
While trusted computing could be used for such verification, I'm not aware of any commercial real world implementation. I'd be genuinely happy to be made aware of one, though, so please feel free to point me to one.
That's not what FHE intends to solve. You have the exact same problem with any kind of remote processing, encrypted or not. I don't see why you think it's important to point out the obvious.