Multi-party computation is also a cool piece of technology. I think that I was always drawn to HE because it involves a completely trustless relationship with the server; in MPC, there's more a of a "m/n parties don't collude" assumption, that can be tricky to implement in practice. (Also side note: you can in theory perform arbitrary computation on encrypted data using FHE; the performance impact is really significant, like 100-10000x, but you can perform unlimited computation).
Surprisingly, these two concepts (MPC and HE) work quite well when combined! There are some computations each is suited to computing efficiently (linear for HE, non-linear for MPC), so when combined for ML, you get really good results. See this landmark paper and the follow on work: https://eprint.iacr.org/2018/073.pdf.
FHE is painfully slow. The only real domains it can work realistically are for very simple computations. You are putting the cart miles ahead of the donkey here.
I think the killer application for homomorphic encryption (HE) will be on the regulated and sensitive data as you have mentioned. In healthcare, high accuracy machine learning requires massive datasets and it is difficult to get your hands on the data mainly due to privacy concerns. If HE can make data available to be analyzed without compromising the patients' anonymity it will be a game changer and perhaps can relax on the accessibility of the sensitive clinical data in the near future.
Just wondering, can we not achieve similar privacy using secret sharing (SS) cryptography for anonymity? With (SS) unlike HE, you can perform arbitrary calculations in the encrypted domain [2].
[1]How to Share a Secret:
https://news.ycombinator.com/item?id=31817716
[2]Secure multi-party computation:
https://en.m.wikipedia.org/wiki/Secure_multi-party_computati...