Depends on what do you mean by "person running the code"

If by person you mean the admin of the machine, then yes. If by person you mean the developer of the FHE-based application, then "maybe" If by person you mean the analyst who would in the end order an AI python model to be executed through the FHE-based software on a machine. Then no, that person will in the end get back human-readable results. Be that a model, or a table from an SQL DB running in FHE

The way I have understood FHE is that any algorithm that would operate on the data would, by definition, be unable to produce any result that was intelligible to anyone except the person holding the original key. At no point during the execution of an FHE algorithm is the data decrypted. The amazing thing is exactly that the code running on the data does not understand the data it is consuming nor the data that it is producing.

Maybe someone who has actually studied homomorphic encryption can chime in.

Thats true. But it will still produce some data, and that data will be viewed by someone eventually who owns the key to decrypt it. FHE tells you nothing about what this product should be. It could as well be a full copy of the original data.

For example: I run an ML model using FHE on some data I shouldn't have access to in plaintext. The expected outcome of this workflow is a trained ML model on that data. FHE tells me nothing about the quality of this model. It could as well be an overfit model that spits out all the sensitive data.

Sorry but I still fail to see how that would be a problem, since the output of the program (e.g. the ML model parameters) would themselves not be intelligible to you. To make _any_ (non-cryptanalytical) inference on the plaintext of the homomorphically encrypted data _necessarily_ requires that the attacker at some points can access or execute some classical code on the plaintext. This would obviously violate the "fully" part of FHE.

Edit: Okay so I might now understand you refer to a scenario where the user submits their data in homomorphic form to the cloud, where an AI model is trained on it. The AI model parameters are later returned to the user's device, which then decrypts them with the user's key and executes a classical model with those parameters, and then resubmits the user's data after processing with the said ML model (unencrypted) back to the cloud. It's true the user usually has no way of auditing the code / model that runs on their device, but isn't that rather easily alleviated by opening up the APIs for communicating with the cloud part of the service?

Close. The first part is fair.

A more real-life example.

I am a pharma company and I want to execute a query on some hospital data. The hospital doesn't want to give me the data in plaintext but they are fine with me getting some aggregate insights from their data that are not PII.

Now lets assume I decide to do that using FHE. I can now compute my query on the encrypted hospital data and I never see the plaintext data.

What do we "win" in this scenario? We can do this computation wherever we want because no matter where the computation is done, the data will be encrypted, so no risk for the infra provider to see that data.

What we don't "automatically win" in this scenario? 1. Guarantees that indeed I am running an SQL query on that data and not something else along with it -> That is only possible to guarantee if the FHE software is properly audited (same with any software tbh, but easier with FHE and similar techs because of the integrity guarantees due to encryption). 2. Guarantees that the SQL query I made will not leak patient data in the end (through linking additional data, or diff attacks) (same with any other SQL query)

People who are deep into these technologies will say "yes of course" thats not an FHE problem. And that is true. But every FHE vendor I've seen blur that difference by not specifying what kind of attacks they protect against when they talk about "protecting privacy".

Heck, most of them they don't even talk about the attestation process and how their clients can make sure that they can trust the software running in encrypted form. Yes, these hold true for all software, but the point (for me) of encryption in-use is to make sure we hold software to a higher trust standard than today, not just replace a trusted party with another one.

This example is confusing because its unclear who the trusted parties are and who you are trying to protect the data from. Quite frankly this feels like you are mostly pointing out that FHE wont work if you use it incorrectly . Normal encryption won't work either if you give the bad guy your key.

> But every FHE vendor I've seen blur that difference by not specifying what kind of attacks they protect against when they talk about "protecting privacy".

Agree with you here. FHE is an impractical technology at this stage. I'm pretty sure all commercial FHE vendors are borderline scammers, and have a loose relationship with the truth.

For clarity, let's assume the hospital stores its records in plaintext. For the pharma company, the hospital encrypts the patient records with a secret key. Now they let the pharma company run their homomorphic algorithm and send the values back. Only problem is the pharma company can not read those results without having access to the key. FHE is completely redundant in this use case - the hospital could have simply run the pharma company's SQL and audited the code and outputs.

What is FHE actually good for then? Let's imagine you are a top secret agent and you get instructions to fly to Bulgaria as a part of your mission. You have other hostile agents constantly monitoring you, trying to understand your next move. But there's a problem - to buy a plane ticket to Bulgaria you need to know the name of its capital city. You can't just type it to Google, because these other agents have actually infiltrated the Google servers and can see everything you search (assume once you actually know the name of the capital, you can somehow buy the actual ticket without "them" knowing..)

Lukcily though, CloudCorp offers a public homomorphic query service for all world capitals. This service allows you to send a query for the capital of any country over an intercepted connection, and get back the result. Even if the hostile agents had infiltrated CloudCorp and were monitoring all your comms, they would not be able know which country's capital you just queried. Not even CloudCorp could do that, you are the only person who knows what you asked and what was the result.

How such service would be implemented is explained in good detail in this tutorial, completele with working code: https://github.com/homenc/HElib/tree/master/examples/BGV_cou...

P.S. The capital of Bulgaria is Sofia.

FHE only reveals information to the person who has the keys for it, not to arbitrary people in the middle. So if you had access to the keys for the input data, then you have access to the keys for the output data.

But the model itself is encrypted. You should assume a model trained on sensitive data at least partially includes the sensitive data and treat it as sensitive, as FHE intrinsically does. If you’re saying once you decrypt them model you need to keep treating it with sensitivity and not give it to untrusted compute in the plain, then yes you’re right. But that’s nothing to do with FHE because you stopped using FHE the moment you decrypted it. What stupidity you do after generations of PhD protected your data in untrusted compute using FHE is your stupidity alone and says nothing about FHE.

The other way I read what you’re saying you’re saying the holder of the model after they decrypt it may not be trusted with the model or the original data. But they hold the decryption key to both. So, why did you share the key to someone you don’t trust? That breaks the model too.

Wouldn't doing the same thing without FHE also result in the same problem?

Yes, OP is apparently under the assumption that FHE was designed to solve all problems, instead of the problems it was designed to solve.

Yeah, and many more. But I've seen multiple people argue that using FHE will magically solve all their privacy problems and its far from true. FHE (and similar technologies) solve a piece of that "puzzle" and most providers somehow gloss it over.

What i mean, is the only person who can learn anything about the data, is the entity who posesses the decryption key.

In any sane deployment of FHE the key holder is the person who owns the data, not the app developer and not the person "running" the program.