> FHE gives you NO guarantee about the code that is running on the encrypted data. I can run a leaky AI model or a SELECT * on encrypted data and still get the output. What I can do (and that's assuming there is open-sourced, auditable code) is to make sure that anyone with hypervisor access on that machine cannot dump my data out during processing.

I might be misunderstanding but i think this is misleading. Any code can be run, but the person running the code cannot see the results (or any side effects), so they cannot leak data.

Depends on what do you mean by "person running the code"

If by person you mean the admin of the machine, then yes. If by person you mean the developer of the FHE-based application, then "maybe" If by person you mean the analyst who would in the end order an AI python model to be executed through the FHE-based software on a machine. Then no, that person will in the end get back human-readable results. Be that a model, or a table from an SQL DB running in FHE

The way I have understood FHE is that any algorithm that would operate on the data would, by definition, be unable to produce any result that was intelligible to anyone except the person holding the original key. At no point during the execution of an FHE algorithm is the data decrypted. The amazing thing is exactly that the code running on the data does not understand the data it is consuming nor the data that it is producing.

Maybe someone who has actually studied homomorphic encryption can chime in.

Thats true. But it will still produce some data, and that data will be viewed by someone eventually who owns the key to decrypt it. FHE tells you nothing about what this product should be. It could as well be a full copy of the original data.

For example: I run an ML model using FHE on some data I shouldn't have access to in plaintext. The expected outcome of this workflow is a trained ML model on that data. FHE tells me nothing about the quality of this model. It could as well be an overfit model that spits out all the sensitive data.

Sorry but I still fail to see how that would be a problem, since the output of the program (e.g. the ML model parameters) would themselves not be intelligible to you. To make _any_ (non-cryptanalytical) inference on the plaintext of the homomorphically encrypted data _necessarily_ requires that the attacker at some points can access or execute some classical code on the plaintext. This would obviously violate the "fully" part of FHE.

Edit: Okay so I might now understand you refer to a scenario where the user submits their data in homomorphic form to the cloud, where an AI model is trained on it. The AI model parameters are later returned to the user's device, which then decrypts them with the user's key and executes a classical model with those parameters, and then resubmits the user's data after processing with the said ML model (unencrypted) back to the cloud. It's true the user usually has no way of auditing the code / model that runs on their device, but isn't that rather easily alleviated by opening up the APIs for communicating with the cloud part of the service?

Close. The first part is fair.

A more real-life example.

I am a pharma company and I want to execute a query on some hospital data. The hospital doesn't want to give me the data in plaintext but they are fine with me getting some aggregate insights from their data that are not PII.

Now lets assume I decide to do that using FHE. I can now compute my query on the encrypted hospital data and I never see the plaintext data.

What do we "win" in this scenario? We can do this computation wherever we want because no matter where the computation is done, the data will be encrypted, so no risk for the infra provider to see that data.

What we don't "automatically win" in this scenario? 1. Guarantees that indeed I am running an SQL query on that data and not something else along with it -> That is only possible to guarantee if the FHE software is properly audited (same with any software tbh, but easier with FHE and similar techs because of the integrity guarantees due to encryption). 2. Guarantees that the SQL query I made will not leak patient data in the end (through linking additional data, or diff attacks) (same with any other SQL query)

People who are deep into these technologies will say "yes of course" thats not an FHE problem. And that is true. But every FHE vendor I've seen blur that difference by not specifying what kind of attacks they protect against when they talk about "protecting privacy".

Heck, most of them they don't even talk about the attestation process and how their clients can make sure that they can trust the software running in encrypted form. Yes, these hold true for all software, but the point (for me) of encryption in-use is to make sure we hold software to a higher trust standard than today, not just replace a trusted party with another one.

This example is confusing because its unclear who the trusted parties are and who you are trying to protect the data from. Quite frankly this feels like you are mostly pointing out that FHE wont work if you use it incorrectly . Normal encryption won't work either if you give the bad guy your key.

> But every FHE vendor I've seen blur that difference by not specifying what kind of attacks they protect against when they talk about "protecting privacy".

Agree with you here. FHE is an impractical technology at this stage. I'm pretty sure all commercial FHE vendors are borderline scammers, and have a loose relationship with the truth.

For clarity, let's assume the hospital stores its records in plaintext. For the pharma company, the hospital encrypts the patient records with a secret key. Now they let the pharma company run their homomorphic algorithm and send the values back. Only problem is the pharma company can not read those results without having access to the key. FHE is completely redundant in this use case - the hospital could have simply run the pharma company's SQL and audited the code and outputs.

What is FHE actually good for then? Let's imagine you are a top secret agent and you get instructions to fly to Bulgaria as a part of your mission. You have other hostile agents constantly monitoring you, trying to understand your next move. But there's a problem - to buy a plane ticket to Bulgaria you need to know the name of its capital city. You can't just type it to Google, because these other agents have actually infiltrated the Google servers and can see everything you search (assume once you actually know the name of the capital, you can somehow buy the actual ticket without "them" knowing..)

Lukcily though, CloudCorp offers a public homomorphic query service for all world capitals. This service allows you to send a query for the capital of any country over an intercepted connection, and get back the result. Even if the hostile agents had infiltrated CloudCorp and were monitoring all your comms, they would not be able know which country's capital you just queried. Not even CloudCorp could do that, you are the only person who knows what you asked and what was the result.

How such service would be implemented is explained in good detail in this tutorial, completele with working code: https://github.com/homenc/HElib/tree/master/examples/BGV_cou...

P.S. The capital of Bulgaria is Sofia.

FHE only reveals information to the person who has the keys for it, not to arbitrary people in the middle. So if you had access to the keys for the input data, then you have access to the keys for the output data.

But the model itself is encrypted. You should assume a model trained on sensitive data at least partially includes the sensitive data and treat it as sensitive, as FHE intrinsically does. If you’re saying once you decrypt them model you need to keep treating it with sensitivity and not give it to untrusted compute in the plain, then yes you’re right. But that’s nothing to do with FHE because you stopped using FHE the moment you decrypted it. What stupidity you do after generations of PhD protected your data in untrusted compute using FHE is your stupidity alone and says nothing about FHE.

The other way I read what you’re saying you’re saying the holder of the model after they decrypt it may not be trusted with the model or the original data. But they hold the decryption key to both. So, why did you share the key to someone you don’t trust? That breaks the model too.

Wouldn't doing the same thing without FHE also result in the same problem?

Yes, OP is apparently under the assumption that FHE was designed to solve all problems, instead of the problems it was designed to solve.

Yeah, and many more. But I've seen multiple people argue that using FHE will magically solve all their privacy problems and its far from true. FHE (and similar technologies) solve a piece of that "puzzle" and most providers somehow gloss it over.

What i mean, is the only person who can learn anything about the data, is the entity who posesses the decryption key.

In any sane deployment of FHE the key holder is the person who owns the data, not the app developer and not the person "running" the program.

> A very powerful concept for remote processing, supply chain security, and overall reducing trust; but completely unrelated to privacy.

It is still related to privacy, but the privacy "attacker" is the execution place, allowing for outsourcing of computations and storage without running into data leaks or violations of data protection laws.

Maybe you use a different definition of privacy?

You are right. It is related to privacy, but its not the whole story. Thats what I am trying to say. Running FHE will not magically solve the "data leakage" problem of your AI models, and I believe that the people/companies who don't make that distinction are misleading.

This isn't an area I know much about so I'll stay out of the main conversation, but would like to point out that considering you wrote "completely unrelated to privacy" in your first comment, following it with "It is related to privacy, but its not the whole story. Thats what I am trying to say." makes me unsure what your point is or if you actually mean or understand it. Sorry for being blunt.

You are right to be blunt. I didn't want to define the separation of input and output privacy in a comment. In retrospect maybe I should have but I can't edit anymore. It is however common HN practice to pick out the words that suit someone and construct an very specific argument based on these words, often missing the spirit of the comment.

Yes, when I wrote the comment I had in mind "output" privacy while FHE is dealing with "input" privacy. It is related to privacy, but not in the way most people think about it.

If you go to a random person and ask them about privacy they will not think about the threat model of a cloud provider leaking their data, but they will think of the thread model of a pharma company knowing exactly what drug they bought and when. That notion of privacy is not covered by FHE(alone). And even the first notion of privacy is covered only if the FHE program has a way to attest itself so you know that what you expect to run is indeed what is running.

> FHE gives you NO guarantee about the code that is running on the encrypted data.

I'm open to correction, but it's my understanding that the strongest form of FHE allow users to submit an encrypted executable with embedded data as input, which is then processed by an untrusted server. I'd definitely call it an ultimate form of privacy. The computational cost is prohibitively expensive, and conditional branch is impossible in the standard implementation, so it's largely an academic exercise. But last time someone on HN told me currently the achievable performance on a modern computer is roughly equivalent to a 1970s mainframe, so I guess some niche applications are still possible.

Weaker forms of FHE don't have this level of privacy, and they do not claim so. Nevertheless, relevant development still represents progress on cryptography and privacy researches as a whole.

In the context of cryptographic protocols we sometimes use "privacy" to refer to the notion of "confidentiality". The latter is, I think, a cleaner word that avoids the collision with human notions of privacy.

In this case the real danger is that the availability of "privacy-preserving technologies" like FHE, MPC and Differential Privacy will actually do more to undermine human privacy than all the non-confidential tech that come before. This will mostly occur by allowing corporations to build sophisticated statistical/ML models using data that would previously never have been allowed out of its confidential silo.

Secure multiparty computation (MPC) does help here.

Suppose you have multiple organizations that want to run some computation on their joint data, without revealing their data to each other. Each organization has their own machine that runs the MPC protocol. They have full control over their machine, and can inspect that the code correctly executes the protocol. Only once all organizations agree, will the computation take place, and within the security model of the protocol, it is guaranteed that only the correct computation output is revealed to the designated parties.

Fully homomorphic encryption is a toolset, it's not a specific configuration.

Your scenario has these parties, 1) a patient whose data we're discussing, 2) the hospital they shared it with, and 3) a pharma company looking to use the data. The hospital wants to promote this use without leaking any PII.

You're right that the hospital has no idea about the queries ("the code") but they control the server and which messages it will send in response.

As you point out, the hospital wouldn't run a FHE database capable of full-text extraction specifically because that would amount to simply sending all the data to the pharma company.

Instead they'd run a specialized FHE-DB server which would, for instance, return only row counts. The pharma company would run secret queries and if the hospital had one or more patients who matched the query the pharma company would know to the contact the hospital and then once paperwork is signed they could rerun the query with a signed token from the hospital and finally the query would return the actual PII.

> FHE gives you NO guarantee about the code that is running on the encrypted data.

You mean in order to validate the data is authentic? Otherwise the code running on the data is irrelevant, as it can't access the data itself (and thus preserves the before-mentioned privacy).

The code decrypts the data at some point (for use, presentation, etc). If the code is crappy, insecure, etc. then the data will be exposed, and the data being encrypted wont help at all...

Decrypted on the local computer, not the untrusted remote computer as it were.

If the same company makes the software at the local end, there's still code there than can expose the data or the encryption key...

Big assumption there.

I agree! That's why remote attestation or simply verifiability is such an important feature of these schemes. Semantic attestation ofc means having access to the source code and that IMO makes open source the natural choice. Audits might be an option where OSS is not desired for other reasons (likely business related). Not an expert on FHE, but confidential computing provides that attestation feature and it's just a matter of the software to make use of it.

> run a leaky AI model or SELECT *

That's the point though isn't it? Only the person who wants the results can get them or even see the inputs. That restricts the data available for shitty AI and precludes any Joe Schmo from scanning the whole database.

If your threat model is instead that you don't trust the FHE endpoint, then much how you want HTTPS termination to happen in a place you control you also in this case just encrypt the stuff you care about on your own devices.

Cryptographic is all about privacy of data-in-transit and data-at-rest. What you're discussing is peoples lack of understanding of (or appreciation for) threat models, or taking a "countermeasure-first" approach to security/privacy.

To my understanding, FHE is designed for the threat model where you don't trust the machine you're calculating on to know the data you're calculating.

As for the argument that it can spew out garbage results or false data and no one would be the wiser, when the encrypted data is provided by you, it's pretty easy to validate when something isn't correct. 100,000 passes on predictable data outcomes for example.