> If your application needs to call a 3rd party service like openAI, the only solution to safely not leak your API key is to have your app only communicate with a backend you own and call the openAI from there.

I've also seen vendors do things like issue client-side keys for AWS IAM users that can access their backend (in AWS) with a super locked-down role. This would be more interesting as a solution if IAM stuff was interoperable between cloud providers (CSP), since this dependency means you can't move to another CSP without bothering your customers. It also doesn't help in the OpenAI case because there isn't a way to mint limited-permission tokens.

Given a cloud service account you can call the provider’s token service, get a bearer token, then use that bearer token to call any other cloud service configured to trust the provider issuer. Most cloud providers support this today with oidc.

That may be true but I would hazard a guess that 90% of mobile apps that talk to 3rd party services have keys stored in their binaries. It may be true as an individual that you should not do that but discipline doesn’t scale. We need a convenient best practice that doesn’t put keys in the binary. Setting up a proxy server that you also need authentication with that talks to all your apis is not gonna get done unless you make it idiot proof.

Almost all Google APIs that have any kind of visualization component that runs on the client require you shipping your API key in your clients.

They have some safeguards e.g. HTTP referrer restrictions but it's not bulletproof.

Google Maps's "API key" is not really a secret. It's used only to identify your application and to generate an iframe that's only allowed to be used on your website. It's bulletproof enough not to be considered as a secret as it can't really be used to impersonate your app if leaked.

> it can't really be used to impersonate your app if leaked

It actually can, I could create an app called com.yourbusiness.someapp, install the app directly without signing it, and use yourbusiness's API key.

For the JavaScript embed APIs I could create a fake root cert, fake DNS, fake HTTP cert signed by the fake root cert, serve https://yourbusiness.com/ from my in-house server to my own laptop, and then issue queries to the Google API using yourbusiness.com as the referrer. (Or hell, just create a Chromium fork that serves up fake referrer headers.)

It's just that it's not big enough of a threat that it has become an issue.

> is to have your app only communicate with a backend you own and call the openAI from there.

I'm a bit baffled anyone puts anything secret on software people are using. This service needs to be online anyway.

Anyway, seems like a lazy programmer thing.

Running a server even to proxy requests takes a lot of work, since you now need your own auth system and have to manage scaling. If you take the plunge, a serverless architecture like Cloudflare Workers makes scaling automatic, but you still have to do some heavy lifting to either have an API key or auth system and abuse protections (otherwise they just spam your API instead of directly stealing your OpenAI api key).

You probably don't need to scale if all you're doing is auth and proxying requests. If you get to the point where you do need to scale, you can probably afford to figure it out.

Er.... Just ask gpt4 to how to do it obviously.

Or, a lazy ChatGPT auto-generated code copy-paste thing.

This is advice so broad and generic, that it's just about useless.

If you don't store any API keys in your binary, how do you handle crash-logging and analytics? How do you integrate with third-party log-in SDKs?

You _could_ vend some of those (not the crash-logging ones, etc) from your API, but then how do you authenticate to _that_, if you can't have any secrets?

You can't login-gate all of those, and many of those are not easily rotated.

There is a difference between an app ID and a secret. A secret is something that can be used to impersonate you or your app towards a 3rd party service if leaked.

3rd party log-in SDKs using OpenID connect can work entirely without client-side secrets using only your app IDs. Crash logging and analytics services API keys are also usually considered to be app IDs, not secrets.

Sure but now the advice is what? first determine which of these 10 api keys your app uses are really secret, for those that aren’t just stick em in the binary and you are done, for those that are set up a proxy server with authentication, store the keys on that, and call the apis through there. …or you could stick all 10 in the binary and be done in 5 minutes. Can you start to see why it is so common that people just stick api keys in the binary? Can’t we have some reasonable dev experience for storing secrets that isn’t 10x the effort?

Determining which API key is supposed to be a secret isn't usually an issue, cause typically site providing you with the secret key clearly states the fact that it's supposed to be a secret. For example this is what OpenAI says about their keys directly on the page where you generate the key: "Your secret API keys are listed below. Do not share your API key with others, or expose it in the browser or other client-side code."

Well, if you have an app that's a thin structure on top of GPT-4, adding your own pipe between your client and OpenAI could add a lot to the cost and complexity of the app. Which is to say it's not surprising that people don't do it.

The genius and the craziness of GPT-4 is you can make whole app with a prompt like "now you're a clown painting custom faces on kids based on their favorite animals" and some glue-code. Needing to add a 3 layer network infrastructure with isn't appealing I'd imagine.

Could be as simple and cost-efficient as a Cloudflare worker that adds your key and passes the query along

Or a 5€/mo Hetzner server running Nginx, though Cloudflares free offering is very generous

You'd need some kind of authentication as well.

Errrr, I assume your app can pass said authentication? If so, then it's meaningless; that's again a secret not under your control.

You have to use Cloudflare Workers in Unbounded mode to do this (especially if you are streaming using EventSource or WebSockets). Bundled mode won't cut it as it closes connection after 50ms of Javascript execution.

That’s 50ms of CPU execution, which does not include waiting for IO.

Not only that, but it makes not much difference if they call the GPT Api directly or through a proxy. Only thing that eould really help is having users register and authenticate through the proxy.

The proxy approach at least lets you rate limit by IP, limit the length of the strings (and thus the token cost), etc. The API key may also grant access to other models, administrative IPs, etc. you don't want people using.

Far, far, far better than nothing.

>the only solution to safely not leak your API key is to have your app only communicate with a backend you own and call the openAI from there.

Don't you still have the same problem? Your backend would also require a key for the client to communicate, which would still be embedded in the client binary.

No.

You go through traditional auth channels (username / password) or you generate a key per app user to talk with your backend.

Or, you keep the backend you control open and implement controls to combat abuse, such as rate limiting and ip blacklisting.

Whatever chosen, the objective is to protect your API key and make sure the application is being used according to its purpose.

An API key directly to openAI allows for any use under the sun — botnets, new prompts, etc., and can drain money, put your account in bad standing, or even get you in (potentially serious) legal trouble. Using your own backend, you can do things like hit the openAI moderation endpoint, inject the correct prompts into whatever you’re sending to openAI, etc.

The main thing is you have a limited API specific to your app offering which significantly lowers the damage possible. You absolutely always want to give users the least privilege necessary for whatever use cases being provided for — this protects both you and your users.

Yes, but you could negotiate a token with each user on startup and if someone starts abusing your service you can block that access (perhaps automatically) or know who it is by authenticating the user.

No, your proxy site can have a login flow or some other unique id process that is per client like any web app.

You have a different problem. You can handle the authentication, and blacklisting, on your side. That's harder to do when the clients communicate with OpenAI or another third party directly, while incurring charges that you have to pay for.

I’ve had a theory that LLMs are going to increase the prevalence of “shadow it”. Aka slightly technical business users using gpt to create their own apps to handle business processes. These apps will then lead to large increases in security incidents where company data gets leaked or hacked or whatever because the apps will just be random unreviewed code.

This is not such a grim future since it’ll mean that SWEs still have a job. My fear is that AIs get powerful enough for MBAs to create robust software without the assistance of SWEs. A world run solely by MBAs is a scary future.