News for hackers. Stuff that matters.
Re:Migrating Dropbox from Nginx to Envoy by eric4smith on Friday July 31, 2020 @01:10AM (#24005869)
It's interesting almost no web server provides an easy way to deal with multi-tenant multi-domain architectures in a good way that includes automatic SSL.

Caddy is the closest, but still not near enough.

There is this small segment of the market that we operate in that requires thousands of TLS connected domains to be hosted behind a dynamic backend. It's services like Tumblr, Wordpress.com, or any other hosting service where you can get a "custom domain" to point to your own blog or site.

NGINX - No.

Apache - Nope.

Caddy - Can do (but need lots of workarounds)

Envoy - Nope.

Everyone focuses on a few hand-coded domains and no automatic TLS. Maybe this part of the market is too small anyway. Sigh.

Re:Migrating Dropbox from Nginx to Envoy by mholt on Friday July 31, 2020 @01:17AM (#24005903)
Several companies use Caddy for exactly this purpose. Fathom Analytics for example uses it for their custom domains feature. Caddy can even reactively provision certs during TLS handshakes. It's a native feature. Why does it require lots of workarounds?
Re:Migrating Dropbox from Nginx to Envoy by bschwindHN on Friday July 31, 2020 @02:25AM (#24006212)
Yeah I'm not sure what they're getting at, I've used Caddy as well for similar "custom domain" features, it was super easy. Thanks for creating it!
Re:Migrating Dropbox from Nginx to Envoy by eric4smith on Friday July 31, 2020 @06:48AM (#24007401)
Yes. Caddy is what we use, since not much else can do it as easily as Caddy can. And it's our go-to tool for several projects that require custom domains. And we really, really, appreciate it!

I'm just saying that it's not something that is documented well or purpose built for that scenario.

Re:Migrating Dropbox from Nginx to Envoy by sladey on Friday July 31, 2020 @11:39AM (#24009740)
Is there any mature integration to achieve this with Kubernetes?
Re:Migrating Dropbox from Nginx to Envoy by mholt on Friday July 31, 2020 @06:35PM (#24015029)
Re:Migrating Dropbox from Nginx to Envoy by elithrar on Friday July 31, 2020 @09:04AM (#24008193)
You can definitely “lazy load” TLS certs into Envoy.

The SDS (Secrets Discovery Service) supports this, and is touched on in TFA: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overv...

You provide a gRPC service that can return the keypair needed for any host, with host config also being dynamic.

https://www.envoyproxy.io/docs/envoy/latest/configuration/se...

Re:Migrating Dropbox from Nginx to Envoy by simplyinfinity on Friday July 31, 2020 @02:19AM (#24006192)
We are using OpenResty with lua-auto-ssl for exactly this purpose, and it works like a charm.
Re:Migrating Dropbox from Nginx to Envoy by xenonite on Friday July 31, 2020 @08:32AM (#24008002)
Lighttpd seems to have solutions for that. Did you have a look at it?

https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_SSL...

> A traditional problem with SSL in combination with name based virtual hosting has been that the SSL connection setup happens before the HTTP request. So at the moment lighttpd needs to send its certificate to the client, it does not know yet which domain the client will be requesting. This means it can only supply the default certificate (and use the corresponding key for encryption) and effectively, SSL can only be enabled for that default domain. There are a number of solutions to this problem, with varying levels of support by clients.

Then, the best approach seems to be the following:

> Server Name Indication (SNI) is a TLS extension to the TLS handshake that allows the client to send the name of the host it wants to contact. The server can then use this information to select the correct certificate.

Re:Migrating Dropbox from Nginx to Envoy by e12e on Friday July 31, 2020 @01:18AM (#24005915)
Traefik makes it fairly easy (inasmuch as it makes anything easy). But it's just a proxy, not a web server.
Re:Migrating Dropbox from Nginx to Envoy by eric4smith on Friday July 31, 2020 @06:50AM (#24007410)
Traefik can work yes - documentation is really terrible though for anything approaching that use case. We had to really futz around with it and eventually went back to Caddy. Our use case is several thousand client domains that just proxy to some backends.
Re:Migrating Dropbox from Nginx to Envoy by e12e on Friday July 31, 2020 @03:01PM (#24012502)
Did you move to caddy 2?

Our initial use-case was ingress for docker swarm - after a fooray into k8s with the "traditional" nginx ingress with its rather hackish let's encrypt contraption.

I briefly looked at caddy 2 - but wasn't able to find any out-of-the box tricks for listening to docker messages and dynamically configure sites in a sane way.

Do you use custom code and configure caddy via the api?

Re:Migrating Dropbox from Nginx to Envoy by mholt on Friday July 31, 2020 @06:34PM (#24015010)
> I briefly looked at caddy 2 - but wasn't able to find any out-of-the box tricks for listening to docker messages and dynamically configure sites in a sane way.

Like this? (Am not a Docker user, but I know this is an insanely popular solution) https://github.com/lucaslorentz/caddy-docker-proxy

There's also a WIP ingress controller: https://github.com/caddyserver/ingress/

Re:Migrating Dropbox from Nginx to Envoy by toast0 on Friday July 31, 2020 @02:11AM (#24006163)
I would think the way to do this would be to run a separate TLS daemon that handles the certificates (including acme challenges, presumably) and then pass the socket to your http server, either by proxying it (preferably to a unix socket), or like actually pass the FD with the session keys.

I don't think hitch (formerly stud) supports acme challenges, but that's where I'd start.

Re:Migrating Dropbox from Nginx to Envoy by andrenth on Sunday August 02, 2020 @01:40PM (#24030099)
Apache can do automatic TLS with mod_md.