News for hackers. Stuff that matters.
Re:About the Tailscale.com outage on March 7, 2024 by agwa on Saturday March 30, 2024 @01:19PM (#39876690)
Why does the proxy need to terminate TLS? If it were just a TCP proxy, then at least the monitoring wouldn't have been fooled into thinking the certificate wasn't about to expire.

Heck, a TCP proxy might even allow automatic renewal to work if the domain validation is being done using a TLS-ALPN challenge.

Re:About the Tailscale.com outage on March 7, 2024 by bradfitz on Saturday March 30, 2024 @04:47PM (#39878493)
It doesn't. That was one of our mistakes and action items to fix.

The original proxy was stood up quickly when it was first discovered IPv6 was broken and the people standing up the proxy didn't know at the time how ACME worked.

We'll be changing it to just a TCP proxy.

Re:About the Tailscale.com outage on March 7, 2024 by ikiris on Saturday March 30, 2024 @05:37PM (#39878933)
> and the people standing up the proxy didn't know at the time how ACME worked

yikes

Re:About the Tailscale.com outage on March 7, 2024 by bradfitz on Saturday March 30, 2024 @05:43PM (#39878971)
To be fair, it didn't help the provider's docs were inconsistent about whether dns-01 or http-01 was to be used.
Re:About the Tailscale.com outage on March 7, 2024 by PokestarFan on Saturday March 30, 2024 @11:38PM (#39881182)
I've done enough fighting with Certbot to learn that the path of least resistance is to use dns-01 with the cloudflare plugin, and just make a very limited access key and store it on the servers that I am using to renew the cert.
Re:About the Tailscale.com outage on March 7, 2024 by amluto on Saturday March 30, 2024 @04:17PM (#39878220)
A non-TLS-terminating proxy is a great thing to host on a service like Hetzner. If you set up CAA correctly, then you are trusting the provider for latency and availability only, and you might as well avoid hilariously expensive services like CloudFront or an EC2-based proxy.

Hmm, it looks like Tailscale is using NetActuate for pkgs.tailscale.com. I bet NetActuate could help serve up a non-terminating proxy with plenty of PoPs at a reasonable price. Their website doesn’t give pricing, but it sounds like the kind of company that doesn’t mark up egress 50x.

Re:About the Tailscale.com outage on March 7, 2024 by SahAssar on Saturday March 30, 2024 @07:30PM (#39879675)
> A non-TLS-terminating proxy is a great thing to host on a service like Hetzner. If you set up CAA correctly, then you are trusting the provider for latency and availability only, and you might as well avoid hilariously expensive services like CloudFront or an EC2-based proxy.

Are you really getting any latency or availability improvements in that case? What does a non-TLS-terminating proxy give you?

Re:About the Tailscale.com outage on March 7, 2024 by amluto on Saturday March 30, 2024 @08:32PM (#39880140)
Functionality. The reason for having a proxy at all seems to be that Vercel doesn’t support IPv4.
Re:About the Tailscale.com outage on March 7, 2024 by RulerOf on Saturday March 30, 2024 @03:54PM (#39878046)
They may have AWS CloudFront CDN in front of it for IPv6. If you're doing that, you're terminating TLS at CloudFront. I don't believe that's optional.
Re:About the Tailscale.com outage on March 7, 2024 by bastawhiz on Saturday March 30, 2024 @01:55PM (#39877025)
Not that it's an amazing reason, but H3 doesn't run over TCP, and running a UDP proxy doesn't sound like a great time.
Re:About the Tailscale.com outage on March 7, 2024 by ignoramous on Saturday March 30, 2024 @03:24PM (#39877802)
> UDP proxy doesn't sound like a great time

QUIC, in particular, is harder to proxy (if you're load balancing, say: https://quicwg.org/ops-drafts/draft-ietf-quic-manageability....).

If it is point-to-point and you control both those points (forward A to B with ports open as approp), proxying any protocol should be straightforward, no?

Re:About the Tailscale.com outage on March 7, 2024 by cayde on Saturday March 30, 2024 @05:18PM (#39878760)
I donet believe they control the Vercel endpoint "B"
Re:About the Tailscale.com outage on March 7, 2024 by fanf2 on Saturday March 30, 2024 @01:32PM (#39876811)
I think a TCP proxy would also work with http challenges.
Re:About the Tailscale.com outage on March 7, 2024 by agwa on Saturday March 30, 2024 @02:37PM (#39877437)
It would, but so would an HTTP proxy. It makes make think the hosting provider doesn't use HTTP challenges.
Re:About the Tailscale.com outage on March 7, 2024 by fanf2 on Saturday March 30, 2024 @04:03PM (#39878118)
An http proxy would need to be configured cleverly enough to serve its own acme challenges directly, and proxy any requests for the backend’s acme challenges. Which is I think the trick that was missed by the tailscale setup.
Re:About the Tailscale.com outage on March 7, 2024 by agwa on Saturday March 30, 2024 @07:59PM (#39879886)
Good point!
Re:About the Tailscale.com outage on March 7, 2024 by p1mrx on Saturday March 30, 2024 @06:19PM (#39879232)
A TCP proxy discards the user's IP address, unless you use something like the PROXY protocol[1], which then needs to be supported by the target HTTPS server. You would also need a way to prevent unauthorized users from injecting their own PROXY header.

This isn't a problem if you don't need the user's IP address at all, but it's often useful for logging and abuse detection.

[1] https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt