News for hackers. Stuff that matters.

janoszen

Hey folks, I'd really like some feedback on this before I push the final version to Maven Central.
Re:Building Your Own CDN for Fun and Profit on Thursday February 15, 2018 @08:13AM (#16383699)
Yes, but it's kind of a whack-a-mole since their reuse times are not public AFAIK, so it would constantly need tuning as they develop the service.
Re:Building Your Own CDN for Fun and Profit on Thursday February 15, 2018 @01:36AM (#16382314)
A POP or edge location is a server (or multiple) that the user traffic is being routed to, hopefully close to the user. A CDN consists of multiple POPs, one in each region, with intelligent traffic routing added (as described in the article).
Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @05:55PM (#16380313)
If you are fine with having slashes at the end of your URLs and you do not want to do anything too complicated like content negotiation for image types, S3 and CloudFront is fine. The moment you turn on Lambda@Edge, to do the magic, things get slow after a period of no traffic.

I plan on expanding on the featureset, so no S3 for me. :)

Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @03:52PM (#16379254)
It depends on the scale. Running a personal blog with sub-1MiB/s traffic is not a problem. I've seen some larger projects though where detailed data analysis had to be employed to debug bad connections... that's not a one-man-job and it was a serious headache to work around some of the less... neutral providers.
Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @01:08PM (#16377808)
Of course it's dockerized, it has to be cool, right? :)

Ansible is running docker-compose up -d on deployment an Traefik is doing the magic. I want to extend it to host multiple sites in the future. (Btw. Ansible ran from a central location is painfully slow because of the large latency to the edge nodes.)

The content itself is deployed using rsync, Ansible was just too painfully slow for that.

Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @01:04PM (#16377785)
(I'm the author.) This whole setup is built for a comparatively low traffic blog, so DNS caching won't help much. (On normal days I get ~100 visitors.) This is compounded by the TTL which is 60s to account for node failures.

The optimization level is in the sub 1 second range, so not having to pay one large RTT penalty for a DNS lookup is quite important. I've measured 300+ms RTT to Australia on the previous box I was using, that impacted the load times quite severely.

Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @12:30PM (#16377497)
I think SNI is fine, all modern browsers seem to support it: https://caniuse.com/#search=sni
Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @12:26PM (#16377449)
Yes, but you would need a crawler that does so in every region, or at least know the IPs of the edge nodes on that CDN. You would probably also hit some rate limit / DDoS protection with the CDN itself.
Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @12:24PM (#16377431)
Thank you for pointing that out, I've updated my bio to reflect that. Hopefully this way it's a little less ambiguous. :)
Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @12:21PM (#16377401)
Thank you to both of you, I've edited the article to clarify that point.
Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @12:14PM (#16377350)
Interesting, although I specifically wanted to build a push CDN (where I can push the content) rather than a pull CDN (that works with an origin) to avoid the added latency with cache misses.
Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @11:59AM (#16377223)
"When it comes to picking a solution, I often choose the less traveled road"

I forgot to add that this applies only to R&D and hobby projects, for production setups I'm a bit more careful. :)

(I'm the author.)

Re:Building Your Own CDN for Fun and Profit on Wednesday February 14, 2018 @10:34AM (#16376475)
Traefik can simply request certificates using the DNS verification method, as opposed to the certbot HTTP verification. (HTTP would not work with a distributed setup like this.) Alternatively, Traefik can also synchronize certificate requests using one of the many key-value stores supported (untested as of yet).

The drawback of the DNS method without synchronization between the nodes is that you run into the LetsEncrypt rate limit quite easily. My expansion to ap-southeast-1 and sa-east-1 is waiting for the LE cooldown.

Disclaimer: I'm the author of the article.

Older »