News for hackers. Stuff that matters.
Re:Rearchitecting GitHub Pages by adriancooney on Wednesday May 27, 2015 @11:17AM (#9611958)
I'm amazed that Github Pages ran on just two servers (well, aside from the MySQL clusters). That is absolutely incredible given the sheer amount of projects and people who rely on it for their sites (me included!). I love the philosophy behind Pages of abstaining from over-engineering and sticking to the simple, proven solutions. It's a great service and I'm a massive fan.
Re:Rearchitecting GitHub Pages by joshmoz on Wednesday May 27, 2015 @11:32AM (#9612073)
It's cool that they were able to do this with two machines, and I don't want to detract from that, but it's probably worth pointing out that a "machine" is not a very useful unit of capacity. These two machines could be dual core i5s or they could be 20 core xeon boxes with hugely varying amounts of memory and storage. Too bad they don't clarify, I'm curious.
Re:Rearchitecting GitHub Pages by ceequof on Wednesday May 27, 2015 @11:37AM (#9612102)

  The fileserver tier consists of pairs of Dell R720s 
  running in active/standby configuration. Each pair is 
  largely similar to the single pair of machines that the 
  old Pages infrastructure ran on.
http://www.dell.com/us/business/p/poweredge-r720/pd

Two Xeon E5-2600s per machine.

Re:Rearchitecting GitHub Pages by ChiperSoft on Wednesday May 27, 2015 @04:20PM (#9614050)
The ES-2600 series comes in 4 to 12 cores, so that's still a pretty wide range of compute power.
Re:Rearchitecting GitHub Pages by vinhboy on Wednesday May 27, 2015 @02:40PM (#9613457)
I am going to remember this article and bring it up every time some ops crazy people try to trick me into over-engineering shit.
Re:Rearchitecting GitHub Pages by clinta on Wednesday May 27, 2015 @09:15PM (#9615512)
As an ops guy, I'm going to remember this article and use it as a talking point for the benefits of static pages when devs want to put everything into their dynamic framework of choice.
Re:Rearchitecting GitHub Pages by tracker1 on Thursday May 28, 2015 @12:23AM (#9616055)
It's a bit of a trade off... static generation of dynamic content isn't so much different from front-side caching, if the bulk of your requests can cache for 30+ minutes, you can setup some pretty decent caching rules on a couple of servers fronting a dynamic backend that doesn't need to be too big to support many requests.

For the most part, the load tends to come down to less than optimized database storage and queries on the backend, often duplicated for every page load for content that could easily be offloaded to the client browser, constructed client-side, and requested on demand.

It's a matter of striking a balance... In this case the content changes very infrequently, and having a publish step to static storage makes sense. It really depends on one's needs.

If you have a million pages you'd need to regenerate daily for a single site, and thousands every hour, then you might think differently.

Re:Rearchitecting GitHub Pages by vog on Wednesday May 27, 2015 @11:23AM (#9612007)
In the same vein, I'm surprised how many static websites go down just because they appear on a news site.
Re:Rearchitecting GitHub Pages by munificent on Wednesday May 27, 2015 @11:45AM (#9612164)
Most of those I see either:

1. Aren't static sites but are things like WordPress blogs running server-side PHP essentially unnecessarily.

2. Don't take down the server, but just hit the tiny bandwidth quota of some cheap shared hosting provider, so the host blocks it.

Re:Rearchitecting GitHub Pages by josephmx on Wednesday May 27, 2015 @12:46PM (#9612618)
A significant amount also seem to saturate their line, especially on photo-heavy static sites
Re:Rearchitecting GitHub Pages by mortehu on Wednesday May 27, 2015 @09:58PM (#9615664)
3. Use the distro's default Apache connection limit of 20, despite the server being able to handle thousands of concurrent connections. The limited connections are then quickly used by users with slow connections.
Re:Rearchitecting GitHub Pages by rajeevk on Thursday May 28, 2015 @01:58AM (#9616281)
They also use CDN. As the pages are cacheable, the CDN will decrease the load on their server significantly.