When dealing with unpredictably high traffic spikes, on days when you cannot afford any kind of downtime - when that downtime comes at great cost, it's not overengineering (and it's hard to determine if it's overengineered until you know the budget and time spent on the actual project - this may have been a relatively simple modification all things considered)

Static content is easy to crank up to web-scale. You can use DNS, any kind of load balancer, all kinds of web services, CDNs, whatever.

Dynamic content is hard to scale (compared to static). You have several layers of added complexity. Yes, you can build wonderful, self-scaleable systems - but at some point they hit a limit, there are many more resources that can be tied up, and troubleshooting and scaling that out beyond anything you've previously imagined on short notice can take time you can't afford.

So - simply de-coupling the dynamic content generation from the static web serving is a great way to make a clean break - you now have a known & tuneable load on your dynamic application (because your'e running it at known intervals, rather than being event driven by user requests) and you have a front-end static infrastructure that you can scale like mad, and even if your back-end collapses, edit by hand.

Surely there are other ways to approach the problem.... but it also depends on the engineers involved, the time taken, and their confidence in their ability to deal with it.

I'm also fairly sure they aren't the first company out there to take this approach to burst scalability issues... but it's curious to note how the NYT actually operates.

TL;DR: Look at the old configuration, and the new configuration. Decide which one will best serve your business in terms of your ability to troubleshoot it when it gets hit by a level of traffic higher than you can plan for, because you have NO idea how high it will go.

In my book, overengineering is defined as doing more work than is necessary to ensure that the risks are within acceptable bounds.

As the author points out, having the Times election site go down on election eve would be a BIG PROBLEM -- massive losses in both reputation and advertising revenue. For the system architect, a failure could possibly mean losing his job.

What the author has laid out is a system that is robust to multiple, simultaneous failures (with possible exception of the loss of AWS, although that's not entirely clear). That just seems like good planning.

The file-system isn't just another data-store: it's I/O and therefore special. On systems with 'sendfile' data can be sent straight from disk to network port without CPU time (and any copies being made). This is used extensively by high-performance webservers and web-accelerators.

sendfile() still requires CPU time - it just passes the job of sending the data over the socket to the kernel to finish up. The previous method would have been a select() loop or similar that ensured the data was sent out to the kernel, involving a bunch of system calls and context swtiches. sendfile lets you hand the job off 100% to the kernel and have your thread move on.

I was under the (possibly mistaken) impression that Direct Memory Access (DMA) hardware could move data straight from the Disk Cache to the Network card without touching the CPU Caches at all.

True - DMA can do that - but then you'd have to have something managing the network stack - the kernel manages tcp/ip.

From the man page:

"sendfile() copies data between one file descriptor and another. Because this copying is done within the kernel, sendfile() is more efficient than the combination of read(2) and write(2), which would require transferring data to and from user space."