> Why even have database queries at all- aside from perhaps comments?

Comments are non-negotiable features for many bloggers. 3rd party javascript is not the same in the eyes of many (including my little network).

Unless things have changed dramatically recently, Hacker News uses the file system to store comments (https://news.ycombinator.com/item?id=5229522) and it seems to work well. I can imagine a few cases where having your comments in a DBMS would be a requirement, but none of them seem applicable to situations where WordPress where one might want to use WordPress anyway.

Of course, this is guess from a priori reasoning. I would be genuinely interested if somebody tried file-based comments for a WordPress (or a similar platform) based site and it fell over or failed in some way.

Reasonable point, insofar as it's unlike that comments will ever be joined with unrelated articles.

The main reason you'd get failures in a flat file system would be the normal reasons you get a relational database in the first place. Either you get update anomalies, or you want to compact the data into single atoms, or you want ad hoc querying rather than a specific pattern of access set in advance.

It doesn't help that until 5.6 MySQL's query planner joined on disk if you tried to join any two tables in which one or both has a TEXT field, regardless of engine and regardless of the actual fields selected.

As you can imagine, this slows things down a bit, considering that's the core join performed for single post generation. It also makes every kind of "recent comments" plugin a performance nightmare (quite aside from their cache-busting power).

Then all you should need at most is a single query to a single table- which could be loaded asynchronously via Ajax for swift loading time and less load on the server (no slow view renders) and that response could be cached with Nginx/Varnish, and further cached on the backend with Redis. I've seen WP sites that I had to debug/fix with 200+ queries per page. That should never happen.

> Then all you should need at most is a single query to a single table

Wordpress divides posts + comments into 4 tables.

> which could be loaded asynchronously via Ajax for swift loading time and less load on the server (no slow view renders)

Wordpress can't assume any particular PHP + webserver configuration. Sometimes they're configured to send while rendering, sometimes the whole process blocks until it runs to completion. Ajax doesn't magically change that.

I also dislike the idea that I should need to download what is a small application to assemble a document ... when I could just download the document straight up.

When page loads get too long, I switch on pagination.

> cached with Nginx/Varnish, and further cached on the backend with Redis

My own setup is memcache for working objects. WP-supercache is configured to write gzip'd pages to disk and nginx is configured to serve .gz versions of page directly from disk whenever they're found. The OS file cache does the rest.

> I've seen WP sites that I had to debug/fix with 200+ queries per page.

When you see hundreds of queries per page, check your plugins and widgets. Wordpress is not my favourite software by any measure, but out of the box it's not that dumb.

This is exactly it for me as well. I'd use Jekyll/Pelican again if comments were somehow built-in. Disqus is still a good service for those who need it.

You need to understand a little history to understand why WordPress is where it is today.

The first blog software to get really popular was Movable Type (http://en.wikipedia.org/wiki/Movable_Type). MT was basically a set of Perl scripts that provided a nice GUI backend to manage content stored in a database. But the actual public-facing sites it created were all static files. When you hit "Publish," it would scan over the database and grind out a new set of static HTML files.

This made scaling a Movable Type site really easy, because even back then Apache could serve static files like a monster on practically any hardware. But it meant that when you changed something on your site, that change didn't show up there immediately; you had to wait for a rebuild operation to complete and a new set of static files to be ground out to see it. The larger your site got, the longer that rebuild operation could take. "Rebuilding" became to Movable Type what "buffering" used to be to RealPlayer (remember that?).

Then, after a few years of MT being the leading product in the then-rapidly growing blog market, the developers behind it decided to jack up the pricing structure for it (http://scott.yang.id.au/2004/05/license-changes-with-movable...). MT was not open source, it was proprietary, for-pay software, so if you wanted to use it you had to pay what they asked for it. And suddenly it looked like, instead of being twenty bucks, a valid MT license was going to cost hundreds.

As you can imagine, people freaked out and started looking at the available alternatives. There were lots, but none of them were particularly impressive at that stage. But some prominent people (see http://web.archive.org/web/20060410125402/http://diveintomar...) noticed a little project called WordPress that, while pretty feature-poor compared to MT, had two big things going for it: it was 100% GPL (so no worries about future price changes), and it used a dynamic publishing model where pages were generated live on request through database queries instead of being baked into static files.

"Hooray!" everyone yelled. "No more rebuilding!" And so the rush to WordPress began. It was pretty obvious at the time that using live, dynamic pages would be a scaling headache. But most blogs never get enough traffic for scaling to become an issue, and dynamic pages meant not having to wait for rebuilding anymore. Never underestimate the power of appealing to impatience.

The irony, of course, is that once WordPress got popular, people started noticing how inefficient dynamic pages could be, and the trend of the "static site generator" was born. SSGs are now very hot and buzzed-around, at least among programmers (they're generally too nerd-optimized to appeal to the general market). But there's nothing really new about the idea; they're just the pendulum swinging back to where Movable Type was in 2001.

Perhaps someday an SSG will dethrone WordPress. If that happens, mark your calendar; you'll want to be prepared when "instant updates!" become a selling point again, five years or so later.

I would say that the plugin offerings is key to its success. Even if I often criticize my wordpress installation I prefer to install a plugin in one click than spend a lot of time at the html/css/js level.

Beyond that I couldn't understand their product feature priorities: no headings in their wysiwyg editor, no good themes. I was looking to something like http://om.co which is wordpress but using commercial themes not available for the general public.

"Some prominent people noticed" isn't quite accurate. Matt Mullenweg actively marketed WP to major bloggers.

I have said elsewhere that the solution to having to choose between static and dynamic is to make site updates POST-based with queuing logic. The term of research is Staged, Event-Driven Architecture (SEDA).

Wordpress kind of makes me realize how incredible PHP is. It's a 'program of pages.' Which can basically hot swap update itself as it runs. You can download plug-ins in the control panel and extend the functionality of your site instantly. I still get mixed up trying to wrap my head around it even after 15 years of using it. It's a program that exists in different files that can be updated individually while the program itself is used. Most of the 'state' is temporary or stored in dedicated memcaches or databases. It makes you wonder what if other applications were written in a PHP style structure - what would that be like?

PHP succeeded in part over mod_perl because of the very statelessness you describe. State is delegated to a database or a memory caching service; each request is answered from scratch.

mod_perl allowed programmers to sink their application code deep into the guts of Apache. In retrospect, this turned out to be a dead-end because it is difficult to install, difficult to update and difficult to scale up.

PHP's lack of power and low cost of operation ("here's your FTP account, have fun") made it much more attractive to shared hosts.

In short, operational advantages outweighed programming advantages.

That would be a Lisp Machine or a Smalltalk environment.

Database features are important for the CMS aspects of Wordpress, not just the blogging aspects.

Although Wordpress crashing under load is what caused me to switch to Octopress.

>Although Wordpress crashing under load is what caused me to switch to Octopress.

It's all about the caching plugins. I had my blog hit 300k visits in the span of 8 hours, peaking at about 100 visits per second on a shared web host. Utilizing cashing plugins means every single person that visited my blog was served static content that refreshed itself every 10 minutes.

My Wordpress blog on Dreamhost shared hosting w/ WP Super Cache and Cloudflare could easily handle 100 visits/second.

200 visits/second caused blank pages half the time.

300 visits/second? Boom.

I've never used WP Super Cache so I can't comment there, but having used W3 Total Cache I can say that the performance I get with W3 Total Cache is exactly the same as when I tried out Jekyll/Octopress since both solutions are only serving static HTML files and nothing else.

Then again, 300 visits per second, estimating 250KB for the entire page (css, js, images, etc) is about 75Mbps which is going to be a lot to ask from any shared hosting provider.

Another thing to look at is using nginx as a web server. It's a lot more lightweight than Apache. Paired with a solid SSD drive and varnish caching, and you can get some really great performance out of WordPress.

I've seen Varnish choke on mainline code: the Recent Comments widget, which can invalidate an entire site whenever a comment is posted anywhere. I have one site which caused both WPEngine and Page.ly to lock up on just 30k views/day because of that.

What I do with my VPS:

1. WP Supercache is configured to write .html.gz copies to disk.

2. Nginx is configured to check if such a file exists before referring anything to PHP. If it finds the .html.gz, it will serve it directly from disk. Ordinary file caching gives it the advantages Varnish promises.

3. WP Supercache appears to correctly handle invalidation when a new comment is posted.

Probably had less to do with wordpress and more to do with shared hosting setup (memory allocation, disk space, process allocation and upstream bandwidth).

Wordpress can handle insane amount of traffic if you know what you are doing. 99% of wordpress borking under sudden spike in traffic issues can be boiled down to

- Not using caching

- Using shared hosting

And I don't understand Wordpress as a CMS. It's frankly a terrible one as well. Poor internationalization support, doesn't meet government accessibility requirements, its non-structure makes maintenance and teamwork almost impossible, its plugin structure is a landmind, terrible (no) dependency management, encourages deployment via FTP, and a terrible security model.

I'm not a Python guy, but Plone kicks its ass in pretty much every way as a CMS.

Wordpress can't seem to choose what they are- and they fail miserable from a technical perspective (yes, I know they have a huge market share, but I'm not one to believe that the markets actually pick the best option. McDonalds burgers are not the height of cuisine).

Like McDonalds, it succeeds because it's the easiest, cheapest option which is good enough (not good, but good enough).

Features like category lists are implemented via database queries. There's little wrong with that; what's wrong is that small result sets can't easily be pinned in memory.

As with website software, cache performance probably has a great deal to do with how WordPress performs.