News for hackers. Stuff that matters.
Re:9 million hits/day with 120 megs RAM by jroes on Wednesday August 31, 2011 @09:39AM (#2945463)
Blogs really don't need a PHP, Rails, or anything backend. It's static content.

Here's how I think blogging should work:

1. Visit a web app where you create your blog post, add pictures, use your rich text editor, that sort of thing.

2. Click the "Publish" button, which generates static HTML and runs any other processing like tag generation or pngcrush.

3. Your static HTML gets pushed out to some server that does the hosting for you. It could even be one of those really cheap shared hosting providers.

If you really want comments, let someone like Disqus or Intense Debate handle it. Pretty much any dynamic feature you need can be outsourced.

Re:9 million hits/day with 120 megs RAM by jacques_chester on Wednesday August 31, 2011 @09:46AM (#2945484)
That's pretty much how MovableType used to work.

Fun fact: it was terrible. Most bloggers I know hated it.

Wordpress is a bucket of poop too, but it has the quality that when you press 'publish', the site is updated in seconds, not minutes.

(I believe MT has changed since then).

Re:9 million hits/day with 120 megs RAM by pwaring on Wednesday August 31, 2011 @09:48AM (#2945488)
It's how Greymatter and a few other systems used to work as well. The main problem was determining which pages should be updated, which usually resulted in the entire blog being rebuilt. This took several minutes if you had a few years of archives, and if anything went wrong part-way through the process you had to start it all again.
Re:9 million hits/day with 120 megs RAM by jacques_chester on Wednesday August 31, 2011 @09:51AM (#2945502)
I've ruminated in the past about a better blogging system based on updating on-disk caches based on POST and not GET. I think everybody who watches Wordpress trip over and burst into flames when you so much as breathe on it has.

The main problem to solve is inter-page dependency. The rest is not that hard /famouswords.

Re:9 million hits/day with 120 megs RAM by pwaring on Wednesday August 31, 2011 @10:20AM (#2945617)
You can get too carried away with scalability though. I run half a dozen WordPress blogs, plus some static sites and email for multiple domains (with AV and spam filtering) within 500MB of RAM. I don't need to implement any caching because I already have sufficient resources.
Re:9 million hits/day with 120 megs RAM by mkr-hn on Wednesday August 31, 2011 @10:22AM (#2945623)
Get this: http://wordpress.org/extend/plugins/w3-total-cache/

It's called a caching plugin, but it also compresses and minifies stuff. Any site can benefit from loading faster. You can turn it on and let it go. It has sane defaults.

Re:9 million hits/day with 120 megs RAM by rawsyntax on Wednesday August 31, 2011 @11:34AM (#2945985)
Thanks for the info, after hearing people praise wordpress as the holy grail of blogging, I found it to be very very slow. Hopefully this plugin will help
Re:9 million hits/day with 120 megs RAM by byoung2 on Wednesday August 31, 2011 @12:45PM (#2946343)
WordPress is usually praised for usability, not performance. To make WordPress faster and more efficient, you need a caching plugin like W3 Total Cache. It caches your pages in static files, caches queries and results in memcache, and rewrites css, images, and js to CDN paths if you have a provider. In front of this, I would have Varnish cache as much as possible in memory, for even faster access. W3 Total cache can even invalidate your Varnish cache when pages are updated.
Re:9 million hits/day with 120 megs RAM by tombot on Wednesday August 31, 2011 @04:16PM (#2947356)
Cannot give Varnish enough praise, it can tubo charge even the most expensive / slowest loading pages and uses minimal resources.
Re:9 million hits/day with 120 megs RAM by easp on Wednesday August 31, 2011 @12:42PM (#2946332)
I found that a major contributor to page generation times on Wordpress is the time taken to load and parse all the PHP files. I installed a PHP opcode cache (APC) as I recall, and it made a huge difference, because the parsed representation of all the Wordpress code is cached in memory (or disk). I'd do that before installing any Wordpress caching plugins, though total cache has other nice features, like minification. Plus it has three different levels of caching. You can cache entire generated pages, and/or objects and/or DB results.
Re:9 million hits/day with 120 megs RAM by jacques_chester on Wednesday August 31, 2011 @08:55PM (#2948213)
> I don't need to implement any caching because I already have sufficient resources.

I want you to print this out and stick it somewhere out of sight. When one day you are linked to on Reddit, Slashdot, a major news site or the like, pull it out for a hearty laugh.

Re:9 million hits/day with 120 megs RAM by pwaring on Thursday September 01, 2011 @03:53AM (#2949054)
There's a risk vs effort balance involved in these sorts of decisions. The risk of any of my sites getting Slashdotted is sufficiently low that it's not worth spending time preparing for such an eventuality, and I'm prepared to accept the consequences if it does happen.
Re:9 million hits/day with 120 megs RAM by jacques_chester on Thursday September 01, 2011 @04:20AM (#2949112)
Installing WP-super-cache or W3 Total Cache takes about 5 minutes and will do the most effective 80%.
Re:9 million hits/day with 120 megs RAM by lobo_tuerto on Thursday September 01, 2011 @01:27PM (#2950906)
Also, I think W3 Total Cache is an order of magnitude easier to install than WP Super Cache.
Re:9 million hits/day with 120 megs RAM by LiveTheDream on Wednesday August 31, 2011 @02:31PM (#2946887)
> I don't need to implement any caching because I already have sufficient resources.

Define "sufficient resources". It might be enough for your normal traffic, but without caching those blogs will crash from the /. effect (unexpected traffic spike).

Re:9 million hits/day with 120 megs RAM by pwaring on Wednesday August 31, 2011 @05:07PM (#2947554)
Sufficient = enough for normal use. My point was that people tend to get worked up and spend time on caching strategies when there's no need for the vast majority of sites.

Even with caching, I suspect Apache would fall over if I got slashdotted.

Re:9 million hits/day with 120 megs RAM by LiveTheDream on Wednesday August 31, 2011 @05:42PM (#2947688)
I agree there is a balance between sane and insane efforts at caching, but I disagree that no caching is a viable choice. Apache and Wordpress with no caching on a smallish VPS will fall over really quickly. This is unacceptable when you can get huge benefits very easily by installing a widely available plugin like WP-Super Cache or WP-Total Cache. It's almost no work to get that set up, and will help you survive a mini-slashdotting. That's going to let you serve an order of magnitude more traffic than usual.

An easy solution like those plugins is such a big win for so little work, I just can't agree with "it's good enough for normal use". The last thing you want is for your blog to fail when you are getting a traffic spike.

Re:9 million hits/day with 120 megs RAM by nknight on Wednesday August 31, 2011 @11:43AM (#2946035)
I'm not sure I understand that.

Around 2002-2003, I built a gigantic, sub-optimal, regex-heavy Perl script for automatic updating of webcomic sites (it's still in use on several sites I know of, ugh).

Each day of the archive had its own separate HTML page generated from a template.

I never got around to the complexity of supporting selective updates, because brute-force regenerating the entire archive every time the script ran, even to several thousand comics, took just seconds on anything but Windows (whose file I/O was just too bloody slow, but even then the run only took a minute or so).

Re:9 million hits/day with 120 megs RAM by pwaring on Wednesday August 31, 2011 @12:13PM (#2946188)
This was on a shared hosting platform, where the provider probably oversold capacity, and without access to the command line. I suspect part of the delay was waiting for the CGI script to send data to the browser.
Re:9 million hits/day with 120 megs RAM by flyosity on Wednesday August 31, 2011 @10:42AM (#2945719)
I've been using MT since 2003 and never thought it was terrible. Publishing times (full-site publishing) was always slow but publishing an entry only took a few seconds. The new Melody project forked MT and updated the UI, it looks great now.
Re:9 million hits/day with 120 megs RAM by byoung2 on Wednesday August 31, 2011 @11:55AM (#2946096)
I used MovableType when rebuilding steves-digicams.com (not by choice), and it really slowed to a crawl when we did a full publish (30,000+ articles). Luckily we only had to do that a few times when setting up the site. I don't see the advantage of static files being worth the hassle of publishing when you can achieve the same or better results using a caching proxy like Varnish. We used Varnish in front of Apache on ultimatecoupons.com (all dynamic, built in the Symfony PHP framework), and once cached, pages load in 15ms (first byte in browser).
Re:9 million hits/day with 120 megs RAM by g-garron on Wednesday August 31, 2011 @12:31PM (#2946276)
http://daringfireball.net/ uses Movable Type and gets a lot of visits, and I'm sure it can handle a lot more. Yes, static content is the key.
Re:9 million hits/day with 120 megs RAM by burgerbrain on Thursday September 01, 2011 @01:30AM (#2948792)
"the site is updated in seconds, not minutes."

That is an implementation issue. That problem is in no way inherent to static content.

Re:9 million hits/day with 120 megs RAM by voltagex on Monday September 05, 2011 @07:00PM (#2963536)
The problem with saying "Wordpress is poop" - as a developer, that problem report has no info I can actually use to fix anything.
Re:9 million hits/day with 120 megs RAM by mkr-hn on Wednesday August 31, 2011 @10:00AM (#2945544)
This is what you get with WordPress and W3 Total Cache (the caching plugin I use). It generates HTML files, then uses some fancy .htaccess magic to send requests to them. It can even push to a CDN. My blog survived being whacked by 3000 hits from HN over the course of 2 hours without blowing up.

Pre-gzipped pages are nice too. :)

Re:9 million hits/day with 120 megs RAM by EponymousCoward on Wednesday August 31, 2011 @11:11AM (#2945863)
3000 hits over 2 hours is < 0.5 hits/second and nothing to write home about, sorry.
Re:9 million hits/day with 120 megs RAM by mkr-hn on Wednesday August 31, 2011 @11:31AM (#2945971)
The raw number wasn't the point. WP is notorious for collapsing under a gentle load without caching.
Re:9 million hits/day with 120 megs RAM by jroes on Wednesday August 31, 2011 @04:12PM (#2947333)
The commenter also didn't take into account that the req/sec was probably not evenly distributed across 2 hours.
Re:9 million hits/day with 120 megs RAM by iqster on Wednesday August 31, 2011 @01:19PM (#2946511)
I can vouch for that. I've had vanilla WP die on me from the gentle load that comes with a link going on HN. I assumed it was performant out of the box ... learned my lesson!
Re:9 million hits/day with 120 megs RAM by tombot on Wednesday August 31, 2011 @10:30AM (#2945671)
W3 Total Cache really is a first rate plugin, we've had great success throwing Varnish https://www.varnish-cache.org/ infront of WordPress
Re:9 million hits/day with 120 megs RAM by ftownes on Monday September 05, 2011 @05:47PM (#2963409)
W3TC integrates with varnish and supports a number of backend web servers.

The permissions changes it asks for are because caching plugins need to create files on the server and most servers are either poorly configured or managment of groups:users is difficult, so temporarily requesting that users modify permissions so that the caching plugin can create the files for the user improves the installation experience via lowering the technical experience needed and the number of steps required.

I don't recommend suPHP in practice as it's quite slow, similar if not better security can be realized by using a reverse proxy, and then running PHP in fastcgi mode for example with a backend web server.

Re:9 million hits/day with 120 megs RAM by xinsight on Wednesday August 31, 2011 @11:58AM (#2946116)
My concern with the wordpress cacheing plugins I've looked at is that they require the webserver to have write access to the webserver directory. This is particularly dangerous on shared webhosts, as a malicious customer on the same server could write php files in your cache directory.
Re:9 million hits/day with 120 megs RAM by tombot on Wednesday August 31, 2011 @04:13PM (#2947336)
I'm pretty sure that W3 Total Cache writes to /wp-content/uploads which you'd need to be writeable anyway.
Re:9 million hits/day with 120 megs RAM by mkr-hn on Wednesday August 31, 2011 @01:48PM (#2946660)
That's what suPHP is for.
Re:9 million hits/day with 120 megs RAM by rdouble on Wednesday August 31, 2011 @10:16AM (#2945601)
Blogger used to work this way. You used to be able to download an archive of your site and host it on your own server! The thing is, unless your site is very simple and small, regenerating static HTML becomes more of a headache than maintaining a system that serves the content out of a database.
Re:9 million hits/day with 120 megs RAM by chopsueyar on Wednesday August 31, 2011 @10:22AM (#2945622)
That is how viaweb worked. All product pages were static, and once you finished editing, you would 'publish' and it would create the static files for your online store.
Re:9 million hits/day with 120 megs RAM by whackedspinach on Wednesday August 31, 2011 @10:24AM (#2945636)
I used Jekyll to generate my static content a few months ago/ Just type a text file, and then push it out to the server. I really thought that was cool.
Re:9 million hits/day with 120 megs RAM by handzhiev on Wednesday August 31, 2011 @01:09PM (#2946460)
"If you really want comments, let someone like Disqus or Intense Debate handle it. Pretty much any dynamic feature you need can be outsourced."

If you are fine with your comments residing somewhere else, yes. I usually prefer to have them with me and to have them search engine indexable (that's free content created from users for me)

Re:9 million hits/day with 120 megs RAM by jroes on Wednesday August 31, 2011 @01:18PM (#2946503)
Good point. An unfortunate casualty. It might be possible to augment the statically generated blog posts with data via the comment providers' APIs periodically.
Re:9 million hits/day with 120 megs RAM by sehugg on Wednesday August 31, 2011 @10:38AM (#2945707)
Vignette StoryServer (http://philip.greenspun.com/wtr/vignette-old) .. if you see commas in a .html URL, thank them for it. And TCL as a scripting language, whoo-hoo :)
Re:9 million hits/day with 120 megs RAM by drzaiusapelord on Wednesday August 31, 2011 @10:06AM (#2945562)
I do this with WP's caching plugin and a caching plugin for Drupal. It bugs me that these aren't part of the official distribution and need to be plugins.
Re:9 million hits/day with 120 megs RAM by phsr on Wednesday August 31, 2011 @10:45AM (#2945732)
In the realm of CMSs, RedDot works this way. It is a bake CMS, when you publish, it creates the whole site as a set of files. You can use it to generate php, asp, jsp etc pages, but it takes the data stored in a database and creates the files from the taxonomy stored there.
Re:9 million hits/day with 120 megs RAM by Maci on Wednesday August 31, 2011 @12:09PM (#2946162)
You must be talking about Stacey then as it does some of the described "magic". :)

http://staceyapp.com/

Re:9 million hits/day with 120 megs RAM by zachrose on Thursday September 01, 2011 @04:51AM (#2949176)
staticmatic deploy amazon_s3

FTW