News for hackers. Stuff that matters.
Re:Bulletproof your blog – survive the Hacker News effect by tikhonj on Sunday December 25, 2011 @07:26PM (#3391905)
He forgot the geekiest option: write your own blogging engine. It's a perfect excuse to use that new language and framework you're learning! Or, if you're not learning anything now, it's a great excuse to start learning.
Re:Bulletproof your blog – survive the Hacker News effect by masnick on Sunday December 25, 2011 @11:53PM (#3392241)
While this is a great learning experience, it is hard to write a blog engine that will stand up to traffic spikes on the first try.

If your goal is learning a new language or framework, this is a great idea. If your goal is to have a reliable, traffic-resistant blog I think it's best to stick with a tried and true platform.

Re:Bulletproof your blog – survive the Hacker News effect by seunosewa on Monday December 26, 2011 @12:53AM (#3392325)
It's not hard. Just write a blog engine that generates static html pages.
Re:Bulletproof your blog – survive the Hacker News effect by masnick on Monday December 26, 2011 @12:58AM (#3392329)
Right, if you write something akin to Jekyll it will work fine.

However, I think most people would probably be trying to learn a framework like Rails or Django if they were writing a blogging engine to learn a new language/framework.

Re:Bulletproof your blog – survive the Hacker News effect by mapleoin on Monday December 26, 2011 @03:50AM (#3392561)
So write your blogging engine as you normally would and then make a button which says: dump to html for each page and then figure out when you want that button to be pressed (i.e. manually or automatically).

More things to learn. Yay!

Re:Bulletproof your blog – survive the Hacker News effect by sunkencity on Monday December 26, 2011 @04:49AM (#3392635)
Build it anyway you like but then wget --mirror before rsync to deploy.
Re:Bulletproof your blog – survive the Hacker News effect by mike-cardwell on Monday December 26, 2011 @07:04AM (#3392768)
Or use caching. My webpages are generated using FastCGI and served by Nginx. So I configured Nginx's FastCGI module to internally cache GET requests for 1 minute.
Re:Bulletproof your blog – survive the Hacker News effect by kijin on Sunday December 25, 2011 @06:50PM (#3391861)
It's a shame that several years after the term "slashdot effect" was coined, self-hosted personal blogs are still vulnerable to this easily preventable phenomenon. The companies that host the majority of the world's small web sites, such as BlueHost, DreamHost, and (heavens forbid) GoDaddy, have not made any innovations in this department. Most VPS's are no different.

NearlyFreeSpeech.NET is really good at handling traffic spikes, for both static and dynamic sites. It combines the convenience of shared hosting with the scalability of cloud computing. You just need to insert more coins when you run out of your prepaid traffic.

Re:Bulletproof your blog – survive the Hacker News effect by marquis on Sunday December 25, 2011 @07:59PM (#3391943)
Where is the financial incentive for these hosting companies to do so? Most users will never see a traffic spike that can take down your average VPS or bluehost site. It seems there is a gap in the market for self-hosted sites that have built-in structures to build upon that allow for traffic spiking.
Re:Bulletproof your blog – survive the Hacker News effect by edanm on Tuesday December 27, 2011 @03:12AM (#3395133)
Isn't the gap filled by companies like Posterous/Tumbler? For the average user, they answer the problem of "put something online and make sure it stays there" perfectly well.
Re:Bulletproof your blog – survive the Hacker News effect by iamben on Monday December 26, 2011 @04:16AM (#3392598)
I had just under 90k uniques in a 24 hour period (and 170k-ish pageviews) on NFS last month and it held up without any problems :-)

I think it cost me about 10 cents. Brilliant.

Re:Bulletproof your blog – survive the Hacker News effect by Joakal on Sunday December 25, 2011 @09:45PM (#3392092)
NearlyFreeSpeech was the cloud before it became popular.

The other services are just hosting many websites on a server.

Re:Bulletproof your blog – survive the Hacker News effect by riobard on Sunday December 25, 2011 @11:45PM (#3392231)
Disadvantages: Not for the non-geeky. You can only post from a computer where you have your static site generator set up

Not necessarily. I have a very simple static site generator (basically a single Python script, plus a few dependencies) running on a EC2 Micro instance. It watches changes in a shared Dropbox folder (using inotify and dropboxd) of a bunch of Markdown files and templates, and generates HTML posts, index page, and RSS automatically. I could post from any machine as long as it has my Dropbox account, even on my iPhone! No more command lines or git stuff, just write. Dropbox will take care of the rest.

Joe Hewitt's Nerve [1] works this way. I believe Marco Arment's Second Crack works similarly, too, but it's not open sourced yet.

[1]: https://github.com/joehewitt/nerve/

Re:Bulletproof your blog – survive the Hacker News effect by bsphil on Sunday December 25, 2011 @11:56PM (#3392246)
Sounds like it still requires more setup than the average non-geek is willing to do.
Re:Bulletproof your blog – survive the Hacker News effect by masnick on Monday December 26, 2011 @12:39AM (#3392309)
Yes -- if you're comfortable with the command line and a scripting language, you can set up a script to handle your static blog. However, I suspect most people who want to blog don't have these skills or don't want to spend the time messing with the tech.
Re:Bulletproof your blog – survive the Hacker News effect by riobard on Monday December 26, 2011 @03:06AM (#3392498)
Sure. I'm not addressing the non-geeky part of the problem :)
Re:Bulletproof your blog – survive the Hacker News effect by jQueryIsAwesome on Monday December 26, 2011 @02:16AM (#3392443)
Actually it would be pretty easy to create a Wordpress plugin to do something very similar; using ob_start of php and keeping the real wordpress installation inside a subfolder.

I am pretty busy this days but if someone wants to create and opensource project that hundreds of people would love, that would surely be one.

Re:Bulletproof your blog – survive the Hacker News effect by zrail on Monday December 26, 2011 @01:00AM (#3392330)
If someone hadn't set this up as a hosted system I would have done so tomorrow, but it turns out there's already a hosted markdown + Dropbox thing out there: http://calepin.co/
Re:Bulletproof your blog – survive the Hacker News effect by riobard on Monday December 26, 2011 @03:05AM (#3392496)
Actually I've tried Calepin before creating my own solution. Calepin is 99% of what I need, but the last 1%, that Calepin requires me to click the "Publish" on their site every time, drives me nuts. I just want to save the Markdown file and it automatically appears on my site.

This isn't Calepin's fault, though. It's because Dropbox doesn't support event notification yet. The offical Dropbox API cannot inform a 3rd-party app that something changes in a user's Dropbox folder. So the app has to poll, which isn't allowed on a large scale, or ask the user to manually notify the app about updates, which is painful. See this thread on Dropbox forum for details http://forums.dropbox.com/topic.php?id=20226&replies=45

Joe's, Marco's, and mine solutions basically avoid the last part and let me just write.

Re:Bulletproof your blog – survive the Hacker News effect by danneu on Sunday December 25, 2011 @07:39PM (#3391919)
I wasn't a big fan of the Jekyll workflow, especially when I wanted to write a simple post from another device. My solution to that, an app that wrapped Jekyll, just felt hacky.

Instead, I just made a Rails app with fifteen minutes spent dropping in Active Admin and RDiscount to render @post.body Markdown into @post.rendered_body HTML. Then, I aggressively cached pages into the static `public/` directory. I chose Rails simply because I'm using Rails for more serious apps and I wanted a casual app that I could take seriously enough to practice on.

Otherwise, I'd have as much fun building the same thing in any other Ruby framework for Ruby practice and then cache the pages into a static directory.

Isn't that what any other static site generator does?

Re:Bulletproof your blog – survive the Hacker News effect by chubs on Sunday December 25, 2011 @07:52PM (#3391934)
Don't most people with high-traffic blogs simply use posterous or tumblr these days so they don't have to worry about these kinds of things? That's what works for me... then the next problem rears its ugly head: bulletproofing your DNS!
Re:Bulletproof your blog – survive the Hacker News effect by kmfrk on Sunday December 25, 2011 @09:34PM (#3392076)
People with basic blogging needs are spoiled beyond belief with services like Tumblr, but if you need more advanced things to play with, it becomes a little overwhelming.

ExpressionEngine understood the need for people with a lot of needs, but I think everyone eventually have to roll their own (using some sort of framework), if they want something that satisfies their perfectionism.

Re:Bulletproof your blog – survive the Hacker News effect by masnick on Sunday December 25, 2011 @11:55PM (#3392244)
Using a platform like Tumblr makes a lot of sense, but lots of people still don't do this as evidenced by all the stories on HN that link to overloaded servers.
Re:Bulletproof your blog – survive the Hacker News effect by jyap on Sunday December 25, 2011 @08:14PM (#3391960)
When my blog hit number 1 on Hacker News earlier this month (http://news.ycombinator.com/item?id=3328619) it never went down. Running Jekyll on GitHub with my own domain julianyap.com.

30,000 hits in 24 hours.

I remember the "move your site off of GoDaddy" post last week going down this week which was run on Tumblr. Elsewhere Tumblr wasn't down so it depends on the resources they allocate to you. I suspect MG Seigler's blog is on a top tier Tumblr resource pool so I wouldnt recommend Tumblr unless you have constant popularity to your blog.

Re:Bulletproof your blog – survive the Hacker News effect by tvorryn on Sunday December 25, 2011 @09:16PM (#3392054)
I was a little disappointed that when he talked about Jekyll blogs, he didn't mention Octopress. The default theme looks pretty good, it has a decent set of plugins and the code display looks great. It was actually pretty easy to set up too.

http://octopress.org/ My blog using the default theme: http://www.trevoroakes.com

Re:Bulletproof your blog – survive the Hacker News effect by masnick on Monday December 26, 2011 @12:15AM (#3392272)
Good point, I'll add this.
Re:Bulletproof your blog – survive the Hacker News effect by _gd3l on Sunday December 25, 2011 @07:13PM (#3391892)
The Jekyll recommendation is a great one. For quick one-off articles something like pen.io is great, as well. And GitHub Pages are just fantastic if you didn't know already :)

In my experience, MediaTemple handles traffic spikes remarkably well. Maybe not ideal for the super-techies, but running WordPress on it with a caching plugin handles traffic pretty well at the $20 / month plan. And using S3 to serve static sites is fantastic. I'm on my phone right now so it's a pain to find the specific article, but Kyle Shank uses S3 and wrote a pretty handy article at http://kyleshank.com. Check it out and try to find the article. Great read.

Re:Bulletproof your blog – survive the Hacker News effect by jjcm on Sunday December 25, 2011 @07:07PM (#3391883)
I personally would recommend hosting on a VPS with a reverse proxy like varnish set up. Even if you've had very minimal experience with linux systems, it's a cinch to setup and you can get quick, fast caching up in a couple of minutes.
Re:Bulletproof your blog – survive the Hacker News effect by ck2 on Sunday December 25, 2011 @06:54PM (#3391870)
Save dynamic page as html from browser.

Upload to server as html.

Add.htaccess rule to redirect old dynamic page to static page. (when traffic dies, remove rule)

Or if you are running WordPress, just install wp-super-cache and when a traffic spike happens, click the "lockdown" button.

Re:Bulletproof your blog – survive the Hacker News effect by sirn on Sunday December 25, 2011 @07:06PM (#3391881)
Easier: just put this into your .htaccess on traffic spike.

    Header append X-Coral-Control "redirect-home"
    RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx
    RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$
    RewriteRule ^(.*)$ http://your.domain.nyud.net/$1 [R,L]
Or even better, just redirect all traffic with news.ycombinator.com or reedit.com referer (before RewriteRule):

    RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?ycombinator\.com [OR]
    RewriteCond %{HTTP_REFERER} ^http://([^/]+\.)?reddit\.com [OR]
You don't even have to manually disable caching after traffic dies down this way.
Re:Bulletproof your blog – survive the Hacker News effect by apsurd on Sunday December 25, 2011 @11:34PM (#3392211)
Since discovering Jekyll and the fact that GitHub Pages will host your content for free, I think this would be the ideal solution. However Jekyll is a pain in the ass to use, which is why I've been working on http://jekyllbootstrap.com/

Jekyll bootstrap is intended to deploy straight-away to GitHub pages so hosting is free. I'm not finished though so any more eyeballs I get on the source will definitely be helpful. Merry Christmas =)

Re:Bulletproof your blog – survive the Hacker News effect by hajrice on Sunday December 25, 2011 @08:49PM (#3392019)
Surprised the author didn't mention WP Engine. We (along with SmartBear, Foursquare and many others) use it at Helpjuice.com for our blog and it works great.
Re:Bulletproof your blog – survive the Hacker News effect by masnick on Monday December 26, 2011 @12:19AM (#3392282)
I hadn't heard of WP Engine. I'll have to check it out. Thanks for the tip.
Re:Bulletproof your blog – survive the Hacker News effect by zrail on Sunday December 25, 2011 @06:35PM (#3391840)
I wrote kind of a hybrid static blog setup the other day[1]. You write posts in markdown, check them in along side the app, and then push to Heroku or wherever else you want to host it. It's just a little Sinatra app, no database or anything.

[1]: http://bugsplat.info/2011-12-14-yet-another-not-very-static-...

Re:Bulletproof your blog – survive the Hacker News effect by oldstrangers on Monday December 26, 2011 @01:31AM (#3392364)
It is easier than ever today to "bulletproof" your blog. I'm running a hybrid server and have found wordpress + cloudflare + w3tc to be a very solid setup. I got hit with 73,230 unique visitors (94.46% new visitors) in a 24 hour period earlier this month and you would've never noticed.
Re:Bulletproof your blog – survive the Hacker News effect by jaequery on Monday December 26, 2011 @02:39PM (#3393729)
i'm sorry but this author has failed to point out the most commong cause for why a site goes down. it's not because of failing to cache...etc...most dbs will cache for you anyway... the main reason is because you probably are on apache and serving static assets directly w/o hosting them off a CDN or nginx/litespeed. if your page had 150 images/js/css combined, that's 150 hits hitting your server per page load. now multiply that by 100's or 1,000's+ visitors HN brings to your site at any given moment, ofcourse it will go down. apache isn't really good at serving static files, just relieving static assets off to cdn/fileserver is huge.
Re:Bulletproof your blog – survive the Hacker News effect by buster on Monday December 26, 2011 @08:04AM (#3392838)
Re:Bulletproof your blog – survive the Hacker News effect by dsl on Monday December 26, 2011 @05:24AM (#3392683)
I'm really surprised by this article. I thought this was a pretty much solved problem now that CloudFlare exists...
Re:Bulletproof your blog – survive the Hacker News effect by nodemaker on Sunday December 25, 2011 @06:45PM (#3391851)
I find it funny (and perhaps ironic!) that people would want their thoughts kept in some obscure row and obscure column of a remote SQL database.

But perhaps it is better than not storing and sharing your thoughts at all!

In any case...Merry Christmas HN!

Re:Bulletproof your blog – survive the Hacker News effect by zrail on Sunday December 25, 2011 @06:47PM (#3391854)
How is that ironic? Thoughts have to be stored somewhere and databases happen to be really good at storing things.
Re:Bulletproof your blog – survive the Hacker News effect by nodemaker on Sunday December 25, 2011 @06:57PM (#3391873)
It is ironic because the real deep reason we share our thoughts is because we want them to stand out!

Databases are indeed good at storing millions and billions of things and they are designed for it.But no human being is individually capable of creating content that really justifies a "dynamic programming oriented query plan generation" each time someone wishes to access his thoughts!

tl;dr My argument is that static site generation for personal blogs is common sense!

Re:Bulletproof your blog – survive the Hacker News effect by zrail on Sunday December 25, 2011 @07:04PM (#3391877)
Sure. My counter-argument is that for multi-tenant hosting like tumblr where millions of people keep their personal blogs, keeping posts in a database is common sense because they're a known quantity. Backup strategies, optimization techniques, operations procedures are all well known and battle tested.

Also, creating content using a web-based editor that saves to a database is very easy for not-so-technical people to wrap their minds around. Static site generators? Not so much.

Re:Bulletproof your blog – survive the Hacker News effect by nodemaker on Sunday December 25, 2011 @08:26PM (#3391989)
Yes sites like tumblr definitely make sense...My argument is against running your self-hosted personal blog with an SQL server! And of course...my argument is for the technical people only....perhaps someone among us can make a static content generation solution for everyone!
Re:Bulletproof your blog – survive the Hacker News effect by de90 on Sunday December 25, 2011 @07:10PM (#3391887)
But for the majority of people constructing the html\css to do that is a chore\"impossible". Storing it into a database and providing a simplistic word editor to write up new things is the easiest way. Storing the information in a db just allows it to happen easily.
Re:Bulletproof your blog – survive the Hacker News effect by isocpprar on Sunday December 25, 2011 @06:56PM (#3391872)
I don't think that's really irony...how is that worse then some obscure file in some obscure server somewhere? Or some obscure block of memory in some obscure cloud somewhere?
Re:Bulletproof your blog – survive the Hacker News effect by nodemaker on Sunday December 25, 2011 @06:59PM (#3391874)
haha good point but thats why I prefer the Dropbox based approach where I just create a text file and put it in my dropbox folder!