If your site was in the top 1,000, you'd have a different experience since the spammers would be adapting to every one of your exotic captchas no matter how many times you change it.
1. fofou.org hosts the forum for my SumatraPDF site, which is 33,279 in Alexa. I think that's enough of a target, more than most forums out there.
2. I don't think it's productive to bring top 1000 websites into this particular discussion. OP's forum wasn't that popular and most forums, by definition, won't be either.
Once you're there you're playing a completely different game and at that level you should have resources to win it. I don't see spam on Wikipedia, StackExchange, Quora, HN etc. When there's will, there's a way to fight spam.
If you have showdead turned on you see a variety of obvious spam (handbags and shoes) and non-obvious spam.
And there have been some aggressive voting rings.
Your software solutions can do only so much. Wikipedia, StackExchange etc has Human monitoring (volunteers who report spam) in addition to automated stuff.
Its tough keeping up with spam.
P.S. Thank you for SumatraPDF!
No matter how large your site, if you give people you trust the tools to stomp out spam/bad content, they will use it and you eventually "win."
I ran a forum using SimpleMachines. Turned on all the spam fighting stuff. Didn't matter. Bots or Mechanical Turks made 10-30 new accounts a day. I tried moderating the accounts but who wants to spend time every day to delete the [email protected] account.
I gave up and shut it down.
At the moment it stops pretty much everything that gets thrown at it, and the rest is stopped with Bad Behaviour and Mollom.
Legit posts still turn up no worries. All the crap just falls away.
I also have fail2ban rules in, so those that trigger modsecurity too many times get added and iptabled. Helps cut down on the repeat attempts.
I don't think it's hard to block spam. I do agree it's time consuming though, keeping everything up to date.
I could say similar things about phpBB, but that's quite a dead horse to beat (hey, at least they use a proper license).
I also wonder if Amazon has a content filter, to block URL patterns in Turk tasks.
As for bots, IP banning goes a long way.
Essentially, like Jesus and the poor, spam we will always have with us (as long as there's an incentive).
Short of evidence that the blog spam was malicious and unaffiliated with the site, there should be no rehabilitation. If the site has valuable content the owner can move it to a new domain and try again. This time without being a scumbag.
There is a solution to his particular problem: better forum software.
I don't want to trivialize the problem of writing spam-resistant forum software but it's not such an insurmontable problem (this forum being a proof positive for that).
For the reference: I've been running a fairly popular forum (http://forums.fofou.org/sumatrapdf) for several years, using forum software that I wrote.
I don't even require the user to log in, I don't require moderation for posts and yet I had zero automatic spam (I attribute this to my unusual captcha http://blog.kowalczyk.info/article/16fw/Best-captcha-is-exot...).
I occasionally get human spam i.e. someone writes a post with the only purpose of linking to some other website. I just hellban them after I see the post in my rss reader.
And I didn't even write any special anti-spam code (other than hell-banning), because I'm lazy. I can easily come up with simple ideas e.g. putting all posts that contain links in moderation queue.
To reiterate: his problems were caused by a crappy forum software that didn't do much to protect from seo spam.
I don't really know how current best off-the-shelf forums fare in this respect.
I would rather not spend my time maintaining my own forum software so I have high hopes for http://www.discourse.org. I'm sure StackOverflow had plenty of spam problems so discourse people should understand the problem.