Those papers may be a few years old and the state of the art may be different. But after an initial look I'm missing the reason that, compared to these captchas, you think that "3% is awful."
>If a CAPTCHA can be solved or guessed in an automated fashion then attackers can just throw more (likely compromised) machines at the problem at little cost.
I'm not ready to buy this. I would think every captcha is going to have some failure rate, even if it is extremely low. If attacks were absolutely free, then it wouldn't matter what the attack success rate was. Computers are fast, but not infinitely fast. Bandwidth is cheap, but not infinitesimally cheap.
Random guessing gives you 3%, which is worse than random guessing on either the MSFT or reCAPTCHA.
This is far worse though. A simple loop over the 30 positions, running the output through an OCR engine would give you nearly 100%.
FWIW, I wrote the paper and AFAIK was the first person to break reCAPTCHA. I worked on the original MSFT Passport/Hotmail CAPTCHA system and improved MySpace's CAPTCHA which took spammer registrations from ~1,000,000/day (automated) to a few thousand (manual) in late 2007.
I really don't think I'm comparing apples and oranges. I'm simply noting that any novel captcha can look good when attackers can't be bothered to actually attack it.
My experience with an unknown site has differed from yours dramatically. I have a small, custom site that I share with less than 10 friends and I have had to progressively step up the security (none -> captchas -> passwords) because it has been attacked with thousands of spam messages on several occasions.
1. The "elbow" technique. Worked OK for years, then spam began to rise.
2. A slight variation to defeat generalized bots. I changed the field label to, "Type 'humour' with American spelling." This held on for a few more years.
3. A hashcash-based solution, where a proof of work is generated using JavaScript in the client and required to post a comment. It's calibrated to take 20-30 seconds on typical computers, and doesn't start until you focus the comment fields so it's not burning CPU for people who just view the page. This is still working well for me. I get occasional spams, but from looking at the logs it appears almost certain that these are from real humans who are actually spending the 20-30 seconds waiting for the hashcash to compute. I only have to delete one or two spam comments a month for the moment.
Wonder what caused your site to suffer so much more.
Attacking your site, with minimal effort, would have yielded a 100% success rate. Attacking this site with a guess would have yielded a 3% rate. Some of the best captchas have been attacked with success rates in the high double digits: http://en.wikipedia.org/wiki/CAPTCHA#Computer_character_reco...
So 3% doesn't look too bad after all.
I do, by the way, agree with you that the vertical lines attack would work really well.