Very cool solution! How easy was it to collect data for the classifier? I'm just wondering if the company had huge detailed logs or if you had to train the classifier in real time.
They had data going back for years which really helped a lot. Still, coming up with all the signals was hard work and re-computing the various weights took days per run. Added complexity is that to ensure the non-absive users' privacy the system had to operate on an absolute minimum of information per user.
The resulting program does not run in real time, it basically runs every few minutes to have a good look at whoever signed up since the previous run and then assigns a probability to each of the new accounts.
This still catches the bad guys long before they can get up to any mischief (they still have to verify their account) and does not get in the way of the speed of the forms processing during sign-up.
I can imagine it being a grueling effort! Part of what has kept me away from using a classifier or any kind of 'neural network'-type solution is that the initial configuration always seems to be something of a shot in the dark, and tweaking the weights is a non-trivial process with, as you said, a lot of wait time. When it works, though, it has that wow factor that clients love. I give you all props for taking that risk and having it pay off!
The system has gone into production a while ago and the results are quite amazing, there is a signficant reduction in the amount of manual work to be done to combat the abusers and the number of false positives is extremely small (and so far each false positive that we identified has led to an improvement of the code and similar false positives were not witnessed afterwards).
These are annoying problems, but any free and open service will sooner or later be faced with having to make their abuse defences scale.