Wouldn't the spammers constantly have random different referrers?

Yes. The Ukrainian spam I get is almost entirely unique referrers each time, so individual results rarely show up enough times to even rank. Semalt.com and something called speedfox are the only ones that really show up consistently enough from the same source for host-by-host blocking to do any good. The others just rotate through different hosts on a routine enough basis that it'd be more work than it was worth blocking them one by one.