It's one thing to say to someone, "even if you don't realize it and don't think it's true, you're at least a little racist," and have to leave it at that. Without some means of substantiating the claim, you don't have a good counter for the immediate "no, I'm not!" that such a statement is very likely to provoke. You're likely to end up with pointless recrimination on both sides, because what else, really, have you got? It's semiotically equivalent to telling someone they've "sinned in their heart", in the absence of any tangible evidence beyond your fervent belief this must be so - and exactly as useful as that, besides.
The IAT offered a means of substantiating the claim. Armed therewith, you could say, "even if you don't realize it, you're at least a little bit racist, and I can prove it." That's a different and rather stronger proposition, and it's something which I think I don't too far miss the mark in saying that it's something a lot of people have been looking for for a long time. I can see why someone who believes he's finally found that thing might find it difficult to acknowledge that there's less substance to it than initially imagined. And for similar reasons, I can understand why an otherwise objective researcher might have the same trouble.
The thing is, it's 100% correct to evaluate new claims in light of your priors. That's absolutely necessary if you want to make good decisions - my expectation that special relativity will not be shown to be inconsistent is so strong that no matter how interesting or new an argument is against it, I will shred it to bits way more viciously than I'd pick apart a simple replication of special relativity's claims.
But.
Some priors don't come from reliable places, and we have to be way more careful about applying them. In general, the more politically or socially relevant an issue is (gender, race, money, morals) the less any of us should trust both a) our own priors, and b) the priors of researchers in the area, who have many ways to unwittingly and without bad intention put fingers on the scale. In those situations we really have to throw our hands up and wait for meta-analyses to tell us what's up.
In this case, it's super worth noting that even if the IAT is a garbage test, that doesn't mean that implicit bias doesn't exist, or that racism is not a problem. It just means the test doesn't measure it, and we shouldn't be funding further research that leans on it as a tool. Which is why I really don't understand why criticisms of the test draw such intense opposition.