Perhaps because "homophobia" isn't a good word for it.

Hitler wasn't a jewphobe.

It's a good word for it. It has an established meaning, it is universally understood, etc etc. The fundamental use of language is to communicate, and I don't really think homophobia fails in this respect. There are plenty of non-"good" words for other things in the English language that get used very often without people quoting them or complaining about their use.

If you went around capturing and torturing spiders, would you be called an arachnophobe?

It's just a bit silly. Yes, words get misused all the time, and English is imperfect, but this one just seems ridiculous to me.

Well, homophobia is generally supposed to be considered in the light of Yoda-ism "Fear leads to hate..."

Verging offtopic I guess but:

As an example though, I'm not a pedophileaphobe - I'm not afraid of pedophiles. I think what they do is wrong.

The number of people who are "afraid" of homosexuals is miniscule. Most people who you'd label "homophobic" actually just believe that homosexuality is wrong, not that it's scary.

So the "homophobic" tag is pretty silly.

They are not scared of homosexuals, but of homosexuality. It represents a threat to set values and a set way of living to certain individuals. If it wasn't considered a threat to those values, why suppress it? It's homophobia.

as trivial aversion do