To me, that sounds like a convincing argument that your specialty is not worth learning. I'd rather work someplace that wants to invest in their employees, rather than expecting them to already be experts in some obscure techniques.
Are you hiring people who are brilliant, or people who are so desperate that they'll spend a couple months preparing for one interview?
Let's not flee to abstraction. Matasano/NCC does software security. We were willing to invest some time and money to bring people up to speed in software security and exploit development.
Not your cup of tea? Totally fine. Not everyone is interested in doing security.
I do have a problem with passing on candidates who were sold on our subspecialty, had an aptitude for it, but could not pass an interview on it "cold".