I think we're saying that most modern polyfills do a feature check before installing the polyfill... thus if you use Chrome with an IE agent, the polyfill still checks before implementing -- it's just a little slower than not installing the polyfill to begin with. Anyway, given the percentage of people who do this, UA detection seems like an acceptable tradeoff.