You have to remember that Polymer is a library that helps you implement Web Components. The APIs for Web Components are being integrated directly into the browser. Chrome already has them, and Firefox is landing them now. Even Microsoft is interested.
This is not another framework.
edit: Check out http://jonrimmer.github.io/are-we-componentized-yet/ for a summary of the progress on the Web Components standards.
So it's not that they are "built for Chrome". It's that they are built for "new HTML5 specs" and only Chrome happens to support them right now. Point me to where Silverlight was an HTML spec?
Now, things like NaCl on the other hand are a lot more similar to what Microsoft used to do, but there's still a major difference, and that is that what they are building is open source and everyone can do their own implementation of it. You could never do that with ActiveX or Silverlight. Plus, you could also say Firefox is doing the same as Google with asm.js, which nobody else has adopted so far.
These things are a little more like that, but at least they are open source. The HTML5 features, though, just happen to be implemented faster by Chrome.
Kinda of remind me of how "open" android is progressing. More and more "features" are pushing into "google experience" layers - also look at how open is the android TV.
BTW, chrome is doing the same on the extension side.
I see the pro/con on the "extension move" - reduce the security risk , etc.
But it is sure feel like the Jedi master is feeling the power of the dark side - "working within the system" is not as fun as just being the Empire.
Regarding IE failures -- do you have any more information? All of the elements should be cross platform. It would be great to file these issues at https://github.com/Polymer, if you haven't already.
BUT, it is slow in FF30, Safari and failed a lot in IE11.
IMO, Big G is trying very hard to push web technologies that tightly couple to Chrome - same as what MS/IE did a few years back trying hard to force ActiveX, Silverlight down the developers' throat.
I'm betting < 20% chance Polymer will still be in IO talk 3 years from now. (remember GWT, Dart, Angular?)