There have been countless other plugins that have tried and failed to get critical mass, but Flash is the only one that's made it. Adobe Reader and Java are the nearest competitors and the install base is much lower.
Why? Initially - download size. For several years (while most people were still on dial-up) the Flash plugin was kept under 200K. Macromedia deliberately kept the plugin as small as possible so that it wasn't a burden to install it. They did internal testing where they artificially increased the size of the plugin and there was an immediate drop in download numbers. At some point as the size increased, the numbers would drop dramatically and they'd know that was the limit they had to work to when adding new features. Upgrades and installation was always kept as easy as possible.
They then cemented the install base with video playback that didn't act like it was stapled onto the browser. Prior to Flash with video, all the web based video options were clunky and almost completely closed to customization. Until there is an open standard for video playback, supported across all browsers, I think Flash will remain ubiquitous.
I can't see Silverlight taking over. It's not cross platform for a start. 3D isn't enough reason for people to install it and, other than Microsoft, no self respecting web site wants to risk alienating users by forcing an install.
Disclosure - I used to work for Macromedia. So I'll freely admit some bias!
Maybe they're waiting to implement Chrome in Linux first?