At the very least, Mayer has been part of a real leading tech organization that grew from nothing to everything in record time. That's experience that is also very difficult to replicate. And, the skills she's known for within Google are all product oriented, and that's where Yahoo has failed so miserably and Google has succeeded so fantastically.
For IBM, the decision to hire a non-technical and experienced CEO turned out OK. IBM turned around. But I'd agree that this is probably an exception to the rule.
I think the key here is fresh thinking and ability to get the political/operational/strategic stars aligned inside the organization to /cough/ execute. An experienced person may bring this to the table, and a lesser experienced person may bring this to the table. The key is whether it's brought to the table.
I've had the opportunity to work with people who were very good at personal public relations, and based on that experience, I'm skeptical. Constantly selling your public image means that you can't be wrong, which always results in revisionist history and oddball situations.
Hopefully I'm wrong -- Yahoo needs help, and she does have a good story.
Prior to joining google in 1999 as employee number 20 "Mayer worked at the UBS research lab (Ubilab) in Zurich, Switzerland, and at SRI International in Menlo Park, California" (wikipedia) for what appears to be only a few years. (She graduated high school in 1993, received a B.S. and and M.S. which must have taken at least 4 or 5 years afaik).
My guess is that she has a great deal of confidence, confidence built on the fact that she was an early google employee and she was able to ride the success of that opportunity. But her entire working career is basically google and I'm not sure she brings the depth of experience necessary to take on turning around the fortunes of Yahoo.