http://en.wikipedia.org/wiki/The_Culture
Vinge and Stross both hint at the worst cases - and they would be pretty bad, simply having using atoms re-purposed might not look that bad by comparison:
"There is life eternal within the eater of souls. Nobody is ever forgotten or allowed to rest in peace. They populate the simulation spaces of its mind, exploring all the possible alternative endings to their life."
Worst-case scenarios are not likely for the same reasons best ones aren't - the default is AI indifference (they use you for atoms)
Since you mentioned worst, this is a must-read-classic :) http://hermiene.net/short-stories/i_have_no_mouth.html
I wish this would stop being repeated. The argument for a solar system tiling smiley face machine has been thoroughly refuted.
First, the bulk of his argument hinges on the conflation of optimization power and value assessment. Those are easily separated. If you program an AI with a given utility function, and a goal of maximising that utility function, it won't modify the utility function in the course of increasing its optimization power. The utility function is supposed to be an invariant.
Now sure, you could try and tell the AI to align its own utility function to that of the human race… whatever that means. What is a human? How do you resolve conflicting values? It is all well and good to assume the presence of checking code, but he gave no hint about how those checks might actually work. If those are mere human-crafted checks, they are bound to be circumvented eventually, as the AI increases its optimization power beyond human level.
The AI doesn't care about human values. It only cares about maximising whatever goal it has been provided with, within the constraints of the checking code.
Second, I am wary of those swarm constraint relaxation intelligences. Since it is not even clear what utility function they might be maximising (if any), I wonder what would happen if they are given the power to self-modify in any significant way. Sure, this might be a lead worth following, but the safety of this design is even murkier.
Third, I'm not sure the guys at MIRI even believe in the doctrine of logical infallibility. They are well aware that perfect Bayesian reasoning requires super-exponential computational power, and is thus impossible to perform exactly. The AI is bound to approximate the Bayesian ideal, and cannot assume its reasoning is perfect —except in the few cases where it managed to produce an actual actual proof, like in 1+1=2, or simple probability calculations with simple prior information.
Overall, all I see is weak arguments grounded in assumptions I don't share. Worth a read, though.
---
If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally unstoppable, then (iii) accomplish whatever it was programmed to accomplished.
If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…
If the goal is to answer some difficult math question, it could tile the planet with computer, destroying the ecosystem and all humans in it, so it can compute the answer, and display it on a screen, for nobody to see it. Or maybe you programmed it to tile the universe with paperclips, because that goal is easy to debug. Oops.
---
As long as we're limiting ourselves to narrow AI (ordinary programs, really), we should be okay, and most of the consequences should be manageable. But as soon as we get to genuine AGI, we must worry about intelligence explosion and value alignment, so we can avoid creating a god that want something we don't.