Spot on comment. But... are you sure "Intelligence" is a fitting description? "Artificial" for sure.
I like the part about producing outcomes that can't readily be ascribed to individual moral choices. It seems this describes human culture in general. So here we all are. Semi-autonomous little components of the larger algorithm called "civilization" which we cannot truly hope to transcend.
Wow! That's a nice paragraph that highlights very accurately why many of us feel 'alienated' in our society.
In other cultures and times one might not even consider this description and instead describe us as useful organs in a greater body, or valuable organisms in a greater ecosystem. Which 'feels' very different.
I think you've nailed it with the second part of your comment.
Things like the military bureaucracy or the financial system can be hostile to us because their incentives are very much out of line with humanistic goals.
I do think it's legitimate to be at least a little concerned about future even more powerful and non-human AIs being hostile in the same way. What happens when/if there are corporations out there run by alien computer minds that seek goals that have no overlap at all with human beings? Would an AI care about climate change? Would it care about the health of the world's biological ecosystem? Would it care about providing a decent life for its employees? It could -- out of mere indifference -- seem quite sociopathic and evil to any humans that live under its influence. Imagine a super-intelligent mind whose goal function is maximizing short term quarterly shareholder value. This isn't just a programmed-in goal function either... this is actually a form of intelligence whose embodiment is the corporation, so it's a survival and fertility imperative. In a sense you can't blame the thing, but you could blame its creators.
Humans can of course do the same, but even with sociopathic humans there is some sense of common overlapping interest. At the very least their motives are comprehensible. AIs may have motives that seem almost "Lovecraftian" to us-- completely alien and bizarre. On top of that biological short term shareholder value imperative, try layering an Aspergers-like obsession about certain patterns in information, or some kind of bizarre AI-conceived religious mission, etc. Extremely intelligent humans can be religious fundamentalists, so why couldn't similar forms of "functional madness" exist among non-human intelligences? One of the pollyanna assumptions of the singularity crowd is that AI would necessarily be rational. Why?
Edit:
I cannot recommend this book enough. It's probably the best and most overlooked work of SF in the past 20 years:
http://www.amazon.com/Blindsight-Peter-Watts-ebook/dp/B003K1...
It deals very intelligently with this kind of thing. It's basically a monster story where the monster is something out of Ph.D level evolutionary ecology-- and it manages to actually be scary. Very intellectually satisfying "reveal." :)
I think what it ultimately comes down to is this: we do not live in a post-scarcity world, and we're not going to for the foreseeable future. Given that our existence involves a certain amount of unavoidable haggling over resources -- and given that our own labor is itself a resource -- we should be a little concerned about what sorts of beings we might end up having to haggle with. This also applies to human genetic engineering and augmentation, since that could also produce essentially alien intelligences.
I'm not against researching these topics. In fact I'd almost call myself a transhumanist / posthumanist in sentiment. But I do think it's advisable to give the matter some serious thought. Bad things can and do happen. We want to try to create positive outcomes, not blunder into the future.
It doesn't even have to be automated, but it increasingly is. It ends up producing harmful outcomes that can't be readily ascribed to the moral choices of individual humans within it.
Or consider that hostile is a transitive verb: hostile towards whom? The military already have automated systems connected to fire control without a human in the loop, although at the moment these are limited to missile defence systems. Consider the possibility of plugging the NSA's metadata based system for calling people terrorists into an automated airstrike or assassination system.