I see your comment as a strange mix of optimism about humanity's creative abilities and fatalism about the end of the world coming soon. Not like this is news, but the end of world has always been right around the corner - most everything is broken here and death is everywhere.
The strangest thing to me about comments like this is the assumption that a nefarious super-intelligence would stop at subjugating us. If a super-intelligence was evil and had power to act, it would just kill us all and sterilize the planet, given the chance.
I just don't believe we have the ability to make a standalone computer super-intelligence - we universally suck at coding at-scale. Don't get me wrong, the AI and machine learning algorithms we have are amazing, but we're not going to get a being out of those experiments.
What I'm worried about is humans: humans continuing to turn tech against other humans for surveillance and telemetry in support of totalitarianism. And, I'm worried about humans augmenting their brains with compute. If there ever will be an evil super-intelligence, I wager it will be a (meta-)man.
There's a very large and vocal group on HN that believes in the advent of AI and regularly comments as is it was a done deal. Almost every robot/AI/ML thread gets hijacked with this idea/meme.
In the meantime Google can't get my first language right, the windows 7/8/10 search is a joke and Facebook wants me to be friends with my SO's exes. And 2017's revolution ? Dial menus in messenger.
I suppose when all you have is a hammer, everything looks like a nail.
I was so convinced Y2K was going to crash society. I was all looted up, ready to bug out when the city started burning. Turns out, there was enough preparation (and money dumped on the problem), and like only one power station that didn't get the upgrades on the east coast went down for 15 minutes.
I'm just as guilty of telling ghost stories and contributing to the disinformation. I remember writing a frantic, doomsday prediction email to my house rep in the lead up to Y2K and I swear based on some of his public comments that I was responsible for him quitting the house and heading for early retirement. I have no proof that was the case, but I'm still not proud of what I wrote.
After all we are building machines in our own image. From how AI thinks to building sex robots.
I think the transition will be more nuanced and peaceful.
I predict that humans will always be vitally important for our dexterity at the 1m scale. We can (and do) use tools for smaller scales and larger scales, but we are truly excellent entities at this scale.
In a world with strong AI, old-school humans would be pathetic. Hybrids, however, could build synergistically on the strengths of all components.
So why not just integrate all those machines better?
This is the good outcome.
In a post-intelligence world, humans will be cute zoo animals. We might have freedom, but I can't imagine us having any measure of real control. Even if modern society continues, it will be a cordoned off bubble, a "human preserve".
If we're unruly and give the robots too much trouble, they could just dispense with us. Or dumb us down and make us tame--domesticate us, like animals.