"If you would have asked people in the 70s if the most influential software of the coming decades would be written by lone wolves in their garages or a group of very smart people with a lot of resources, they would have bet on the latter. But the smart money turned out to be on Gates, Jobs, Wozniak and countless others."
This is a myth. Gates, Jobs, and Wozniak were all smart people (Harvard, Reed, genius) with a lot of resources (access to mainframe as a student, access to Silicon Valley infrastructure and talent, which emerged originally around military contracts and DARPA grants.)
> I know that if I'm struck with sudden inspiration for a new approach to the problem I'm not going to ask the government for permission. I'm going to spin up 100 cloud computing instances and see if I'm right or not. And if I am, even god won't be able to help us if sama's worst case scenarios come to pass.
Taken at face value, this sentence just seems to indict your moral judgement -- why would you spin up 100 instances to see if you're correct if you realize the existential threat in doing so?
I totally agree with this guy. Regulation is futile, you can't regulate at a micro level in all the parts of the world. If you can't properly regulate drug trafficking in Santa Monica how do you plan to regulate a couple of scientists hidden in the soviet tundra?
IMHO it's inevitable our encounter with an AI, even if that AI is a superhuman with augmented capabilities, or derives from an arms race (like the Manhattan Project).. So I already posted a possible solution: wait for it with EMPs in place... (Just in case)
There are people who believe that high level machine intelligence will take massive computing resources. It may, or may not. Most of the recent big advances in traditional machine learning were implemented on easy to obtain personal hardware. I hate to invoke buzz words, but even the much hyped deep learning was first developed and proven by Hinton on commodity machines. Other advances, like IBM's Watson, required 1940s style buildings full of servers. Hardware/power usage seems like one of the only reliable ways to tell who is doing any kind of serious AGI/SMI research and that looks like another 50/50 at best. It just doesn't have the resource and logistical signature of other kinds of research.
AGI/SMI is still so theoretical that it can be hard to see as a clear and present danger. But if it is a real threat, it's the stuff of nightmares because there isn't anything we can do. I know people who are working on it right now. It doesn't seem like they're making a lot of progress, but they are trying and doing so completely outside the reach of any regulatory framework. I know that if I'm struck with sudden inspiration for a new approach to the problem I'm not going to ask the government for permission. I'm going to spin up 100 cloud computing instances and see if I'm right or not. And if I am, even god won't be able to help us if sama's worst case scenarios come to pass.
I wrote more on this topic in a comment yesterday: https://news.ycombinator.com/item?id=9130671