It's not actually clear what "the singularity" is? Is it something running out of control or it's still controllable? There is a blurry line. People are afraid because they think it's sort of uncontrollable explosion.
The second question is about AGI. What is it? Is it something 'alive' or just a generic AI calculator with no 'creature' features. Like self preservation at least.
I think our view of these two things will change soon as we get a close up picture. Pretty much like Turing test doesn't look great anymore. As even dumb chatbots can pass.
But I realize my definition is in the minority. :/
Of course if we ever manage to make a 1:1 cybernetic brain that works exactly like a human's brain, and is also a complete black box, we'll have achieved AGI. I'm not sure how useful that will be, but I'll have to admit it is AGI.
So maybe I should say, "interesting AGI" is technology that can improve itself exponentially. :-D
But even if you had a system, there's still the resource cost to run the algorithm (needs to be bounded or else you've just made a finite jump) and the gains you make need to not decay (or again you've just made a finite jump).
And all this needs to be compared against investing in humans - which seem pretty clearly to have AGI properties (but with some really bad constant factors. 20 year training time - ridiculous!)
To me it seems things are a long way off and least a couple of major innovations away. But at least there's some ideas of the problems to tackle, which is a big step up!
This is close to singularity. Except 'does' instead of 'can'. A big difference ;)
Probably we need several AGI terms. Because sub-human robot capable of doing many not pre-programmed thing is sort of it. Still not smart enough to improve itself.
Actually most humans, the smartest creatures, cannot improve even current AI. Demand for self improvement will put its IQ in top 0.01% of all known intelligent creatures. Which is probably too much for just AGI, we may not recognize it when it will be already here. And there is another question. With such IQ do we really want to keep it slave forever?
Have you thought this through though? I mean you'd first have to know in what way to improve, which would become more challenging as you became more perfect, you'd have to want to become perfect (which let's be honest might be boring) and then I guess there's the fact that if you kept evolving / improving yourself exponentially, then you'd no longer exist because you'd likely be morphing into other forms all the time?
In a way, maybe the only thing I can think of that is observably doing something like this is the universe itself.
This reasoning, taken to its logical extent, would mean completely defunding all medical research and instead redeploying all research to best utilizing our current technology.
That is, hopefully, an obviously faulty decision.
We must balance research with using current findings to help people. Stopping all drug development and telling people with currently untreatable diseases "haha, too bad, we aren't even going to try", is cruel.
The ultimate goal of many technology utopians is the end of all death. We have examples from the natural world of incredibly long lived higher life forms, and simple life forms that are immortal, so it isn't scientifically impossible. The number of lives saved is literally, infinite. It is really hard to argue with "infinite upside".
(The absolute societal shit show that such a technology would bring about is a rather important question...)
> "we're killing more people by delaying time inventing the hypothetical life saving technology" is not very useful either, because you can't actually say how many lives would be saved versus harmed.
For many future inventions, of course estimates can be made of the number of lives saved.
The more pie in the sky research is, the larger those error bars become. "LLMs may one day help us sift through research and cure all forms of cancer" is, technically true, but the error bars on it are so high that you don't want to devote all the world's resources to LLMs just on the hopes of eliminating all forms of cancer 50+ years from now.
"This company is working on AI Vision+Robotics technology so people don't have to do this super hazardous job that claims a bunch of lives. They are an estimated 5 years from productizing and they already have purchase contracts in place" is a much different statement.
And I disagree that "most of their assumptions go unchallenged". I see challenges to the assumptions of lesswrong, e/acc, and other such stuff often here. (Maybe less often than is warranted, but still fairly often.)
"It is the mark of an educated mind to be able to entertain a thought without accepting it." - often attributed to Aristotle, but apparently not actually from him. Still a good thought. We can post about this stuff, and discuss it, without buying it.
I agree with your pushbacks.
This is akin to falling for TikTok/etc. ragebait and trying to have a discourse in the comment’s section.
The subcultures allow for the exchange of material and non-material goods — primarily money and “woo.” Everything else is just a tool to that end.
10. I can’t think of a better form of “effective altruism” than paying for LessWrong members’ vasectomies. Not because I’m concerned about their genes. They’d just be terrible parents, and gods those kids would need lots of therapy.
And e/acc is more of a little club than it is a cult. But I guess you could also consider it to be a criminal gang very much not unlike the Crips and the Bloods. Why? Because they're part of a counter-culture and naturally they want to diverge from the mainstream thought process. And the criminal activity e/acc engages in is non-compliant computer engineering. Which the Cathedral (pardon my language) dislikes.
Basically, this is evolution in action. You're looking at the beginnings of a new species. Millions of new species are coming, really.
I'd like to make some modest assertions, to push back on fringe ideas I've seen repeated here:
1. For "the singularity" to happen, we probably need something more to happen than just chatGPT to ingest more data or use more processing power.
2. Even if, somehow, chatGPT turned into skynet, we'd hopefully be able to unplug the computer.
3. If you want to save lives, it's probably more useful to think about lives saved today than hypothetical lives you could save 100 years from now. Not that you shouldn't consider the long term, but your ability to predict the future gets worse and worse the farther you project out.
4. If you want to save lives, it's probably more useful to save actual lives, than say, hypothetical simulated lives that exist inside of a computer.
5. The argument that "we're killing more people by delaying time inventing the hypothetical life saving technology" is not very useful either, because you can't actually say how many lives would be saved versus harmed. And mostly it just sounds like a front for "give me more money and less regulations".
6. Reading a bunch of science fiction and getting into fights on an internet forum is not a substitute for education and experience. Unless you've spent a good time studying viruses, you are not a virologist, and while the consensus among virologists can be wrong, you should have the intellectual humility to realize that you are probably not equipped to tell, unless you have expertise in the field.
7. Anything that smacks of eugenics is most likely pseudoscience.
8. If someone talks like a racist / sexist / nazi, or acts like a racist / sexist / nazi, they probably are one. It's probably not a joke, or a test.