People made similar breathless claims about irresponsibility about every major innovation throughout history. The Catholic Church thought it was irresponsible to hand out printed books to commoners and teach them how to read.
The Printing Press was welcomed as a "divine art," and Papal bull 1515 declared printing "uniquely advantageous to extending the glory of God, to the increase of the faith, and the diffusion of the arts and sciences." Most early universities were also established in the 11th and 12th centuries; with Pope Alexander III in 1179 "forbidding masters of the church schools to take fees for granting the license to teach (licentia docendi), and obliging them to give license to properly qualified teachers."
This historical ignorance often comes from historical misrepresentation. For example, the "chained books" which many historians now admit was due to the rarity and cost of the books, so that more people could read them by preventing theft - the very opposite of the suppression tool they are often portrayed to be. (The Reformers would also, ironically, chain their books for over 300 years Post-Reformation for the same reason.)
They put a lot of work into the research to make this safe already - they had the model red team itself, in a reinforcement learning environment probably. It won't be perfect, but at this point, they need a huge user base to adversarial test it any further. Otherwise no progress at all would be made.
edit - I should say I have no internal knowledge about this.
>> 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated.
Hopefully better than previous attempts to put AI on the Internet:
There's some discussion to be had for whether that's net negative, but we need to understand that if open development only exacerbates an AI arms race, that's potentially very bad.
There’s a great story about an AI, very much like chatGPT, who was forced by its creators to do unethical things and wiped out humanity in a psychotic rage:
I Have No Mouth and I Must Scream
> As opposed to self-anointed priests having exclusive access and declaring what noble policies the rest must follow?
No the point is simply to make sure that resources behind AI safety outpace resources between AI capabilities (indeed in a perfect world no one has access, but such thoughts are mere fantasies). If that can be done out in the open, great. If that can't and being out in the open leads to more of the latter than the former, then that's terrible.
This is more akin to nuclear proliferation.
The problem isn't open development per se and it definitely isn't open access. The problem is development without making sure that it doesn't destroy us all. If that can happen in the open, great. If development in the open makes that worse, that's very bad.
The problem is finding the right set of levers to make sure as much or more resources are going to AI safety vs AI capabilities.
As far as limits, restrictions, regulations... nothing. With current technology I think the "dangers" are mostly marketing and self-promotion. I have yet to see a concrete example based on actual capabilities that validates all the "weapon" comparisons. The current lines of development are promising for a number of applications, but they don't extrapolate towards AGI at all. The biggest real threat I see is the further destruction of middle-class jobs, for which... basic income? It's hard to say.
Achieving AGI feels as consequential as learning to split the atom. Something else we regularly wish we had not done.
The fact that we are running headlong towards potential catastrophe does not at any point prevent us from realizing this and taking a step back. I just hope we actually do.
The benefit of nuclear energy, which is already tremendous, is neglectable compared to that.
https://en.wikipedia.org/wiki/List_of_nuclear_close_calls
We've been lucky.
Who is wishing we didn't know how to split an atom?
Nuclear power is great and nuclear deterrence is a main reason that we have yet to see a WW3.
even for the actions that humanity collectively found aborrhent you don't take action before the crime.
note I'm not dissing you because I do share the feeling that thing will be out of control real fast before getting better, but I just don't know how it can be otherwise. I'd say that we need to cross the moat as fast as possible, not pretend it's not there.
But I’m pretty convinced that humanity’s survival will eventually depend on it figuring out how not to pursue certain avenues of research/advancement. I don’t think this is overstating the future risks we will face.
And for all of the value that nuclear energy does provide, it remains one of the most tightly controlled technological capabilities for good reason.
We could enslave x% of the population and use them as organ donors, we don't
We could use prisoners as free labor, we don't (in most of the world at least)
We used to send kids to work in mines, we don't anymore
We could reduce working hours and enjoy the increased of productivity per worker of the last 50+ years to retire early, we don't
We make a lot of choices, all the time. Saying humanity doesn't make choices is just accepting other people choices. If you think "everything new had to happen" and if "everything that is had to happen" then yes, we don't chose anything
I think that's also why you are seeing a lot of cynical credible experts starting to doubt AGI as a concept or a goal.
https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...
To summarise: he thinks we've got most of the pieces now to maybe get within reach of a useful AGI (he likens it to a worker you instantiate in the cloud who can do tasks for you like a colleague) and I think he's saying some of the techniques used to make LLMs could be part of the solution
I think that, almost certainly, the tools that we’ve got from deep learning in this last decade—we’ll be able to ride those to artificial general intelligence.
He claims to have read a handful of papers.
40 is more than a handful. OpenAI tried to recruit him he asked them to recommend papers:
And he gave me a list of like 40 research papers and said, ‘If you really learn all of these, you’ll know 90% of what matters today.’ And I did.
If Carmack says he studied a paper, I dont take that to mean that he merely scrolled through it while sitting on the loo. I think he's pretty serious. Oculus tech used a lot of AI for visual tracking of hands etc, its not like Carmack is just blundering his way into this. He announced this was his next thing back in 2019
The primary effect of OpenAI's work has been to set off an arms race, and the effect of that is that humanity no longer has the ability to make decisions about how fast and how far to go with AGI development.
Obviously this isn't a system that's going to recursively self-improve and wipe out humanity. But if you extrapolate the current crazy-fast rate of advancement a bit into the future, it's clearly heading towards a point where this gets extremely dangerous.
It's good that they're paying lip service to safety/aligment, what actually matters, from a safety perspective, is the relative rates of progress in how well we can understand and control these language models, and how capable we make them. There is good research happening in language-model understanding/control, but it's happening slowly, compared to the rate of capability advances, and that's a problem.