The risks need to be discussed and understood, along with the benefits, and publicly. That's the only sensible way forward. Denying that the technology is here already and pretending it can be "paused" doesn't assist in alleviating their concerns.
It's absurd to think any of it can be put back inside the box it came out of. Now that it is here, how best to mitigate any bad sides it may have? Simple, continue to develop it - as it will be the only viable source of effective counter-measures.
If people are genuinely concerned about lack of access to the OpenAI models then work at training open ones!
OpenAI has a maybe 6 month lead and that's nothing. Plus it's much easier being the follower when you know what is possible.
(To be clear, I know at least a few of the projects already working on this. I just want to make it clear that is the intellectually honest approach).
Perplexity AI's app is definitely better than GPT 3.5 for many things although it isn't clear how they are doing everything there.
That's simply not true. Nobody would have gotten where GPT is today without transformers. That's not a trivial bit of insight anybody could have had. Stopping research funding and publications will prevent rapid evolution.
Rapid evolution is well underway. Lone individuals are able to push the envelope of what's possible even with just a new basic interop, maybe in an afternoon. It's much too late to be discussing things like moratoriums.
Maybe such things could prevent emergence when the basics don't exist yet, but not when we're all already walking around holding a capable factory in our hands and can create a new product line in a few lines of Python.
Yes, plenty of low hanging fruit around; Heck, I can probably literally ask chatGPT to implement for me a few ideas I've got.
OTOH, I've known since secondary school of two distinct ways to make a chemical weapon using only things commonly found in normal kitchens, and absolutely none of the post 9/11 aftershock attacks that got in the news over the next decade did anything remotely so simple, so that example makes me confident that even bad rules passed in haste — as many of them were and remain — can actually help.
(And that's despite my GCSE Chemistry being only grade B).
Clearly, it's a very hard problem with massive uncertainties. But we can take actions that will significantly decrease the risk of utter catastrophe.
I don't even think world-ending catastrophe is that likely. But it seems a real enough possibility that we should take it qeriously.
I'm not familiar with Gary Marcus's arguments, but perhaps there's a bit of an misinterpretation or mind-reading going on with this specific point? Not sure, but one of the first comments on the article said the following as a possible explanation.
> Gary Marcus has tried to explain this. Current AI bots are dangerous precisely because they combine LLM abilities with LLM unreliability and other LLM weaknesses.
The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark.
A lot of people, incumbents and non-incumbents, relevant experts and non-experts, are saying we need to figure out AI safety. Some have been saying it for years, other just recently, but if you want to dismiss their views, you're going to need to address their arguments, not just ad hominem dismissals.
Marcus said:
> We must demand transparency, and if we don’t get it, we must contemplate shutting these projects down.
https://garymarcus.substack.com/p/the-sparks-of-agi-or-the-e...
(While at the same time still saying "it's nothing to do with AGI")
It would be much more effective, and meaningful, to challenge their arguments.
I don't think their arguments deserve anything more than that.
Current LLMs are merely sophisticated statistical tools. There is zero evidence that they could ever be developed into something that could take intentional action on it's own, or somehow physically threaten humans.
LLMs are useful for improving human productivity, and we're going to see some ugly results when criminals and psychopaths use those tools for their own ends. But this is no different from any other tool like the printing press. It is not a valid reason to restrict research.
This is wrong. They are have the capability for in-context learning, which doesn't match most definitions of "statistical tools"
Yawn. At a minimum they are tools of which we do not understand how exactly they work internally.
A lot of people working on LLMs say that they believe there is a path to AGI. I'm very skeptical of claims that there's zero evidence in support of their views. I know some of these people and while they might be wrong, they're not stupid, grifters, malicious, or otherwise off-their-rockers.
What would you consider to be evidence that these (or some other technology) could be a path to be a serious physical threat? It's only meaningful for there to be "zero evidence" if there's something that could work as evidence. What is it?
If the people working in this field have some actual hard data then I'll be happy to take a look at it. But if all they have is an opinion then let's go with mine instead.
If you want me to take this issue seriously then show me an AGI roughly on the level of a mouse or whatever. And by AGI I mean something that can reach goals by solving complex, poorly defined problems within limited resource constraints (including time). By that measure we're not even at the insect level.
DNN RL agents can do that. Of course you'll wave it away as "not general" or "mouse is obviously better". But you won't be able to define that precisely, just the same you're not able to prove ChatGPT "doesn't really reason".
PS. Oh nevermind, I've read your other comments below.
This is incredibly ignorant and I expect better from this forum.
https://www.microsoft.com/en-us/research/publication/sparks-...
https://www.assemblyai.com/blog/emergent-abilities-of-large-...
If you'd like you can define "emergent capabilities in an AGI sense".
But that doesn't make me sanguine about them. The printing press was amazing and it required new social conceptions (copyright). Nuclear weapons did "little" other than amplify human destructive capability but required a whole lot of thought on how to deal with it, some of it very strange like MAD and the logic of building doomsday devices. We're in the middle of dealing with other problems we barely understand from the extension of communications technology that may already have gotten out of hand.
We seem like we're limited in our habits of social reflection. We seem to prefer the idea that we can so we must, and if we don't someone else will in an overarching fundamentally competitive contest. It deprives us of the ability to cooperate thoughtfully in thinking about the ends. Invention without responsibility will have suboptimal and possibly horrifying outcomes.
(* I am much less certain that there isn't some combination system or future development that could result in an autonomous self-directed AGI. LLMs alone probably not, but put an LLM in an embodied system with its own goals and sensory capacities and who knows)
Yes. This line of argument terrifies me not only because it's fatalist, but because the logical result of it is that it results in the worst possible outcomes.
It smells a lot like "we need to destroy society because if we don't do it, someone else will." Before anyone jumps on me about this, I'm not saying LLMs will destroy society, but this argument is almost always put in response to people who are arguing that they will destroy society.
Some of us look at patterns and are, understandably, cynical given some of the steps taken ( including those that effectively made OpenAI private the moment its potential payoff became somewhat evident ).
So yeah. There is money on the table and some real stakes that could be lost by the handful of recent owners. Those incumbent and non-incumbent voices are only being amplified now ( as you noted they DID exist before all this ), because it is convenient for the narrative.
They are not being dismissed. They are simply being used.
Down with these ** control freaks, so much of everything nowadays prioritizes safety over freedom. This time let's put freedom first.
where to draw the line i have no idea
I contend that doesn't matter.
There is sufficient compute available now at consumer levels to make it too late to stop training LLMs.
If cloud A100s became unavailable tomorrow it'd be awkward, but there is enough progress being made on training on lower RAM cards to show it is possible.
Slowing things down is a real effect that will impact how circumstances unfold.
Do I trust that a "slow down / pause" would be done robustly and sensibly? I wish I was more optimistic about that.
At the very least, we should definitely continue having this conversation.
That's not unique to LLMs, of course. It's what has happened before every time something has obsoleted a bunch of jobs. There's no reason to think this time would be any different.
I wouldn’t expect any regulation or laws to slow it down at this point. You might get the U.S. to crack down on it and stop innovation, GPL or proprietary license holders might win a lawsuit, etc.
But I suspect the net effect would be to push research and innovation out of the jurisdiction that made that decision and into another that isn’t willing to kill LLMs in their economy.
Personally, after seeing what LLMs can do first hand, I’d likely move jurisdictions if the U.S. cracked down on AI progress. There is a not-0 chance that putting bread on my table in 10 years requires it.
Fine-insured bounties 'fixes' this. A strong enough financial incentive can lead to bounty hunters bringing extranationals into the state where FIBs are legal to reap the rewards; further, if the fine is pegged to some % of estimated TC, the incentive scales directly with the value of the technology you are attempting to dissuade people from capitalizing upon.
(That might be useful assuming you think Yud's criticism is valid, which I don't really anymore. I think normal redistributive methods are going to be more than enough. Food for thought, though.)
This is my overriding concern. This is a wave crest that if you don't ride it, it may well crush you or take the ocean with it and leave you in a desert. It is as interesting as fire as a tool for civilization going forward.
Did you recall that your planet is currently in a sixth extinction event, brought about, yes, through the repeated overuse of fire?
Zeus was right. Prometheus belonged on that rock.
And this is exactly the risk. These things will just go other places while the US trips over its salty billionaires trying to create political red tape to allow their own businesses to catch up.
Surely Elon has no reservations about putting premature self driving cars on the road. Or are we just going to pretend that the last decade of his business failures have nothing to do with this hypocrisy? At least GPT hasn’t killed anyone yet.
An open letter full of forged signatures and conflicts of interest aren’t convincing enough for the repercussions of the government stepping in. More than anything, this just reeks of playing on peoples fear of change.
New things are scary, but in my opinion, this kind of drastic measure starts with broad interest panels that first evaluate and detail the risks. Ooo scary doesn’t cut it.
What world are you living in if you think Tesla and SpaceX have been in the past 10 years business failures?
Hard to tell, given that…
(0) newspapers apparently rename things for clicks: https://www.brusselstimes.com/belgium/430098/belgian-man-com...
(1) him being a hypocrite tells you a mistake has been made, not which of the contradictory positions is wrong.
My resolution is: GPT isn't the only AI, and Musk is pretty open about wanting to put FSD AI into those androids of his, so if the Optimus AI was as good as Musk sells it as being, it would absolutely be covered by this.
(2) how do you even compare the fatality rates here?
GPT doesn't move, and stationary Teslas aren't very dangerous, so miles-per-death seems a poor choice.
User-hours per death? Perhaps, but I don't know how many user-hours either of them have had yet.
And then (3), while I don't really trust self-promotional statistics, those are the only ones I have for Tesla, which says that switching it on increased the distance between accidents: https://www.tesla.com/VehicleSafetyReport
Better sources appreciated, if anyone has them! :)
(4) finally, as this isn't just about GPT but all AI: how much of the change in the USA's suicide rate since the release of Facebook can be attributed to the content recommendation AI that Facebook uses? What share of culpability do they and Twitter bare for the Myanmar genocide thanks to imperfections in the automation of abuse detection and removal? Did the search AI of Google and YouTube promote conspiracies, and if so how much blame do they get for the deaths proximally caused by anti-vaccination? And in reverse, over-zealous AI have seen fraud where it did not exist, and people have committed suicide as a result: https://www.theverge.com/2021/4/23/22399721/uk-post-office-s...
It's naked self-interest.
I'm not convinced that he's wrong.
[0] https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...
> When the insider conversation is about the grief of seeing your daughter lose her first tooth, and thinking she's not going to get a chance to grow up, I believe we are past the point of playing political chess about a six-month moratorium.
Gary Marcus has been aware that neural nets work for a while now, but he is only in the spotlight for his contrarian take, if he stops having a contrarian take he disappears, because it's not like he is producing any research worth of discussion otherwise. So you can expect him to stay contrarian forever. What might have been a genuine take initially is now his job, that's how he makes money, and it's so associated with him that it's probably his identity as well.
If anything twitter disincentivizes the practice because it has commoditized cynicism: It's harder to build a brand by being the well known naysayer of X when someone can source whatever negative view they want by simply typing it into a search box.
Commoditization will be complete once the media starts quoting GPT4 for reliably opposing views.
To assume that the person you disagree with can only hold their view out of cynical self-interest, wishful thinking or plain insanity is to assume that you are so obviously right that there can be no valid debate. That is a bad starting position and I'd recommend against it as a matter of principle. Apart from anything else, however convinced you are of your own rightness it's plain rude to assume everyone else is equally convinced, and ad-hominem to ascribe negative motives to those who disagree.
As for Gary Marcus, as far as I've seen he's been consistent in his views and respectful in the way he's argued them. To read about him on HN you'd think he's spent the last few years badmouthing every AI researcher around, but I haven't seen that, just disagreement with people's statements - i.e. healthy debate. I haven't followed him closely though, so if you know of any cases where he's gone beyond that and said the sorts of things about AI researchers that people routinely say about him I'd interested to see them.
Yann LeCun's name never was on the letter. Where did this meme come from.
The tweet where Yann denies that he signed the letter: https://twitter.com/ylecun/status/1640910484030255109. You can see a screenshot in the replies of the originally deleted tweet.
There was nothing in place to stop us from taking the tradeoff of transportation efficiency over the at the time unknown negative effects. But should we call the invention of the ICE and other related technologies a mistake that irreparably doomed the planet and our future generations? I have no idea. It's a tough question we were incapable of asking at the time, and might reveal some hard truths about the nature of progress and the price we have to pay for it. At least with the effects known at this point, we can start asking it, and it's a step up from obliviousness.
It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded.
We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it).
I'm so offended by the idea of this that I'll personally fund $10k of training runs myself in a jurisdiction where it isn't banned if this ever became law in the US.