Marcus said:
> We must demand transparency, and if we don’t get it, we must contemplate shutting these projects down.
https://garymarcus.substack.com/p/the-sparks-of-agi-or-the-e...
(While at the same time still saying "it's nothing to do with AGI")
It would be much more effective, and meaningful, to challenge their arguments.
I don't think their arguments deserve anything more than that.
Current LLMs are merely sophisticated statistical tools. There is zero evidence that they could ever be developed into something that could take intentional action on it's own, or somehow physically threaten humans.
LLMs are useful for improving human productivity, and we're going to see some ugly results when criminals and psychopaths use those tools for their own ends. But this is no different from any other tool like the printing press. It is not a valid reason to restrict research.
This is wrong. They are have the capability for in-context learning, which doesn't match most definitions of "statistical tools"
Yawn. At a minimum they are tools of which we do not understand how exactly they work internally.
A lot of people working on LLMs say that they believe there is a path to AGI. I'm very skeptical of claims that there's zero evidence in support of their views. I know some of these people and while they might be wrong, they're not stupid, grifters, malicious, or otherwise off-their-rockers.
What would you consider to be evidence that these (or some other technology) could be a path to be a serious physical threat? It's only meaningful for there to be "zero evidence" if there's something that could work as evidence. What is it?
If the people working in this field have some actual hard data then I'll be happy to take a look at it. But if all they have is an opinion then let's go with mine instead.
If you want me to take this issue seriously then show me an AGI roughly on the level of a mouse or whatever. And by AGI I mean something that can reach goals by solving complex, poorly defined problems within limited resource constraints (including time). By that measure we're not even at the insect level.
DNN RL agents can do that. Of course you'll wave it away as "not general" or "mouse is obviously better". But you won't be able to define that precisely, just the same you're not able to prove ChatGPT "doesn't really reason".
PS. Oh nevermind, I've read your other comments below.
This is incredibly ignorant and I expect better from this forum.
https://www.microsoft.com/en-us/research/publication/sparks-...
https://www.assemblyai.com/blog/emergent-abilities-of-large-...
If you'd like you can define "emergent capabilities in an AGI sense".
But that doesn't make me sanguine about them. The printing press was amazing and it required new social conceptions (copyright). Nuclear weapons did "little" other than amplify human destructive capability but required a whole lot of thought on how to deal with it, some of it very strange like MAD and the logic of building doomsday devices. We're in the middle of dealing with other problems we barely understand from the extension of communications technology that may already have gotten out of hand.
We seem like we're limited in our habits of social reflection. We seem to prefer the idea that we can so we must, and if we don't someone else will in an overarching fundamentally competitive contest. It deprives us of the ability to cooperate thoughtfully in thinking about the ends. Invention without responsibility will have suboptimal and possibly horrifying outcomes.
(* I am much less certain that there isn't some combination system or future development that could result in an autonomous self-directed AGI. LLMs alone probably not, but put an LLM in an embodied system with its own goals and sensory capacities and who knows)
Yes. This line of argument terrifies me not only because it's fatalist, but because the logical result of it is that it results in the worst possible outcomes.
It smells a lot like "we need to destroy society because if we don't do it, someone else will." Before anyone jumps on me about this, I'm not saying LLMs will destroy society, but this argument is almost always put in response to people who are arguing that they will destroy society.
Some of us look at patterns and are, understandably, cynical given some of the steps taken ( including those that effectively made OpenAI private the moment its potential payoff became somewhat evident ).
So yeah. There is money on the table and some real stakes that could be lost by the handful of recent owners. Those incumbent and non-incumbent voices are only being amplified now ( as you noted they DID exist before all this ), because it is convenient for the narrative.
They are not being dismissed. They are simply being used.
The letter was signed by a lot of people, including many AI researchers, including people working on LLMs. Any dismissal that reduces to [one incumbent interest] opposes this" is missing the mark.
A lot of people, incumbents and non-incumbents, relevant experts and non-experts, are saying we need to figure out AI safety. Some have been saying it for years, other just recently, but if you want to dismiss their views, you're going to need to address their arguments, not just ad hominem dismissals.