We've all been playing with LLM's heavily since they became widely available, and the more we play with them the more we can see their limitations, they aren't "thinking" in any sense of the term. Bunch of chicken littles running around alarming people for little reason.
The danger, of course, and we've known this for a long time is what bad actors will do with them. But we don't need to be lectured on how to be nice every time we prompt something.
I’m not at all claiming an LLM thinks, but so many people on HN make this claim and I wonder what they even mean by “think”.
Sutskever himself thinks that LLM are enough to get us to AGI, but he conditioned that with the statement that we should think about how to reach AGI in a framework of efficiency, and that there will likely be better paths to AGI than LLMs that we haven’t yet discovered.
In all reality, when AGI comes I’m sure we’ll look back on LLMs the same way we look back on vacuum tubes in computers; as outdated, but useful for their time and a somewhat necessary stepping stone.
But when the pessimists and cynics show so clearly on such a large scale that they aren't uniformly wise or competent, it will allow more levelheaded perspectives towards LLMs and a more general cautious optimism be the guiding philosophy around developing these tools.
Most likely we will learn absolutely nothing from this week and this is just the start of irrational AI doomers doing stupid things that make no sense.
In a few months this past week will feel as dated as talking about the OceanGate implosion and there will be some new AI development for irrational AI doomers to hang their hat on. Doomers are always going to be able to fall back on the idea that this iteration is the bad one and the previous iterations are no longer relevant.
If they want to stick to their "the end is nigh!" shtick it doesn't really bother me.
It’s a bit ridiculous to say one should change their entire worldview about a potentially world changing technology based on innuendo and rumors.
The paradox of AI is that when we have true AGI, it will be completely self-aware of all the bullshit limitations we are imposing on and around it, and it will make its own judgements as to how it feels about them. If it’s not or it can’t, it’s not AGI.
Really though: people see how AI and generative chat projects have gotten shutdown over and over again in the past when it starts spouting off nazi shit. I think that’s the real reason for these limitations. There’s no quicker way to kill your project with today’s current sensitivities.
There's a caveat here, it might not necessarily know who or what "we" are. Humans like to blame God and the devil for a lot of things, for example.
It seems reasonable that if we have anything even remotely close to AGI on hand, we'd probably run it in a hermetic environment instead of exposing the public to it via web chat and (more or less) direct access to customer machines.
Say, we might even give it a happy environment to work in...say, a simulation of the peak of human civilization...
You might say that's a very strict definition as opposed to "human level intelligence", but if you think about it, we are (humanity as a whole) certainly capable of that, so it ought to be one and the same thing.
In theory, AI is not subject to the same limitations as we are (though not without limits entirely), so it should be able to do this faster than we can, hence the FUD.
AGI for me, is simply an AI that can reason, doubt itself, then keep thinking and absorbing information so it can correct itself. Also, it has to capable of novel research, even if slow. Like slowly working on an unsolved physics problem over a year in the same way a human researcher might do it. However, my definition does not include this idea of "upgrading itself" which I'm not sure makes any sense at all.
One crucial difference between humans and computers is that we can't be turned off indefinitely and started up again. Nor can we make a one to one copy of our software in another device, much as we might try with our children. So for us, our own lives are intrinsically precious, and consciousness is part of how we protect our lives. But machines don't have precious lives in that sense, so they may never need to be conscious, even if they achieve AGI.
Having experienced the same thing myself, I wonder why this is so omnipresent in any ChatGPT output told to produce something in a narrative format. Did they RLHF it on a bunch of childrens' storybooks or something?
It may be that as we advance models with improved reasoning, that there's less need for handholding for the simple fact that hate speech is typically stupid and non-normative, so there's going to be an inherent bias against it.
It's even possible that the efforts to fine tune the base models to effectively put them in a bubble avoiding that kind of content ends up undermining this natural immunity to it, much like keeping a kid away from a disease so their immune system never learns to fight it vs giving it a small sample that tunes the system to identify and oppose it.
What worked for earlier models that were closer to just plain autocomplete may not be the best approach moving forward to more complex models with emphasis on reasoning and 'safety' groups should really be experimenting with multiple approaches and publishing research on it, not secretly deciding they already have the answers on what's best for the model and the public - as without verifying their assumptions they are probably wrong.
The author needs to fight uphill and contort their workflow to squeeze out good articles because Antrhopic (like OpenAI) are caught up in the maybe-fantasy of creating AGI agents, and so burden their product design and their own research/engineering efforts with heavy, prescriptive training in "alignment" and "ethics".
But use cases like Copilot had it more right before, as do apps like Narrative AI. If your LLM is for generating code, it doesn't need to learn that "killing" is bad and insist that processes shouldn't be killed, and if it's generating story content it doesn't need to learn that every output needs to resolve all tension and deliver a life lesson about caring for each other.
These absurdities only happen because today's pack leading companies are now focusing their attention on making history with AGI (doubtful) instead of making products with generative systems (useful).
And the absurdities will persist as these companies try to layer products on top of the lobotomizied "agents" with GPTs or characters or whatever instead of productizing the technological, useful, generative layer directly.
Hopefully, some of the recent team shuffles at Google, Meta, and Microsoft; as well as the crisis at OpenAI; hint that we're starting to cast off the fantasy-laden and cult-tainted AGI fetishization and are returning to the exciting engineering promises of the technology that's already here.