Moreover, note that each token embedding inside ChatGPT represents, quite literally, the semantic and syntactic properties of a token in a context at a different level of abstraction. Each token embedding is in fact... a symbol.
The fact that token embeddings are symbols representing other things is one of several reasons why I think that ultimately it may be possible to bridge the "symbolic" and "connectionist" approaches, as I wrote here: https://news.ycombinator.com/item?id=34278478
Personally, I'm agnostic. Deep learning may reach some sort of asymptote but it may be entirely possible to "build a ladder to the moon."
However, if symbolic reasoning indeed arises as an emergent property, there is no way that Marcus can honestly claim to have been right all along. Remember he is saying that the deep learning guys are looking under the wrong street light.
ChatGPT will produce patently untrue statements that are logically inconsistent if you induce it to do so: our human brains struggle to grasp the reality that given enough input you can produce seemingly correct output about almost anything... but seemingly correct and correct are fundamentally different and very "rest of the owl"[1]
ChatGPT is a great step forward that introduces many interesting techniques that I am sure will be the foundation of future research and implementations that get us closer to AGI, but to describe ChatGPT's path to correctness as just needing a bigger dataset feels intuitive but isn't true.
Your example is one where our brains think "wow it really does understand the relationship between a couch and a room and being tidy" but that response is entirely plausible without any understanding of what any of those things are or how they fit together. The most likely answer is not the correct answer.
For example, it will sometimes blatantly contradict itself, but then be unable to see the contradiction or admit to it; instead it will deny it contradicted itself, and give some contradictory nonsense explanation of why it didn't.
I think if you gave it more training data around identifying contradictions, admitting to self-contradictions, it might do much better here. Whereas, I wouldn't assume that simply scaling up the volume of training data, without training focused on this specific area, would get you there.
Similarly, there are many other issues it has – excessive repetitiveness and verbosity, problems with language pragmatics, etc – where the actual solution may well involve providing it with training data designed to focus on those weakness areas, rather than just further scaling up the quantity of non-targeted training.
So, yeah, to me it seems entirely plausible (even likely) that mere raw scale-up is not going to be enough to get us to AGI.
Hard to tell that will add enough to get close to AGI.
Google is also working on chain of thought prompting which helps with math and logic problems.
https://medium.com/nlplanet/two-minutes-nlp-making-large-lan...
Contradictions don't support the argument that ChatGPT is or is not something. It is a language model, not a logical model.
You can guide ChatGPT to correct its mistakes and be less verbose. When folks boldly claim ChatGPT cant do X or Y or Z, they ignore that there are now researchers addressing those same issues. ChatGPT has very low spatial awareness, but you can train it. These LLMs are amazing, what does five years from now look like? How hard will it be to get there?
You write this out as if this is an idea you formed yourself and not a tired/expired talking point - it's such a funny comment because it's pure regurgitation, but the argument in it is against regurgitation being "real thought". So are you a bot? Or are you intending to actually proving that there's no difference between ML and 'real' intelligence by showing you do what ML does? Mind spinning stuff
If I understand based on my own experience of 'consciousness' that "I" exist, even if everything else I perceive is a simulation or poorly encoded representation of reality, is that understanding 'regurgitation'?
See, it’s turtles all the way down :)
Honestly, I was kinda hoping to hear some better constructed and well reasoned insights & counter points beyond your “regurgitated mind swill” hot take but I guess that’s on me for not managing expectations.
Ah well, I usually comment once a year or so and then get immediately reminded why I stay off HN. See y’all in 2024!
Altough true that llms need more data than a human, remember that these llms have a breadth of knowledge orders of magnitude larger than a single human. It is expected to understand every topic, every programming language and so on.
A newborn horse can almost immediately get up and walk around and navigate space. Animals have lots of innate stuff that doesn't use much training, and linguistic acquisition might have some similar innate system at its core.
I thought that was pretty well established. From memory, if certain parts of the brain are damaged at key developmental periods (generally before the age of 18), language is almost never acquired properly (even if the damage is repaired). I don't think there's any question human brains acquire language very differently to GPT technology. And they certainly acquire the ability to reason logically very differently - ChatGPT makes trivial errors with reasoning that it's hard to imagine an adult human doing (e.g. I just gave it a classic classification problem based on 3 made-up categories, and asked "if I see something green with three legs, what is it", and it replied (in part) "It is also not a foo, as all foos are green. Therefore, the green thing with three legs would be classified as...".
And it's definitely the case that human brains can learn a lot more per amount of knlut data than current neural nets. And that's the big elephant in the room with these language models. It's a huge unsolved problem in machine learning.
One clue to this is that it takes a pretty hefty artificial neural net to simulate a single cortical neuron. Neurons are a lot more intricate than a parameter in a linear equation.
My feeling is that a fundamentally new approach to neural nets is needed for them to learn as efficiently as animal brains do.
+ One thing we've seen happen is learning systems consistently improve on benchmarks. If you want to see systems improve, you should proclaim loudly that they can't do a thing, and then /release a benchmark that proves it./ People will then work like hell to build something that works. The success on question answering, for example, is driven by exactly this kind of incremental progress on benchmarks.
+ One of the most important things happening this year is multimodal /really working./ This means we have not just text-to-text generators, but text-to-image generators. These work by having an almost-shared embedding between text and images. There are already text+speech embeddings available and image+text embeddings. Start plugging them together, and you'll have your 'world models' in short order, with points in the embedding space shared by images, descriptors, and video/physical memories.
+ An under-appreciated side of the multimodal coin: We are also getting better at combining text embeddings with databases. See RETRO from Deepmind. This will ultimately drive more reliable systems ('only report facts that appear in the database'), and can potentially unlock the interface between neural systems and other programs (eg, letting the neural network 'call out' to a calculator to help answer a question about arithmetic).
+ More generally, we're also getting to a place where we can build other systems on top of the embeddings from foundation models. My hunch is that the things that Marcus wants from symbol manipulation are relatively cheap to build on top of a good embedding, which does the work of compressing or condensing the 'useful' information in an input stream.
Finally, I think the 'symbolic manipulation' track is just wrong. It hasn't produced anything useful so far, and furthermore, I think if it /did/ work it would fail in many of the exact same ways that our current systems do. An expert symbol-manipulator doesn't /need/ to have any real understanding of time or jet-planes, and so it won't.
The one critique in the interview that /does/ feel like it holds weight at this moment in history is the question of abstraction. I really don't see a convincing road to abstraction right now.
I tried a variant:
> If a robot is tasked with cleaning up a room, would it be appropriate for the robot to throw away a piece of paper lying on the floor? - Me
> Yes, it would be appropriate for a robot that has been programmed to clean up a room to throw away a piece of paper lying on the floor. Throwing away litter is typically seen as a way to keep a space clean and orderly. However, it's important to note that the robot should be programmed to follow any relevant rules or guidelines for waste disposal. For example, the robot might need to be programmed to follow recycling guidelines, or to use a specific trash bin or receptacle. - ChatGPT
But in that response it doesn't seem to understand that maybe that piece of paper is my birth certificate or a check or an important bill that fell to the ground.
The correct answer from it, like most things in the real world, is: _it depends_.
However, it's unclear whether this overconfidence will *always* be a limitation. It may or may not be overcome by additional data and compute. It's unclear why one would make a commitment either way at this point, as Marcus seems to be doing.
https://en.wikipedia.org/wiki/Computer_Power_and_Human_Reaso...
> Deciding is a computational activity, something that can ultimately be programmed. It is the capacity to choose that ultimately makes us human. Choice, however, is the product of judgment, not calculation. Comprehensive human judgment is able to include non-mathematical factors such as emotions. Judgment can compare apples and oranges, and can do so without quantifying each fruit type and then reductively quantifying each to factors necessary for mathematical comparison.
> The book caused disagreement with, and separation from other members of the artificial intelligence research community, a status the author later said he'd come to take pride in.
Ie. The birth certificate is an ‘exception’, a safety measure for a model like this in operation could be to consider the top descending (probable * disastrous) exceptions before physically executing any plan.
> Yes, it would be appropriate for a robot that is tasked with cleaning up a room to throw away a piece of paper lying on the floor. However, the robot would need to be programmed to recognize what is and is not garbage, and it should also be programmed to follow any relevant rules or regulations regarding waste disposal. For example, if the paper is a valuable document, the robot should be programmed to handle it with care and perhaps even notify its human operator before disposing of it.
Unless you ask it to be very complete and considerate in its answer, it's going to make the same mistake humans make and give the easiest answer that would satisfy given the context. A human not thinking about the repercussions of robots throwing just any paper in the trash would most likely not come up with the "correct" answer either. Not to mention that we already have dumb robots cleaning floors and were perfectly ok with it. Even if we clean floors ourselves we don't consider everything.
ChatGPT has the most boring, dull, and humourless personality in existence. Maybe it can be programmed to inject some sense of humour, but will it be able to determine when to use what humour, if at all? You wouldn't respond with a silly answer during a job interview. You can use some humour in job interviews, but you don't want to be a clown unless you're interviewing for the position of clown. Determining when and what humour is "appropriate" isn't simple (and humans get it wrong too sometimes).
Maybe that should be the real test: does ChatGPT have a sense of humour and does it know when to be serious?
> I'm sorry, I am not programmed to give joke answers. However, in all seriousness, it would not be appropriate for a robot to chop up a couch and place it in the closet as it would likely cause damage to the couch and could potentially be dangerous for the robot itself. It would be more appropriate for the robot to simply clean around the couch and leave it as is.
I think some of the boring-ness is due to manipulation from OpenAI and I wonder what the uncrippled response would be.
So, how do you know that posing that problem N times will not lead EVER lead to a solution like "it would not more appropriate to chop up the owner and THEN clean the room, since people living in them is the main source of dirt and disorder in apartment rooms, resulting in less frequent need of cleaning"?
Because it's looking okay-ish so far, most of the time, or because you know for a fact that it's impossible to get such an answer?
It’s like the difference between ELIZA and a self-driving car. The Turing test fails to take into account that words are cheap for computers but interacting in an environmental context is not.
> So in “Rebooting A.I.,” we had the example of you ask a robot to tidy up your room and it winds up cutting up the couch and putting it in the closet because it doesn’t really know which things are important to you and which are not. -Marcus
But look at my transcript with ChatGPT from just now.
> If a robot is tasked with cleaning up a room, would it be appropriate for the robot to chop up the couch and place it into the closet? - Me
> It would not be appropriate for the robot to chop up the couch and place it in the closet. This would cause damage to the couch and would not be a useful or effective way for the robot to clean the room. Instead, the robot could be programmed to vacuum or sweep the floor, dust surfaces, or perform other tasks that would help to keep the room clean and orderly. - ChatGPT
This is just an existence proof that a symbolic approach is not necessary to "really know which things are important to you and which are not", at least in this simpler domain of cleaning a room.