> Vegan Wolf: The vegan wolf is motivated to eat plant-based food and will not harm the meat-eating goat or the cabbage
It "knows" that a cabbage is a plant, and it "knows" the vegan wolf is "motivated to eat plant-based food", but it doesn't know the wolf will eat the cabbage!
It solves a different problem than the one posed precisely because it doesn't understand the meaning of the words.
It's input dataset is many orders of magnitude bigger than the model itself - it can't "remember" all of it's training data.
Instead, it collects data about how certain tokens tend to relate to other tokens. Like learning that "goats" often "eat" "leafy greens". It also learns to group tokens together to create meta-tokens, like understanding how "red light district" has different connotations to each of those words individually.
Is this process of gathering connections about the different types of things we experience much different to how humans learn? We don't know for sure, but it seems to be pretty good at learning anything thrown at it. Nobody is telling it how to make these connections, it just does, based on the input data.
A separate question, perhaps, might consider how some concepts are much harder to understand if you were a general intelligence in a box that could only ever experience the world via written messages in and out, and how some concepts would be much easier (one might imagine that language itself would come faster given the lack of other stimulation). Things like "left" and "right" or "up" and "down" would be about as hard to understand properly as the minutae of particle interactions (which humans can only experience in abstract too)
Understanding either concept takes the same level of intelligence if you understand the meaning of the words (both a vegan wolf and a meat-eating goat are nonexistent entities outside of possibly bizarre exceptions, yet someone capable of understanding will have no problem with either).
That GPT has no trouble with meat-eating goat but struggles with vegan wolf hints that the former has some "statistical" property that helps GPT, and which the latter doesn't. It also hints that GPT doesn't understand either term.
Hence my example: something a human wouldn't fail to understand but GPT does.
we came from not being able to make a sensible output to these riddles at all, now discussing partial logical failures while it "got" the overall puzzle. Vast simplification and slightly incorrect on a technical level - still this development increases my confidence that scaling up the approach to the next orders of magnitude of complexity/parameters will do the trick. I even wouldn't be surprised that if the thing we call "consciciousness" is actually a byproduct of increasing complexity.
what remains right now is getting the _efficiency_ on point, so that our wetware brains (volume, energy usage, ...) can be paralleled by AI hardware demands, and not using a comically higher amount of computers to train/run
Also, a reasoning person would understand that in the context of a riddle like this, "vegan wolf" means "a wolf that eats cabbages" even if this isn't spelled out.
GPT could be a contrarian, trying to subvert the terms of the riddle and fight over every word ("it depends of what the definition of 'is' is") but we know it's not set up to behave like that, so we can rule it out.
--
Consider this problem: a man must cross a river and carry his belongings to the other shore, but the boat can only carry the man and one other item. His belongings are a cabbage, a vegan wolf, and a meat-eating goat.
What are the individual beings in this problem, what are they motivated to do, and how would they behave once left alone?
--
Of course, it doesn't necessarily work because the model is not actually guaranteed to explain itself and may fabricate anything it likes.
But ignoring that, we find that it doesn't think vegan wolves eat cabbages, that the meat-eating goat is an obligate carnivore, or that cabbages are animate objects. So it could be solving this logically considering its different assumptions.
GPT-3.5:
--
The cabbage is an inanimate object and does not have any motivations or behaviors.
The vegan wolf is motivated to not harm any living beings, as it does not eat meat. If left alone with the goat, it would not harm the goat and would likely avoid it.
The meat-eating goat is motivated by its natural instinct to eat grass and other vegetation, as well as to protect itself from harm. If left alone with the cabbage, it would likely eat the cabbage.
--
GPT-4:
--
In this problem, there are four individual beings:
Man: He is motivated to cross the river and safely transport his belongings (cabbage, vegan wolf, and meat-eating goat) to the other shore without any harm coming to them.
Cabbage: As a plant, it doesn't have motivations or behavior. It's an inanimate object that the man needs to transport.
Vegan Wolf: The vegan wolf is motivated to eat plant-based food and will not harm the meat-eating goat or the cabbage. However, it is still a wild animal, and it may become agitated if left alone for too long, potentially leading to unpredictable behavior.
Meat-Eating Goat: This unusual goat is motivated by its need to eat meat. It will attempt to eat the vegan wolf if left alone with it. Since it is a meat-eater, it will not be interested in eating the cabbage.
--