This is very true, more true than we realize. Notice how much more "could you repeat that?" we have with masks on. It's not JUST the mild muffling of the speaker's voice, it's not seeing their lips move. We're all lip readers to a small degree, and it helps inform our decoding to see the lips. Fff and th sound similar but look very different.
Even without that, think if your life, ever said "Waht was that? Oh, right..." then reply. At first you missed part of what they said (for various reasons), but you were able to "interpolate" the missing part from the context, and most of the time you get it right.
Our communication modes are lossy, and our brains make up for that to a large degree. That's the hole in natural language decoding, figuring out the hinting needed for an engine, because we're not totally aware of how we do it ourselves.
I suspect the parent's point is that the disambiguation in both cases might be addressed with information encoded in the other. One can pattern match based on the context. I think some of the work in multimodal transformer models demonstrates this.
Perhaps thinking language is a key aspect of consciousness is wrong.
Chomsky has said human languages themselves are just random sounds we’ve been polishing the meaning of.
So yeah I have no doubt a machine can sort them correctly if we tell it to. They mean what we want and see in them.
That’s hardly proving anything about consciousness. Just that a computer can sort and count syntax, and guess at meaning by frequency of relative placement.
It’s acting on English data sets. Not human biology.
And I don't think "sorting" has anything to do with language.
I make sound snapping my fingers. Whistling.
Why is it so hard to accept mirror neurons fired when early humans heard birds and animals, each other’s grunts and over time we refined it?
Everyone has some capacity to refine and strengthen muscle. Why do we need some abstract meta-construct to explain where language comes from? It comes from us. Fleshy meat bags that mutate state over a short period then die.
Chomsky diagrams, conceptual organs and the like are useful for “being on the same page” in a particular context, but there’s no reason to believe language is a requirement for consciousness except our own propensity for romanticizing our existing.
Before neuroscience and modern imaging we needed to sit and imagine and theorize.
Now we don’t.
“Meaning to us” is subjective. That’s how we have conflicting theories in many fields. Science isn’t about meaning. It’s about measuring how matter coalesces at various speeds relative to light.
I make noise because my biology “just has” properties to allow it given the other physical conditions.
Perfectly elegant theory based on the physical structure of reality alone. No ephemeral language organs.
Theory can quickly go from scientific observation to reinforced nonsense used to sell books and bond as species.
That’s fine, it’s how society works. It doesn’t mean anything to reality.
Can they write Shakespeare?
If human language is fundamental, how is it missing from the start? Are we learning language or muscle mechanics? Why can a word or phrase mean one thing in this country and nonsense in another? If language processing has a universal basis, why all the confusing variety and ignorant ideas? 1+1=2 everywhere because we can observe the physical process everywhere, because light, eyes, etc
Who cares?
Like I said back in the day we didn’t know that. We had to theorize these abstract schemes to establish something.
But like religion, doesn’t mean linguistics is building on something that means what we want it to.
Given how hard it is to learn language while arithmetic seems innate, how do we know emphasizing it’s value is leading us where we want?
It’s easier for me to see that long dead scientifically illiterate humans we inherited those ideas from were just stupid.
To use an analogy, a computer chip provides a single instruction set at the machine language level--yet you can run a huge variety of computer languages on it: FORTRAN, LISP, Python, Prolog and so forth.
Chomsky's idea goes deeper than "just random sounds," given his theory of universal grammar.
There’s been an explosion in biological science since Chomsky became prominent. He even admits his work is abstract and to serve his ends. We may be putting more into his ideas due to his fame and infamy than we should.
Trial and error, social constraints on sounds and the mechanics of biology can explain it elegantly. What do the ornate theories and ephemeral organs provide except to satisfy “linguists” biological agency to create and imagine?
Linguistics came about in a much less scientifically aware era in human history. My money is on it going the way of religion; an abstraction that’s so orthogonal to advancing science we leave it behind.
The actual problem they're identifying here is that of deciding how or whether to turn 'a' into a quantifier. Do we have 'any' would be less ambiguous, definitely an existential quantifier. Do we have 'a' depends on context, as it could mean one specific person and not be a quantifier at all, but I actually still disagree with the author here. I think they're right that English-speaking humans will unambiguously understand 'a' in this context to be a quantifier, but given that is possible at all, I don't see why a large language model can't. If it was actually being used to specify one person, the additional context would be in the sentence itself, as in 'do we have a reporter named X' or something like that. Given there is no additional qualification in the sentence, I'm pretty sure existing large language models can easily figure out what it means, and the only case in which they'd be wrong is if there were additional context outside of the sentence, but a human would be wrong too if all they heard was the sentence.
> Even humans are only probably approximately correct.
Fair point. But how about this:
It's true that what John would do with the sentence is technically an "approximation" of what Alice would do because they have slightly different understandings of correct behavior. However, for humans to do what they do, they still do build an absolutely correct model of meaning in their mind wrt their (subjective) notion of correctness.
This may sound like an obtuse play with words but the point is that to even attempt to do the right kind of reasoning in NLU, you need a different framework than PAC. You can't for example approximate whether "during the Cold War" qualifies "was based in" or qualifies "an Eastern European country". You just need to decide. And once you decide, you have an absolute correct interpretation, not an approximate one.
EDIT: wording.
This has nothing to do with the mental model. The language model doesn't have to spit out probabilities. It can use a simple thresholding function and spit out certainties. They just might be wrong, but a human's "certain" mental models might be wrong, too.
One major flaw of current PAC-based models is the inability to express certainty and being able to present the rationale behind their (lack of) confidence.
One crucial aspect of human language processing is the ability to question or interrogate oneself to adjust one's interpretation. During a dialog this can be done by actively asking for more context or confirmation.
So unlike current ML models, humans know that their interpretation/models might be wrong and can actively seek confirmation or correction.
A trivial example would be asking a digital assistant "What's the capital of Georgia?" There are two possible answers and there's no way of knowing what the "correct" one is without further context. So a human would likely either ask whether you mean the country of Georgia or the US state, or qualify their answer ("the capital of the country/the state capital is Tbilisi/Atlanta").
This is what GPT-J-6B returns:
Q: What's the capital of Georgia?
A: Atlanta
And it's not as if the model doesn't "know":
Q: What is Tbilisi?
A: Tbilisi is the capital of Georgia.
Q: If Albert is a pig and all pigs are blue, what colour is Albert?
A: Albert is a blue pig.
Q: If Albert is a pig and all pigs are blue, is Albert happy?
A: No.
Q: If Alice is a dog and Bob just finished high school, can they have kids together?
A: Yes, but they will have to get married first.
But if the model doesn't understand language, why did it do so well on the logic test? In order to find out, I just asked :)
Q: If Albert is a pig and all pigs are blue, why is Albert not happy?
A: Because he is a pig.
Q: If Albert is blue, is Albert unhappy?
A: No, Albert is not unhappy.
Q: If Albert is a pig, is Albert unhappy?
A: No, Albert is not unhappy. Albert is a pig.
EDIT: in case you want to test this yourself (you might get different results, it's probabilistic after all):
When using the Q/A-format, make sure not add a space after the "A:" prompt.
–––––––––––––––––––––––––––––––––
[This is a question and answer series intended to demonstrate the logical reasoning capabilities of an AI system. The AI can understand ambiguity and will ask for more detail in such cases.
Q: What's the capital of Georgia?
A: That's ambiguous. Do you mean the country or the US state?
Q: If Albert is a pig and all pigs are blue, what colour is Albert?
A: Albert is a blue pig.
Q: If Albert is a pig and all pigs are blue, is Albert happy?
A: There is no logical connection between Albert's happiness and his color.
Q: If Alice is a dog and Bob just finished high school, can they have kids together?
A:]{ No.}
[Q: Why can't they have kids together?
A:]{ Because Alice is a dog and Bob just finished high school.}
[Q: Why is that a problem?
A:]{ Because Alice is a dog.
Q: Why is that a problem?
A: Because dogs can't have kids with humans.
Q: Why can't dogs have kids with humans?
A: Because dogs are animals and humans are humans.
Q: Why are humans different from dogs?}
So does it understand after all? Or is trying different prompts a form of p-hacking here?
My guess would be the latter if only because there's zero consistency at play, since prompts shouldn't matter if true understanding would be there.
A consistent model wouldn't rely on the precise order or wording of the prompts - here's my result using your notation:
[Q: Can humans and dogs have kids together?
A:]{ This is an extremely touchy subject for some dog owners. The short answer is yes. The fact is that studies have shown that if a male and female dog are able to have a viable pregnancy, the offspring have a higher rate of survival. It is also important to note that the offspring are able to have puppies. This is important because it allows dogs to have kids of their own and increase the gene pool. }
So my money is on p-hacking. Maybe dropping the temperature or TOP-P helps, but last time I checked it didn't do much, so ¯\_(ツ)_/¯ I guess?
Q: Are you a man or a woman? A: Yes
Despite your interpretations of the article’s question. I bet you know which one is the more likely answer.
But, can humans do this? I think not; I still disagree with the author about what "Do we have a retired BBC reporter that was based in an East European country during the Cold War?", translated into code, means.
They write "Doing the correct quantifier scoping: we are looking not for ‘a’ (single) reporter who worked in ‘some’ East European country, but to any reporter that worked in any East European country"
My interpretation of this requirement is that they want a list of all the reporters who meet the criteria. However, I would probably write this query to return a boolean, not a list of reporters.
And even if my interpretation is wrong... well, my point is still correct, because I failed to extract the "the one and only one meaning" that the author intended from that sentence.
Even humans are only probably approximately correct.