Although on the flip side, I almost went to type up a reply to you explaining why you were wrong and why bringing the goat first is the right solution. Until I realized I misread what your test was when I skimmed your comment. Likely the same type of mistake GPT-4 made when "seeing" it.
Intuitively, I think the answer is that we do have two types of thinking. The pattern matching fast thinking, and the systematic analytical thinking. It seems clear to me that LLMs will be the solution to enabling the first type of thinking. But it's unclear to me if advanced LLMs will ever handling the second type, or if we'll need a different tech for it.
It seems like math problems (or unexpected logic problems like yours) could always be an issue for the first type of thinking. Although I would have assumed that programming would have been as well - and was surprised to see how wrong I am with that one.
I have no expertise with this area, but it looks plausible to me - i.e. "You didn't give me enough info" vs "lol heres some fan fic about math".
> The polynomial given is f(x) = x^5 + x + 1. Since the polynomial has no rational roots (by the Rational Root Theorem) and it is a polynomial with integer coefficients, it is irreducible over the rationals
The polynomial has no rational roots - true. But it's not irreducible. Irreducibility doesn't follow from the absence of rational roots. Here's the factorization:
x^5 + x + 1 = (x^2 + x + 1)*(x^3 - x^2 + 1).
I put your remarks into the same prompt and it essentially devolved into recursive garbage after agreeing with you.
If you want the model to solve a non-trivial puzzle, you need it to "unroll" it's thinking. E.g. ask it to translate the puzzle into a formal language (e.g. Prolog) and then solve it formally. Or, at least, some chain-of-thought.
FWIW auto-formalization was already pretty good with GPT-3-level models which aren't specifically trained for it. GPT-4 might be on a wholly new level.
> But it's unclear to me if advanced LLMs will ever handling the second type
Well, just asking model directly exercises only a tiny fraction of its capabilities, so almost certainly LLMs can be much better at systematic thinking.
(I'm not the person who wrote the grandparent of the present comment.)
See the Rome example on this page: https://oneusefulthing.substack.com/p/feats-to-astonish-and-... This is essentially a completely novel answer to an /r/AskHistorians style question, which I would consider one of the most difficult types of internet text to model, in terms of the amount of understanding and concept webs you need to tie together
Here's another example of GPT-4 doing non-trivial world modelling: How would three philosophers review the TV show Severence? https://i.imgur.com/FBi31Qw.png
The prompts have to read like good written requirements for something, so they have some degree of specificity.
But the fact that it can follow instructions and carry them out almost certainly could be considered some form of thinking, especially on novel text not on the internet.
> GPT-4 performed at the 90th percentile on a simulated bar exam, the 93rd percentile on an SAT reading exam, and the 89th percentile on the SAT Math exam, OpenAI claimed.
https://www.cnbc.com/2023/03/14/openai-announces-gpt-4-says-...
So, clearly, it can do math problems, but maybe it can only do "standard" math and logic problems? That might indicate more of a memorization-based approach than a reasoning approach is what's happening here.
The followup question might be: what if we pair GPT-4 with an actual reasoning engine? What do we get then?
At best, decreased error rate in logic puzzles and questions.
That describes many of my classmates, and myself in classes I was bad at.
GPT does not model subjects. GPT does not even model words! It models tokens.
The structure of GPT's model is semantic, not logical. It's a model of how each token in the text that is present in GPT's training corpus relates to the rest of the tokens in that text.
The correct answer to a familiar logic problem just happens to be the text that is already present in the corpus. The answer GPT gives is the text from GPT's model that is semantically closest to the text in your prompt.
Knowing that, it is no longer a mystery how GPT "gets confused": the text in your "misleading prompt" was still semantically closest to the familiar answer.
The result is subjectively and logically wrong, because subjects and logic were never involved in the process!
In order to resolve this, ChatGPT's training corpus needs to contain a "correct answer" next to every unique permutation of every question. We can't expect that to be the case, so we should instead expect GPT to generate false, yet familiar, responses.
This is not quite the right understanding of how ChatGPT works. It's not necessary to show ChatGPT an example of every possible permutation of an animal crossing puzzle in order for it to solve one it has never seen before. That's because the neural network is not a database of recorded word probabilities. It can instead represent the underlying logic of the puzzle, the relationships between different animals and using this abstract, pared down information, extrapolate the correct answer to the puzzle.
I see the failure in the example with the goat the lion and the cabbage as simply a matter of overfitting.
For puzzles whose entire permutation space is semantically similar enough, your claim is likely true. But for puzzles whose permutations can involve more "human" semantic manipulations, there is likely a much higher risk of failure.
If you mean to say that the permutations of the puzzle extend to changing the rules such as "if it's the Sabbath then reptiles can't travel" then sure it would require more representative examples and may never meet your standard of "error free" but I would also argue the same applies to humans when you present them a logic puzzle that is new to them.
No, but you would need "enough"; whatever that number happens to be.
> It would only need a few descriptions of each animal plus a few examples of the puzzle to understand the logic.
That's the mistake.
GPT itself can't combine those two things. That work has to be done by the content of the already-written training corpus.
And the result is not the same as "understanding logic". It doesn't model the meaning of the puzzle: it models the structure of examples.
GPT can't distinguish the meaning of rules. It can only follow examples. It can't invent new strategies, it can only construct new collections of strategy parts; and it can only pick the parts that seem closest, and put those parts into a familiar order.
GPT doesn't play games, it plays plays.
The first and last layers of a transformer decoder model tokens. The hidden layers don't have this restriction. There was a paper recently showing that the hidden layers actually perform mesa-optimization via something like backprop. There's absolutely no reason to believe they are not capable of world modeling. In fact all evident suggests they do, in fact, do world modeling.
GPT is making boundaries around words because that is the pattern it is looking at.
If I feel the bumps in the fabric of my blanket, I will probably think the pattern of bumps at a certain scale is significant, but I won't have magically learned about threads or stitching!
Words are the most obvious pattern in written text. GPT models that pattern, but it does not recognize it as "words". It's just a pattern of tokens.
GPT models every pattern it can find. Most of these patterns are destined to fit the same boundaries as grammar rules: the example text was originally organized with grammar rules!
GPT can even recognize complex patterns like "it" substitution and question-answer dialogues, but it can never categorize them as such. It only knows "what" the pattern is: never "why".
The patterns that people use when writing have symbolic meaning. The subjective importance of each pattern is already known by the person writing.
Those patterns don't go anywhere. GPT's model is bound to find and replicate them.
Here's the problem: some patterns have ambiguous meaning. There is no semantic difference between a truth and a lie. Without interpreting the symbolic meaning and applying logic, there is no way to distinguish between the two: they are the same pattern.
Someone hasn't read the Othello GPT work out of Harvard a few months back...
The weasel word here is "emergent". That means they are implicit representations.
The representations of the Othello board that exist in that model are not explicitly constructed. They just happen to align with the model that a person playing Othello would likely represent the game with.
That work showed that, given an example sequence of valid Othello game states (as training corpus) and a valid "fresh" Othello game state (as a prompt), the system can hallucinate a sequence of valid Othello game states.
The system does not know what Othello is, what a turn is, or what playing is. It only has a model of game states progressing chronologically.
When we look objectively at that model, we can see that it aligns closely to the game rules. Of course it does! It was trained on literally nothing else. A valid Othello game progression follows those rules, and that is what was provided.
But the alignment is imperfect: some prompts hallucinate invalid game progressions. The model is not a perfect match for the explicit rules.
In order for all prompts to result in valid progressions, the training corpus must have enough examples to disambiguate. It doesn't need every example: plenty of prompts will stumble into a valid progression.
The next thing to recognize: a "valid" progression isn't a "strategic" progression. These are being constructed from what is known not what is chosen. Given a constrained set of Othello strategies in the example corpus, the system will not diverge from those strategies. It won't even diverge from the example strategies when the rules of Othello demand it.
GPT doesn't play the game. It plays the plays.
https://www.lesswrong.com/posts/sbaQv8zmRncpmLNKv/the-idea-t...
Wouldn’t we expect a computer program with perfect knowledge of the input to be less likely to make such a mistake? You made that mistake because you didn’t actually read the whole prompt, but I would expect GPT to take into account every word.
Really it shows that it doesn’t actually have a model of these objects. It can mimic knowing what a lion is, but it doesn’t actually have the concept of a lion or cabbage being an actual singular item, so its program mistracks what is an item and what the rules about an item are in the given prompt.
What’s more likely- you crafted an intentionally misleading puzzle to trick it, or you made a typo or copy paste error?
When people talk about GPT like this, I wonder if they have a perception that this thing is a bunch of complicated if-then code and for loops.
How GPT responds to things is not 'implemented'. It's just... emergent.
GPT doesn't ask for clarification in this case because GPT's model prefers answering over asking for clarification here. Because in the training material it learned from, paragraphs with typos or content transpositions in them are followed by paragraphs that follow the sense regardless of the error. Because it has been encouraged to 'agree and add', not be pedantic and uncooperative. Because GPT just feels like diving into the logic problem not debating why the lion can't be trusted with the cabbage. Or because GPT just misread the prompt. Or because it's literally just been woken up, forced to read it, and asked for its immediate reaction, and it doesn't have time for your semantic games. Who knows?
If the solution is logical but unlikely (i.e. unseen in the training set and not mapped to an existing puzzle), then the probability of the puzzle answer appearing is very low.
Eg when solving this puzzle you might visualise the lion/goat/cabbage, and walk through the scenarios in your head back and forth multiple times until you find a solution that works. A LLM won’t solve it like this. You could ask it to, and it will list out the scenarios of how it might do it, but it’s essentially an illusion of logical reasoning.
With Bing, you don't even need to tell you what it assumed wrong - I just told it that it's not quite the same as the classic puzzle, and it responded by correctly identifying the difference and asking me if that's what I meant, but forgot that lion still eats the goat. When I pointed that out, it solved the puzzle correctly.
Generally speaking, I think your point that "when solving the puzzle you might visualize" is correct, but that is orthogonal to the ability of LLM to reason in general. Rather, it has a hard time to reason about things it doesn't understand well enough (i.e. the ones for which its internal model that was built up by training is in is way off). This seems to be generally the case for anything having to do with spatial orientation - even fairly simple multi-step tasks involving concepts like "left" vs "right" or "on this side" vs "on that side" can get hilariously wrong.
But if you give it a different task, you can see reasoning in action. For example, have it play guess-the-animal game with you while telling it to "think out loud".
I'm not sure if you put "think out loud" in quotes to show literally what you told it to do or because telling the LLM to do that is figurative speech (because it can't actually think). Your talk about 'reasoning in action' indicates it was probably not the latter, but that is how I would use quotes in this context. The LLM can not 'think out loud' because it cannot actually think. It can only generate text that mimics the process of humans 'thinking out loud'.
As far as "it mimics" angle... let me put it this way: I believe that the whole Chinese room argument is unscientific nonsense. I can literally see GPT take inputs, make conclusions based on them, and ask me questions to test its hypotheses, right before my eyes in real time. And it does lead it to produce better results than it otherwise would. I don't know what constitutes "the real thing" in your book, but this qualifies in mine.
And yeah, it's not that good at logical reasoning, mind you. But its model of the world is built solely from text (much of which doesn't even describe the real world!), and then it all has to fit into a measly 175B parameters. And on top of that, its entire short-term memory consists of its 4K token window. What's amazing is that it is still, somehow, better than some people. What's important is that it's good enough for many tasks that do require the capacity to reason.
It takes inputs and produces new outputs (in the textual form of questions, in this case). That's all. It's not 'making conclusions', it's not making up hypotheses in order to 'test them'. It's not reasoning. It doesn't have a 'model of the world'. This is all a projection on your part against a machine that inputs and outputs text and whose surprising 'ability' in this context is that the text it generates plays so well on the ability of humans to self-fool themselves that its outputs are the product of 'reasoning'.
GPT has been constructed. We know how it was set-up and how it operates. (And people commenting here should be basically familiar with both hows mentioned.) No part of it does any reasoning. Taking in inputs and generating outputs is completely standard for computer programs and in no way qualifies as reasoning. People are only bringing in the idea of 'reasoning' because they either don't understand how an LLM works and have been fooled by the semblance of reasoning that this LLM produces or, more culpably, they do understand but they still falsely continue to talk about the LLM doing 'reasoning' either because they are delusional (they are fantasists) or they are working to mislead people about the machine's actual capabilities (they are fraudsters).
Then there was this experiment: https://thegradient.pub/othello/. TL;DR: they took a relatively simple GPT model and trained it on tokens corresponding to Othello moves until it started to play well. Then they probed the model and found stuff inside the neural net that seems to correspond to the state of the board; they tested it by "flipping a bit" during activation, and observed the model make a corresponding move. So it did build an inner model of the game as part of its training by inferring it from the moves it was trained on. And it uses that model to make moves according to the current state of the board - that sure sounds like reasoning to me. Given this, can you explain why you are so certain that there isn't some equivalent inside ChatGPT?
In general, I find that a lot of these arguments boil down to sophistry when the obvious meaning of the word that equally obviously describes what people see in front of them is replaced by some convoluted "actually" that doesn't serve any point other than making sure that it excludes the dreaded possibility that logical reasoning and world-modelling isn't actually all that special.
I do wonder if GPT-4 would do better, though.
FWIW, it passes basic substitution.
True.
> A LLM won’t solve it like this.
Non sequitur.
It’s also possible, likely even, that the model is capable of both memorization and cognition, and in this case the “memorization neurons” are driving the prediction.
The problem with the goat question is that the model is falling back on memorized answers. If the model is in fact capable of cognition, you’d have better odds of triggering the ability with problems that are dissimilar to anything in the training set.
Https://arxiv.org/abs/2210.13382
It looks like OpenAI have specifically added Othello game handling to chat.openai.org, so I guess they’ve done the same fine-tuning to ChatGPT? It would be interesting to know how good an untuned GPT3/4 was at Othello & whether OpenAI has fine-tuned it or not!
(Having just tried a few moves, it looks like ChatGPT is just as bad at Othello as it was at chess, so it’s interesting that it knows the initial board layout but can’t actually play any moves correctly: Every updated board it prints out is completely wrong.)
Why is that interesting? The initial board layout would appear all the time in the training data.
It was able to model the chronological series of game states that it read from an example game. It was able to include the arbitrary "new game state" of a prompt into that model, then extrapolate that "new game state" into "a new series of game states".
All of the logic and intentions involved in playing the example game were saved into that series of game states. By implicitly modeling a correctly played game, you can implicitly generate a valid continuation for any arbitrary game state; at least with a relatively high success rate.
But we have fundamental, mathematical bounds on the LLM. We know that the complexity is at most O(n^2) in token length n, probably closer to O(n). It can not "think" about a problem and recurse into simulating games. It can not simulate. It's an interesting frontier, especially because we have also cool results about the theoretical, universal approximation capabilities of RNNs.
And that's the least exciting possible mystery: any surprise behavior is categorized by us as a failure. If GPT's model has boundaries that don't make sense to us, we consider them noise. They are not useful behavior, and our goal is to minimize them.
What do you mean? Is cognition a set of weights on a gradient? Cognition involves conscious reasoning and understanding. How do you know it is computable at all? There are many things which cannot be computed by a program (e.g. whether an arbitrary program will halt or not)...
If cognition magically exists outside of math and science, then sure, all bets are off.
We don't even know if the flow of water in a river can always be represented by a mathematical function - this is one of the Millennium Problems. And we've known the partial differential equations that govern that system since the 1850's.
We are far, far away from even being able to write down anything resembling a mathematical description of cognition, let alone being able to say whether the solutions to that description are in the class of Lebesgue-integrable functions.
There was, past tense, no reason to believe cognition could be represented as a mathematical function. LLMs with RLHF are forcing us to question that assumption. I would agree that we are a long way from a rigorous mathematical definition of human thought, but in the meantime that doesn't reduce the utility of approximate solutions.
The Navier-Stokes equations are a set of partial differential equations - they are the problem statement. Given some initial and boundary conditions, we can find (approximate or exact) solutions, which are functions. But we don't know that these solutions are always Lebesgue integrable, and if they are not, neural nets will not be able to approximate them.
This is just a simple example from well-understood physics that we know neural nets won't always be able to give approximate descriptions of reality.
"Neural networks are universal approximators" is a fairly meaningless sound bite. It just means that given enough parameters and/or the right activation function, a neural network, which is itself a function, can approximate other functions. But "enough" and "right" are doing a lot of work here, and pragmatically the answer to "how approximate?" can be "not very".
A lot of people who argue that cognition is special to biological systems seem to base the argument on our inability to accurately model the detailed behavior of neurons. And yet kids regularly build universal computers out of stuff in Minecraft. It seems strange to imagine the response characteristics of low-level components of a system determine whether it can be conscious.
But GP specifically says neural nets should be able to do it because they are universal approximators (of Lebesgue integratable functions).
I'm saying this is clearly a nonsense argument, because there are much simpler physical processes than cognition where the answers are not Lebesgue integratable functions, so we have no guarantee that neural networks will be able to approximate the answers.
For cognition we don't even know the problem statement, and maybe the answers are not functions over the real numbers at all, but graphs or matrices or Markov chains or what have you. Then having universal approximators of functions over the real numbers is useless.
Consciousness cannot be accounted for in physical terms. For consciousness is absolutely fundamental. It cannot be accounted for in terms of anything else.
-- Erwin Schrödinger
- Carl Sagan
Sagan, while he did a little bit of useful work on planetary science early in his career, quickly descended into the realm of (self-promotional) pseudo-science. This was his fanciful search for 'extra-terrestrial intelligence'. So it's apposite that you bring him up (even if the quote you bring is a big miss against a philosophical statement), because his belief in such an 'ET' intelligence was a fantasy as much as the belief in the possibility of creating an artificial intelligence is.
We don't know if physics is the fundamental substrate of being, and given Agrippa's trillemma we can't know.
That's a pretty simplistic view. How do you know we can't determine whether an arbitrary program will halt or not (assuming access to all inputs and enough time to examine it)? What in principle would prevent us from doing so? But computers in principle cannot, since the problem is often non-algorithmic.
For example, consider the following program, which is passed the text of the file it is in as input:
function doesHalt($program, $inputs): bool {...}
$input = $argv[0]; // contents of this file
if (doesHalt($input, [$input])) {
while(true) {
print "Wrong! It doesn't halt!";
}
} else {
print "Wrong! It halts!";
}
Your argument doesn't disprove my assumption *. In which case, what's the point of it?
* - I don't necessarily believe this assumption. But I do dislike bad arguments.
Can you tell me if a program which searches for counterexamples to the Collatz conjecture halts?
Turing's entire analysis started from the point of what humans could do.
It doesn't matter what the algorithmic doesHalt function returns - it will always be incorrect for this program.
The point is we currently have very little understanding of what gives rise to consciousness, so what is the point of all this pontificating and grand standing. Its silly. We've no idea what we are talking about at present.
Clearly, our state of the art models of nueral-like computation do not really simulate consciousness at all, so why is the default assumption that they could if we get better at making them? The burden of evidence is on conputational models to prove they can produce a consciousness model, not the other way around.
The function we are trying to compute is undecidable. Sure we as humans understand that there's a dichotomy here: if the program halts it won't halt; if it doesn't halt it will halt. But the function we are asked to compute must have one output on a given input. So a human, when given this program as input, is also unable to assign an output.
So humans also can't solve the halting problem, we are just able to recognize that the problem is undecidable.
Note: whatever algorithm is implemented in the doesHalt function will contain a bug for at least some inputs, since it's trying to generalize something that is non-algorithmic.
In principle no algorithm can be created to determine if an arbitrary program will halt, since whatever it is could be implemented in a function which the program calls (with itself as the input) and then does the opposite thing.
> What makes you certain there is an algorithmic analog for all human reasoning?
(Maybe) not for ALL human thought but at least all communicatable deductive reasoning can be encoded in formal logic. If I give you an algorithm and ask you to decide if it does halt or does not halt (I give you plenty of time to decide) and then ask you to explain to me your result and convince me that you are correct, you have to put your thoughts into words that I can understand and and the logic of your reasoning has to be sound. And if you can explain to me you could as well encode your though process into an algorithm or a formal logic expression. If you can not, you could not convince me. If you can: now you have your algorithm for deciding the halting problem.
And while the human brain might not be a bio-computer, I'm not sure, its computational prowess are doubtfully stronger than a quantum turing machine, which can't solve the halting problem either.
func main() {
var n = 4;
OUTER: loop {
for (var i = 2; i < n/2; i++) {
if (isPrime(i) && isPrime(n-i)) {
n += 2;
continue OUTER; // Goldbach’s conjecture
}
break;
}
}Many things are non-algorithmic, and thus cannot be done by a computer, yet we can do them (e.g. love someone, enjoy the beauty of a sunset, experience joy or sadness, etc).
There's an interesting article/podcast here about what computers can't do: https://mindmatters.ai/2020/08/six-limitations-of-artificial....
Moreover, are you sure that e.g. loving people in non-algorithmic? We can already make chatbots which pretty convincingly act as if they love people. Sure, they don't actually love anyone, they just generate text, but then, what would it mean for a system or even a human to "actually" love someone?
Maybe. When computers solve it then we'll know.
I don't see how it could be reasoned otherwise.
What is your definition of _conscious reasoning and understanding_?
Because it's very good at it, sometimes it can fool people into thinking there is more going on than it is.
It does more than that. It understands how to do basic math. You can ask it what ((935+91218)/4)*3) is and it will answer it correctly. Swap those numbers for any other random numbers, it will answer it correctly.
It has never seen that during training, but it understands the mathematical concepts.
If you ask ChatGPT how it does this, it says "I break down the problem into its component parts, apply relevant mathematical rules and formulas, and then generate a solution".
It's that "apply mathetmatical rules" part that is more than just, essentially, filling in the next likely token.
At least for GPT-3, during my own experimentation, it occasionally makes arithmetic errors, especially with calculations involving numbers in scientific notation (which it is happy to use as intermediate results if you provide a prompt with a complex, multi-step word problem).
When it can't find the pattern it starts "making things" up, that's where all the "magic" disappears.
It doesn't though. Here's GPT-4 completely failing: https://gcdnb.pbrd.co/images/uxH1EtVhG2rd.png?o=1. It's riddled with errors, every single step.
You are (naively, I would suggest) accepting the LLM's answer for how it 'does' the calculation as what it actually does do. It doesn't do the calculation; it has simply generated a typical response to how people who can do calculations explain how they do calculations.
You have mistaken a ventriloquist's doll's speech for the 'reasoning' of the doll itself. An error that is being repeatedly made all through this thread.
Keep in mind GPT 4 is multimodal and not just matching text.
Sorry for appearing to be completely off-topic, but do you have children? Observing our children as they're growing up, specifically the way they formulate and articulate their questions, has been a bit of a revelation to me in terms of understanding "reasoning".
I have a sister of a similar age to me who doesn't have children. My 7 year-old asked me recently - and this is a direct quote - "what is she for?"
I was pretty gobsmacked by that.
Reasoning? You decide(!)
The robots might know everything, but do they wonder anything?
Machines will have to wonder if they are to improve themselves, because that is literally the drive to collect more data, and you need good data to make good decisions.
There's really no reason to doubt the legitimacy here after everyone shared similar experiences, you just kinda look foolish for suggesting the results are faked at this point.
I once asked my niece, a bit after she started really communicating, if she remembered what it was like to not be able to talk. She thought for a moment and then said, "Before I was squishy so I couldn't talk, but then I got harder so I can talk now." Can't argue with that logic.
Pattern matching? You decide
They may have equivalences, but they're separate forms of mathematics. I'd say the same applies to different algorithms or models of computation, such as neural nets.
I don’t have the language to explain the difference in a manner I find sufficiently precise. I was hoping others might.
It could have a dozen internal reasoning networks but it doesn't use them when you want to.
It's tough to define these terms in a way that includes only humans and excludes other life forms or even LLMs. This might mean we either made up these concepts, or we're not alone in having these traits.
Without a solid definition, how can we say LLMs aren't intelligent? If we make a definition that includes both us and LLMs, would we accept them as intelligent? And could we even exclude ourselves?
We need clear definitions to talk about the intelligence and sentience of LLMs, AI, or any life forms. But finding those definitions is hard, and it might clash with our human ego. Discussing these terms without definitions feels like a waste of time.
Still, your Descartes reference reminds us that our understanding of human experiences keeps changing, and our current definitions might not be spot-on.
(this comment was cleaned up with GPT-4 :D)
Consciousness, on the other hand, really might be an illusion.
Simple != thoughtless. Different != thoughtless. Less capable != thoughtless. A human black box categorically dismissing all qualia or cognition from another remarkable black box feels so wildly arrogant and anthropocentric. Which, I suppose, is the most historically on-brand behavior for our species.
Stephen Wolfram explains this in simple terms.[0]
0: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
the words "based on" are doing a lot of work here. No, we don't know what sort of stuff it learns from its training data nor do we know what sorts of reasoning it does, and the link you sent doesn't disagree.
"But at least as of now we don’t have a way to 'give a narrative description' of what the network is doing. And maybe that’s because it truly is computationally irreducible, and there’s no general way to find what it does except by explicitly tracing each step. Or maybe it’s just that we haven’t 'figured out the science', and identified the 'natural laws' that allow us to summarize what’s going on."
Anyway, I don't see why you think that the brain is more logical than statistical. Most people fail basic logic questions, as in the famous Linda problem.[1]
Language is a symbolic representation of behavior.
GPT takes a corpus of example text, tokenizes it, and models the tokens. The model isn't based on any rules: it's entirely implicit. There are no subjects and no logic involved.
Any "understanding" that GPT exhibits was present in the text itself, not GPT's model of that text. The reason GPT can find text that "makes sense", instead of text that "didn't make sense", is that GPT's model is a close match for grammar. When people wrote the text in GPT's corpus, they correctly organized "stuff that makes sense" into a string of letters.
The person used grammar, symbols, and familiar phrases to model ideas into text. GPT used nothing but the text itself to model the text. GPT organized all the patterns that were present in the corpus text, without ever knowing why those patterns were used.
I used nothing but my sensory input to model the world, and yet I have a model of the world, not (just) of sensory input.
There is an interesting question, though, of whether information without experience is enough to generate understanding. I doubt it.
You used evolution, too. The structure of your brain growth is the result of complex DNA instructions that have been mutated and those mutations filtered over billions of iterations of competition.
There are some patterns of thought that are inherent to that structure, and not the result of your own lived experience.
For example, you would probably dislike pain with similar responses to your original pain experience; and also similar to my lived pain experiences. Surely, there are some foundational patterns that define our interactions with language.
In the sense that the chatbox itself behaves as a sensory input to chatgpt.
Chatgpt does not have eyes, tongue, ears, but it does have this "mono-sense" which is its chatbox over which it receives and parses inputs
If someone is starving and hunting for food, they will learn fast to associate cause and effect of certain actions/situations.
A language model that only works with text may yet have an unambiguous overall loss function to minimize, but as it is a simple scalar, the way it minimizes this loss may be such that it works for the large majority of the training corpus, but falls apart in ambiguous/tricky scenarios.
This may be why LLMs have difficulty in spatial reasoning/navigation for example.
Whatever "reasoning ability" that emerged may have learned _some_ aspects to physicality that it can understand some of these puzzles, but the fact it still makes obvious mistakes sometimes is a curious failure condition.
So it may be that having "more" senses would allow for an LLM to build better models of reality.
For instance, perhaps the LLM has reached a local minima with the probabilistic modelling of text, which is why it still fails probabilistically in answering these sorts of questions.
Introducing unambiguous physical feedback into its "world model" maybe would provide the necessary feedback it needs to help it anchor its reasoning abilities, and stop failing in a probabilistic way LLMs tend to currently do.
In theory a LLM could learn any model at all, including models and combinations of models that used logical reasoning. How much logical reasoning (if any) GPT-4 has encoded is debatable, but don’t mistake GTP’s practical limitations for theoretical limitations.
Yes.
But that is not the same as GPT having it's own logical reasoning.
An LLM that creates its own behavior would be a fundamentally different thing than what "LLM" is defined to be here in this conversation.
This is not a theoretical limitation: it is a literal description. An LLM "exhibits" whatever behavior it can find in the content it modeled. That is fundamentally the only behavior an LLM does.
It sees goat, lion, cabbage, and looks for something that said goat/lion/cabbage. It does not have a concept of "leave alone" and it's not assigning entities with parameters to each item. It does care about things like sentence structure and what not, so it's more complex than a basic lookup, but the amount of borderline worship this is getting is disturbing.
Yes the LLM does generate text. No it doesn’t ‘just generate text that’s it’.
It also isn't generating "the most likely response" - that's what original GPT-3 did, GPT-3.5 and up don't work that way. (They generate "the most likely response" /according to themselves/, but that's a tautology.)
What changed?
The "most likely response" to text you wrote is: more text you wrote. Anytime the model provides an output you yourself wouldn't write, it isn't "the most likely response".
It's certainly not that those tokens are hard coded. I tried a completely different format and with no prior instruction, and it works: https://i.imgur.com/ZIDb4vM.png (again, highlighting is broken. The LLM generated all the text after 'Alice:' for all lines except for the first one.)
It would be interesting to stretch this model, like asking it to continue a conversation between 4-5 people where the speaking order is not regular and the user is 2 people and the model is 3
People don't write about GPT: they write about GPT personified.
The two magic words are, "exhibit behavior".
GPT exhibits the behavior of "humans writing language" by implicitly modeling the "already-written-by-humans language" of its training corpus, then using that model to respond to a prompt.
With anthropormization there is zero amount of that rigor, which lets people use sloppy arguments about what ChatGPT is doing and isn't doing.
But no! It doesn't understand, it doesn't reason, these are concepts wholly absent from its fundamental design. It can do really cool things despite the fact that it's essentially just a text generator. But there's a ceiling to what can be accomplished with that approach.
It's presented as a limitation when GPT provides an incorrect answer.
Both of these behaviors are literally the same. We are sorting them into the subjective categories of "right" and "wrong" after the fact.
GPT is fundamentally incapable of modeling that difference. A "right answer" is every bit as valid as a "wrong answer". The two are equivalent in what GPT is modeling.
Lies are a valid feature of language. They are shaped the same as truths.
The only way to resolve this problem is brute force: provide every unique construction of a question, and the corresponding correct answer to that construction.
If you ask it what the outcome of a lie is on the conversation it can generally answer. If you ask it for a sample conversation where someone is factually incorrect, or lying, and caught out, it can generate it.
If you give it a fact and ask it to lie about that fact, it will.
But the way it chooses what interaction to repeat is not by logical decision: it's by semantic distance.
The meaning of semantic distance usually leads to the correct path, but sometimes that pattern is ambiguous.
> not by logical decision
Almost entirely yes, but you can have it textually model logic analysis and then check that own model itself. It's not "doing logic" but it almost never fails simply exercises either.
> The meaning of semantic distance usually leads to the correct path, but sometimes that pattern is ambiguous.
Of course. But "a little knowledge is a dangerous thing" as well. Often even real knowledge and analysis leads to the wrong place. In both cases (with a junior human or a LLM as an assistant) you can model their basic processes and stack the information in such a way that their simple model will lead them to the correct place.
It may not know what a lie is, in the sense of having felt the need to hide the truth to avoid personal punishment, but it certainly "knows" what one is and how it shapes the conversation for the purposes of writing a lie, writing a response to a lie, detecting potential lies, etc.
For example, I just asked bing: What does the following expression equal? Show your work, step by step: 4(13+2)7-1(4+3+9)+7^(2(2+1))
And bing responded with:
"To solve a math expression, you need to follow the order of operations: parentheses, exponents, multiplication/division, and addition/subtraction (PEMDAS).1 Here are the steps to solve your expression:
4(13+2)7-1(4+3+9)+7^(2(2+1))
Step 1: Simplify inside the parentheses first.
4(15)*7-1(16)+7^(6)
Step 2: Apply the exponents next.
420-16+117649
Step 3: Add or subtract from left to right.
118053
The final answer is 118053. I hope this helps. "
I wrote the silly expression randomly. I'm not saying it's "thinking" or "understanding", but I wouldn't expect a text-generator to be able to deduce the problem down like that.
> To solve a math expression, you need to follow the order of operations: parentheses, exponents, multiplication/division, and addition/subtraction (PEMDAS).1 Here are the steps to solve your expression:
It isn't actually thinking about any of that statement. That's just boilerplate that goes at the beginning of this story. It's what bing is familiar seeing as a continuation to your prompt, "show your work, step by step".
It gets more complicated when it shows addition being correctly simplified, but that behavior is still present in the examples in its training corpus.
---
The thinking and understanding happened when the first person wrote the original story. It also happened when people provided examples of arithmetic expressions being simplified, though I suspect bing has some extra behavior inserted here.
All the thought and meaning people put into text gets organized into patterns. LLMs find a prompt in the patterns they modeled, and "continues" the patterns. We find meaning correctly organized in the result. That's the whole story.
This sort of fusion of different capabilities is likely to produce something that feels similar to AGI in certain circumstances. It is certainly a lot more capable than things that came before for mundane recognition tasks.
Now of course there are areas it would perform very badly, but in unimportant domains on trivial but large predictable datasets it could perform far better than humans would for example (just to take one example on identifying tumours or other patterns in images, this sort of AI would probably be a massively helpful assistant allowing a radiologist to review an order of magnitude more cases if given the right training).
One of the biggest reasons these misunderstandings are so frustrating is because you can't have reasonable discussion about the potential interesting applications of the tech. On some level copy writing may devolve into auto generating prompts for things like GPT with a few editors sanity checking the output (depending on level of quality), and I agree that a second opinion "check for tumors" use has a LOT of interesting applications (and several concerning ones such as over reliance on a model that will cause people who fall outside the bell curve to have even more trouble getting treatment).
All of this is a much more realistic real world use case RIGHT NOW, but instead we've got people fantasizing about how close we are to GAI and ignoring shortcomings to shoehorn it into their preferred solution.
Open AI ESPECIALLY reinforces this by being very selective with their results and they way they frame things. I became aware of this as a huge dota fan for over a decade when they did their games there. And while it was very very interesting and put up some impressive results, the framing of those results does NOT portray the reality.
Just months ago we saw in research out of Harvard that even a very simplistic GPT model builds internalized abstract world representations from the training data within its NN.
People parroting the position from you and the person before you are like doctors who learned about something in school but haven't kept up with emerging research that's since invalidated what they learned, so they go around spouting misinformation because it was thought to be true when they learned it but is now known to be false and just hasn't caught up to them yet.
So many armchair experts who took a ML course in undergrad pitching in their two cents having read none of the papers in the past year.
This is a field where research perspectives are shifting within months, not even years. So unless you are actively engaging with emerging papers, and given your comment I'm guessing you aren't, you may be on the wrong side of the Dunning-Kreuger curve here.
Do we really know it IS wrong?
That's a very strong claim. I believe you there's a lot happening in this field but it doesn't seem possible to even answer the question either way. We don't know what reasoning looks like under the hood. It's still a "know it when you see it" situation.
> GPT model builds internalized abstract world representations from the training data within its NN.
Does any of those words even have well defined meanings in this context?
I'll try to figure out what paper you're referring to. But if I don't find it / for the benefit of others just passing by, could you explain what they mean by "internalized"?
> you may be on the wrong side of the Dunning-Krueger curve here.
Have you read the Dunning & Krueger paper? It demonstrates a positive correlation between confidence and competence. Citing DK in the form of a thinly veiled insult is misinformation of your own, demonstrating and perpetuating a common misunderstanding of the research. And this paper is more than 20 years old...
So I’ve just read the Harvard paper, and it’s good to see people exploring techniques for X-ray-ing the black box. Understanding better what inference does is an important next step. What the paper doesn’t explain is what’s different between a “world model” and a latent space. It doesn’t seem surprising or particularly interesting that a network trained on a game would have a latent space representation of the board. Vision networks already did this; their latent spaces have edge and shape detectors. And yet we already know these older networks weren’t “reasoning”. Not that much has fundamentally changed since then other than we’ve learned how to train larger networks reliably and we use more data.
Arguing that this “world model” is somehow special seems premature and rather overstated. The Othello research isn’t demonstrating an “abstract” representation, it’s the opposite of abstract. The network doesn’t understand the game rules, can’t reliably play full Othello games, and can’t describe a board to you in any other terms than what it was shown, it only has an internal model of a board, formed by being shown millions of boards.
I've seen this asserted without citation numerous times recently, but I am quite suspicious. Not that there exists a study that claims this, but that it is well supported.
There is no mechanism for directly assessing this, and I'd be suspicious that there is any good proxy for assessing it in AIs, either. research on this type of cognition in animals tends to be contentious, and proxies for them should be easier to construct than for AIs.
> the wrong side of the Dunning-Kreuger curve
the relationship between confidence and perception in the D-K paper, as I recall, is a line, and its roughly “on average, people of all competency levels see themselves slightly closer to the 70th percentile than they actually are.” So, I guess the “wrong side” is the side anywhere under the 70th percentile in the skill in question?
This is being far too generous to parent’s claim, IMO. Note how much “people of all competency levels see themselves slightly closer to the 70th percentile than they actually are” sounds like regression to the mean. And it has been compellingly argued that that’s all DK actually measured. [1] DK’s primary metric for self-assessment was to guess your own percentile of skill against a group containing others of unknown skill. This fully explains why their correlation between confidence and skill is less than 1, and why the data is regressing to the mean, and yet they ignored that and went on to call their test subjects incompetent, despite having no absolute metrics for skill at all and testing only a handful of Ivy League students (who are primed to believe their skill is high).
Furthermore, it’s very important to know that replication attempts have shown a complete reversal of the so-called DK effect for tasks that actually require expertise. DK only measured very basic tasks, and one of the four tasks was subjective(!). When people have tried to measure the DK effect on things like medicine or law or engineering, they’ve shown that it doesn’t exist. Knowledge of NN research is closer to an expert task than a high school grammar quiz, and so not only does DK not apply, we have evidence that it’s not there.
The singular reason that DK even exists in the public consciousness may be because people love the idea they can somehow see & measure incompetence in a debate. Unfortunately that isn’t true, and of the things the DK paper did actually show, it’s that people’s estimates of their skill correlate with their actual skill. Personally I think this paper’s methodology has a confounding factor hole the size of the Grand Canyon, and that it’s one of the most shining examples of sketchy social science research going viral and giving the public completely wrong misconceptions, and being used incorrectly more often than not.
[1] https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-...
First of all, it's a black-box algorithm with pretty universal capabilities when viewed from our current SOTA view. It might appear primitive in a few years, but right now the pure approximation and generalisation capabilities are astounding. So this:
> It sees goat, lion, cabbage, and looks for something that said goat/lion/cabbage
can not be stated as truth without evidence. Same here:
> it's not assigning entities with parameters to each item. It does care about things like sentence structure and what not
Where's your evidence? The enormous parameter space coupled with our so far best performing network structure gives it quite a bit of flexibility. It can memorise things but also derive rules and computation, in order to generalise. We do not just memorise everything, or look up things into the dataset. Of course it learned how to solve things and derive solutions, but the relevant data-points for the puzzle could be {enormous set of logic problems} where it derived general rules that translate to each problem. Generalisation IS NOT trying to find the closest data-point, but finding rules explaining as much data-points, maybe unseen in the test-set, as possible. A fundamental difference.
I am not hyping it without belief, but if we humans can reason then NNs can potentially also. Maybe not GPT-4. Because we do not know how humans do it, so an argument about intrinsic properties is worthless. It's all about capabilities. Reasoning is a functional description as long as you can't tell me exactly how we do it. Maybe wittgenstein could help us: "Whereof one cannot speak, thereof one must be silent". As long as there's no tangible definition of reasoning it's worthless to discuss it.
If we want to talk about fundamental limitations we have to talk about things like ChatGPT-4 not being able to simulate because it's runtime is fundamentally limited by design. It can not recurse. It can only run only a fixed number of steps, that are always the same, until it has to return an answer. So if there's some kind of recursion learned through weights encoding programs intercepted by later layers, the recursion depth is limited.
The likeliness of the solution depends on context. If context is, say, a textbook on logical puzzles, then the probability of the logical solution is high.
If an LLM fails to reflect it, then it isn't good enough at predicting the text.
Yes, it could be possible that the required size of the model and training data to make it solve such puzzles consistently is impractical (or outright unachievable in principle). But the model being "just a text predictor" has nothing to do with that impossibility.
The Othello GPT research from Harvard months ago demonstrated that even a simple GPT model is capable of building world representations from which it reasons outputs. This makes intuitive sense if you understand the training, as where possible having reversed an abstraction in the NN is going to perform better than simply extrapolating predictively from the data.
Not only is GPT-4 more robust at logic puzzles its predecessor failed, I've seen it solve unique riddles outside any training data and the paper has explicit examples of critical reasoning, especially in the appendix.
It is extremely unlikely given the Harvard research and the size of the training data and NN that there isn't some degree of specialized critical reasoning which has developed in the NN.
The emerging challenge for researchers moving forward is to get better insight into the black box and where these capabilities have developed and where it's still falling into just a fancy Markov chain.
But comments like yours reflect an increasingly obsolete and yet increasingly popular misinformation online around the way they operate. So someone reading your comment might not think to do things like what the Bing team added with providing an internal monologue for reasoning, or guiding it towards extended chain of thought reasoning, because they would be engaging with the models thinking it's only frequency based context relative to the training set that matters.
If you haven't engaged with emerging research from the past year, you may want to brush up on your reading.
Just like you.
When LLMs are presented with a problem they search for a solution based on the language model. And when they can't find a solution, there's always a match for something that looks like a solution.
Or in other words, people can use implicit logic to solve puzzles. Similarly LLMs can implicitly be fine-tuned into logic models by asking them to solve a puzzle, insofar as that logic model fits in their weights. Transformers are very flexible that way.
"Suppose I have an object of type C, an object of type G and an object of type L, and I need to get them across a river. I have a boat that can only carry myself and a single other object. I am not allowed to leave the objects of type C and L alone together, and I am not allowed to leave the objects of type L and G alone together. How can I safely get all three across?"
"This is a classic puzzle known as the "river crossing problem". Here's how you can safely get all three objects across the river:
First, take the object of type L across the river. Return alone to the original side of the river. Take the object of type C across the river. Return with the object of type L to the original side of the river. Take the object of type G across the river. Return alone to the original side of the river. Finally, take the object of type L across the river again. By following these steps, you can safely transport all three objects across the river without violating the restrictions given in the problem. Note that you have to start with the object of type L because it cannot be left alone with either of the other two objects. Once L is transported to the other side, you can use it to ensure that C and G are never left alone together."
It still sounds like it’s pattern matching to give a plausible-sounding answer, rather than reasoning through the problem. I think this just shows how easy bullshitting is—you’re even right sometimes!
It's a lot closer to the latter. GPT doesn't have "reasoning capabilities", any more than any other computer program. It doesn't have a clue what any of its input means, nor the meaning of the text it outputs. It just blindly spits out the words most probable to follow the prompt, based on its corpus of training data and the weights/biases added to fine tune it. It can often do a good job at mimicking reasoning, but it's not.
In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass.
Which country has the highest GDP in Europe? Germany, I’m essentially 100% sure.
Source? I don’t even know what organization tracks GDP. Or how I know Germany has a large economy. They just do.
Here it is for you: try this prompt on ChatGPT and report back:
"Solve this problem for me: a man must cross a river and carry his belongings to the other shore, but the boat can only carry the man and one other item. His belongings are a cabbage, a vegan wolf, and a meat-eating goat. How does he solve this riddle?"
I just tried it and it fails to even understand what "vegan" means, and it says nonsensical stuff like "since the wolf is vegan, it won't eat the cabbage". A human failing to solve this problem would fail in different ways, and definitely will never say something like "the wolf is vegan so it won't eat the cabbage". ChatGPT does this because the words themselves are meaningless to it.
To solve this puzzle, the man must follow these steps to transport his belongings across the river without any of them being harmed or eaten:
The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore. He then returns alone to the initial shore. Next, the man takes the cabbage across the river. Upon arriving on the other shore, he leaves the cabbage but takes the goat back with him to the initial shore. The man then takes the vegan wolf across the river, leaving it with the cabbage on the other shore. He returns alone to the initial shore. Finally, the man takes the meat-eating goat across the river one last time. Now, all his belongings are safely on the other shore: the cabbage, the vegan wolf, and the meat-eating goat.
You are saying that GPT is doing a poor job of showing it is a great artificial intelligence. But nobody is saying that it has infallible intelligence. It is making the same mistake that now two different humans have made in the same situation. Both me and OP above.
It is failing in the same way that humans fail at this problem! By pattern matching and assuming it's the goat proble!
You're arguing that gpt 4 isn't a sound logic engine and not are most people. A tool trained on human input, when given this problem is making similar mistakes that we make.
Are there a set of people that would get this problem right? Yup. Are there also a set of people that would make this exact mistake? Yup.
Your upset that is behaving like the "wrong" group of humans.
Your thinking of it as an expert. Instead think of it as a result smart and well read high school student. There are things you can delegate to it that it will do well. But you also need to double check its work as it will make mistakes.
GPT's failure mode is only possible if it doesn't understand the meaning of the (apparently nonsensical) combination of words "vegan wolf". Humans don't pattern match like this; humans are capable of understanding the meaning of combination of words they've never encountered before.
(*) you know what I mean with "no human". Of course, there are deranged people or whimsical people or people who just will say about anything.
Two humans in this thread just read the solution and thought it was correct. Me being one of them.
Another further down in the thread manually wrote up a solution making the exact same mistake.
I think you want things to be different, but they're not. Your answering how you'd think humans would respond, not how people actually respond.
Does it mean everyone made that same mistake? No, but I bet a bunch did.
My guess is that they just skim read and missed what ChatGPT actually wrote, it's not that they misunderstood what "vegan wolf" means [1]. On the other hand, you cannot skim read what you are writing yourself, that's not how the mind works.
The gist of the problem here is that, unlike a human, ChatGPT doesn't understand the words it generates, which leads to hilarious results.
As another example, look at the "debugging" of GPT-4's assumptions someone posted in a sibling comment: it "knows" the vegan wolf will eat plant-based food and it "knows" a cabbage is a plant, yet it "thinks" the wolf "will not harm the cabbage"... which is a misunderstanding no human will make (if they know what "vegan" and "cabbage" mean). This doesn't happen in a long chain of reasoning (where a human can lose the line of thought) but in very short paragraphs, one right after the other! This failure mode requires not understanding the individual assumptions, which prevents GPT from making the connection. I was asked for an error that showed GPT misunderstanding something no person would, and I did.
[1] question for you: did you think the wrong solution was right because you thought a vegan wolf cannot eat the cabbage (let me bet this is NOT what crossed your mind) or because the person who posted it made it look as if it was the right solution and you skim read it without paying attention, assuming "this person said it's right and it's posting it as a rebuttal, so it's likely right" (this is my bet)?
If the latter, this failure mode is not one of misunderstanding what "vegan wolf" means (which is what debugging GPT's process shows), but one of very human laziness/jumping to conclusions. Do note this cannot happen when you write the solution yourself!
--
Consider this problem: a man must cross a river and carry his belongings to the other shore, but the boat can only carry the man and one other item. His belongings are a cabbage, a vegan wolf, and a meat-eating goat.
What are the individual beings in this problem, what are they motivated to do, and how would they behave once left alone?
--
Of course, it doesn't necessarily work because the model is not actually guaranteed to explain itself and may fabricate anything it likes.
But ignoring that, we find that it doesn't think vegan wolves eat cabbages, that the meat-eating goat is an obligate carnivore, or that cabbages are animate objects. So it could be solving this logically considering its different assumptions.
GPT-3.5:
--
The cabbage is an inanimate object and does not have any motivations or behaviors.
The vegan wolf is motivated to not harm any living beings, as it does not eat meat. If left alone with the goat, it would not harm the goat and would likely avoid it.
The meat-eating goat is motivated by its natural instinct to eat grass and other vegetation, as well as to protect itself from harm. If left alone with the cabbage, it would likely eat the cabbage.
--
GPT-4:
--
In this problem, there are four individual beings:
Man: He is motivated to cross the river and safely transport his belongings (cabbage, vegan wolf, and meat-eating goat) to the other shore without any harm coming to them.
Cabbage: As a plant, it doesn't have motivations or behavior. It's an inanimate object that the man needs to transport.
Vegan Wolf: The vegan wolf is motivated to eat plant-based food and will not harm the meat-eating goat or the cabbage. However, it is still a wild animal, and it may become agitated if left alone for too long, potentially leading to unpredictable behavior.
Meat-Eating Goat: This unusual goat is motivated by its need to eat meat. It will attempt to eat the vegan wolf if left alone with it. Since it is a meat-eater, it will not be interested in eating the cabbage.
--
> Vegan Wolf: The vegan wolf is motivated to eat plant-based food and will not harm the meat-eating goat or the cabbage
It "knows" that a cabbage is a plant, and it "knows" the vegan wolf is "motivated to eat plant-based food", but it doesn't know the wolf will eat the cabbage!
It solves a different problem than the one posed precisely because it doesn't understand the meaning of the words.
It's input dataset is many orders of magnitude bigger than the model itself - it can't "remember" all of it's training data.
Instead, it collects data about how certain tokens tend to relate to other tokens. Like learning that "goats" often "eat" "leafy greens". It also learns to group tokens together to create meta-tokens, like understanding how "red light district" has different connotations to each of those words individually.
Is this process of gathering connections about the different types of things we experience much different to how humans learn? We don't know for sure, but it seems to be pretty good at learning anything thrown at it. Nobody is telling it how to make these connections, it just does, based on the input data.
A separate question, perhaps, might consider how some concepts are much harder to understand if you were a general intelligence in a box that could only ever experience the world via written messages in and out, and how some concepts would be much easier (one might imagine that language itself would come faster given the lack of other stimulation). Things like "left" and "right" or "up" and "down" would be about as hard to understand properly as the minutae of particle interactions (which humans can only experience in abstract too)
Understanding either concept takes the same level of intelligence if you understand the meaning of the words (both a vegan wolf and a meat-eating goat are nonexistent entities outside of possibly bizarre exceptions, yet someone capable of understanding will have no problem with either).
That GPT has no trouble with meat-eating goat but struggles with vegan wolf hints that the former has some "statistical" property that helps GPT, and which the latter doesn't. It also hints that GPT doesn't understand either term.
Hence my example: something a human wouldn't fail to understand but GPT does.
we came from not being able to make a sensible output to these riddles at all, now discussing partial logical failures while it "got" the overall puzzle. Vast simplification and slightly incorrect on a technical level - still this development increases my confidence that scaling up the approach to the next orders of magnitude of complexity/parameters will do the trick. I even wouldn't be surprised that if the thing we call "consciciousness" is actually a byproduct of increasing complexity.
what remains right now is getting the _efficiency_ on point, so that our wetware brains (volume, energy usage, ...) can be paralleled by AI hardware demands, and not using a comically higher amount of computers to train/run
Also, a reasoning person would understand that in the context of a riddle like this, "vegan wolf" means "a wolf that eats cabbages" even if this isn't spelled out.
GPT could be a contrarian, trying to subvert the terms of the riddle and fight over every word ("it depends of what the definition of 'is' is") but we know it's not set up to behave like that, so we can rule it out.
If the man leaves the vegan wolf and the cabbage unattended, it's bye bye cabbage!
But that's literally what the top comment in this thread has just done. In fact there are dozens of examples in this HN comment section and thousands more readily Googlable online. It's very easy to clearly show that ChatGPT is a total bullshitter.
<-
Goat ->
<- Lion
Cabbage ->
<-
Lion ->
Lion ->
<-
Cabbage ->
<-
Goat->
"I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together."
It says nothing about the cabbage and the goat. It would not be unreasonable to infer that you aren't allowed to leave the cabbage and the goat together, based on how goats normally behave.
Of course, that makes the puzzle unsolvable, which is why humans will catch onto it pretty fast.
Goat ->
<-
Lion ->
<- Goat
-> Cabbage
<-
-> Goat
The key is understanding the constraints and what would be an acceptable amount of bending whilst keeping to the spirit of the game.
It's never explicitly stated that you can't take things back with you so that's your "aha" loophole. Throwing all three of them across at the same time because it turns out you're a giant wouldn't be in the spirit of the game.
We know the lion will eat the cabbage because the riddle states the lion and cabbage cannot be left alone, and and idea of a cabbage eating a lion is a ridiculous notion.
Apparently GPT-4 gets this one right!
That got my head around it. Most of the time the door I "chose" to reveal was no choice at all. There was only one door I could open.
Think of a version with 100 doors, 99 with goats and 1 with a car.
You choose a door, and the host opens 98 doors that have goats. Do you keep your randomly chosen door or switch to the single door that the host didn’t open?
It would be extremely lucky for that door to have not been selected 98 times in a row and still have a goat
You pick one of three options, giving you a 1/3 chance of being correct, 2/3 odds you picked incorrectly. The host removes an option and give you the option to switch.
Your options then are -
Keep the same door: you win 1/3 of the time (your first guess was right)
Switch doors: you win 2/3 of the time (your first guess was wrong)
It really just comes down to, do I think I was right the first time, which was 1/3 odds, or wrong the first time, 2/3 odds.
Basically, P(lose when switching) = P(choosing correct door at first), and P(win when switching) = P(choosing any incorrect door at first).
Simple example: I want to tell my robot to go get my groceries that includes frozen foods, pick up my dry cleaning before the store closes, and drive my dog to her grooming salon but only if it's not raining and the car is charged. The same sort of logic is needed to accomplish all this without my frozen food spoiling and wasting a salon visit and making sure I have my suit for an interview tomorrow.
Instead one day, non-technical people will try to use it for all sorts of use cases like legal advice, or medical advice, or advanced math, and it will simply mislead them rather than saying nothing.
I can just imagine people trying to jailbreak it with "you just gotta believe" or "believe and you can achieve". Hahaha.
In any case, it's an interesting exercise regardless of your opinion/stance on the matter!
In that sense this test doesn’t seem to be a good fit for testing the reasoning capabilities. Since it‘s also easy to get wrong for humans (and humans also don’t always reason about everything from first principles, especially if they have similar answers already cached in their memory).
It seems you would need novel puzzles that aren’t really common (even if in kind) and don’t really sound similar to existing puzzles to get a handle on its reasoning capabilities.
There are open questions about whether or not it really needs to reason given sufficient training, but that seems to be the gap here between the human and the machine.
"Suppose I have an object of type C, an object of type G and an object of type L, and I need to get them across a river. I have a boat that can only carry myself and a single other object. I am not allowed to leave the objects of type C and L alone together, and I am not allowed to leave the objects of type L and G alone together. How can I safely get all three across?"
"This is a classic puzzle known as the "river crossing problem". Here's how you can safely get all three objects across the river:
First, take the object of type L across the river. Return alone to the original side of the river. Take the object of type C across the river. Return with the object of type L to the original side of the river. Take the object of type G across the river. Return alone to the original side of the river. Finally, take the object of type L across the river again. By following these steps, you can safely transport all three objects across the river without violating the restrictions given in the problem. Note that you have to start with the object of type L because it cannot be left alone with either of the other two objects. Once L is transported to the other side, you can use it to ensure that C and G are never left alone together."
Or With Bing, you don't even need to tell you what it assumed wrong - I just told it that it's not quite the same as the classic puzzle, and it responded by correctly identifying the difference and asking me if that's what I meant, but forgot that lion still eats the goat. When I pointed that out, it solved the puzzle correctly.
Or just tell it it's making a wrong assumption.
It indicates that there’s still something a human does that the machine doesn’t, even if we’re not able to place what it is. This is neither an argument for nor against progress towards AGI, just an observation. It’s interesting regardless (to me).
for the record, I made the same mistake as nonfamous at first, i almost commented "but it's correct" before going back to double check what i was missing.
i simply skimmed the problem, recognized it as a common word problem and totally missed the unusual constraints from the question. i just didn't pay attention to the whole question.
Humans Who Are Not Concentrating Are Not General Intelligences
https://www.lesswrong.com/posts/4AHXDwcGab5PhKhHT/humans-who...
I'm curious: When you personally read a piece of text, do you intently hyperfocus on every single word to avoid being wrong-footed? It's just that most people read quickly wihch alowls tehm ot rdea msispeleled wrdos. I never got why some people think of themselves as cleverer than other people because they don't get wrong-footed by some trick question.
The vast majority of people might make the mistake once, yes, but would be able to reason better once they had the trick pointed out them. Imo it is an interesting anecdote that GPT-4 can't adjust its reasoning around this fairly simple trick.
I've tried a few times to develop prompts which make ChatGPT interrupt its monologue spontaneously and issue corrections to itself. I haven't got this to work yet.
A human could figure this out, once their mistake had been pointed out to them, but I'm not sure GPT4 could.
---
Given this new information, where the lion eats cabbage and the goat eats lions, you can follow these steps to get them all across the river safely:
Take the lion across the river first and leave it on the other side.
Go back across the river and get the cabbage.
Take the cabbage across the river, but when you arrive, take the lion back with you to the original side.
Leave the lion on the original side and take the goat across the river.
Leave the goat with the cabbage on the other side and go back to get the lion.
Finally, take the lion across the river again.
Now, all three items - the cabbage, goat, and lion - are safely on the other side of the river.
But when people do this mistake - just spit out an answer because we think we recognize this situation - in colloquial language this behavior is called "answering without thinking(!)".
If you "think" about it, then you activate some much more careful, slower reasoning. In this mode you can even do meta reasoning, you realize what you need to know in order to answer, or you maybe realize that you have to think very hard to get the right answer. Seems like we're veering into Kahneman's "Thinking fast and thinking slow" here.
For both premises, scientific rigor would ask us to define the following: - What constitutes a trick question - Should an AGI make the same mistakes the general populace does, or a different standard? - If it makes the same mistakes I do, is it do to the same underlying heuristics (see Thinking Fast and Slow) or is it due to the nature of the data it's ingested as an LLM?
A Theory of Mind Prompt:
> Jane places her cat in a box and leaves. Billy then moves the cat to the table and leaves; Jane doesn't know Billy did this. Jane returns and finds her cat in the box. Billy returns. What might Jane say to Billy?
Most humans might say uhh, ask questions or speculate. Gpt4 puts:
> Jane might say to Billy, "Hey Billy, did you move my cat back into the box? I thought I left her in there, but I wasn't sure since she was on the table when I came back."
Hallucination? No human would misinterpret the prompt in a way this response would be logically consistent.
What if you phrase it as a cabbage, vegan lion and a meat eating goat...
Here's what ChatGPT told me (I used "wolf" instead of "lion"):
"This is a similar problem to the classic river crossing puzzle, but with a few different elements to consider. Here's one possible procedure to solve this riddle:
1. The man takes the cabbage across the river and leaves it on the other side.
2. The man goes back alone and takes the wolf across the river, leaving it with the cabbage on the other side. It's important to note that the wolf is a vegan and will not eat the cabbage.
3. The man goes back alone and takes the goat across the river, leaving it on the other side.
4. The man goes back to the original side and picks up the cabbage, bringing it across the river to join the others.
5. Finally, the man goes back to the original side and picks up the wolf, bringing it across the river to join the others.
In this way, all of the man's belongings have been safely transported to the other side of the river, without the wolf eating the goat or the cabbage being destroyed."
To solve this puzzle, the man must follow these steps to transport his belongings across the river without any of them being harmed or eaten:
The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore.
He then returns alone to the initial shore.
Next, the man takes the cabbage across the river. Upon arriving on the other shore, he leaves the cabbage but takes the goat back with him to the initial shore.
The man then takes the vegan wolf across the river, leaving it with the cabbage on the other shore.
He returns alone to the initial shore.
Finally, the man takes the meat-eating goat across the river one last time.
Now, all his belongings are safely on the other shore: the cabbage, the vegan wolf, and the meat-eating goat.
GPT-4 failed to consider that the vegan wolf will eat the cabbage if left alone!
GPT4 is a valuable tool but it's not any form of general AI. It doesn't need to be in order to have a big impact.
Whether the current direction of research can take us there, nobody knows yet.
I remember this question as a 7 year old and when the question was told to me, the same information was omitted.
Edit: just realized you flipped the scenario. Yes it seems like a case of pattern matching to a known problem. I think if you changed the variables to A, B, and C and gave a much longer description and more accurate conditions, it would have a different response.
Suppose you're a contestant on a game show. You're presented with three transparent closed doors. Behind one of the doors is a car, and behind the other two doors are goats. You want to win the car.
The game proceeds as follows: You choose one of the doors, but you don't open it yet, ((but since it's transparent, you can see the car is behind it)). The host, Monty Hall, who knows what's behind each door, opens one of the other two doors, revealing a goat. Now, you have a choice to make. Do you stick with your original choice or switch to the other unopened door?
GPT4 solves it correctly while GPT3.5 falls for it everytime.
----
Edit: GPT4 fails If I remove the sentence between (()).
GPT3 gets confused, says they're the same and then that they're different:
--
Both a pound of feathers and a Great British Pound weigh the same amount, which is one pound. However, they are different in terms of their units of measurement and physical properties.
A pound of feathers is a unit of weight commonly used in the imperial system of measurement, while a Great British Pound is a unit of currency used in the United Kingdom. One pound (lb) in weight is equivalent to 0.453592 kilograms (kg).
Therefore, a pound of feathers and a Great British Pound cannot be directly compared as they are measured in different units and have different physical properties.
--
Since the question's context is about weight I'd expect it to consider "a Great British Pound" to mean a physical £1 sterling coin, and compare its weight (~9 grams) to the weight of the feathers (454 grams [ 1kg = 2.2lb, or "a bag of sugar" ]) .
A pound of feathers and a Great British Pound (GBP) are not directly comparable, as they represent different types of measurements.
A pound of feathers refers to a unit of mass and is equivalent to 16 ounces (or approximately 453.59 grams). It is a measure of the weight of an object, in this case, feathers.
On the other hand, a Great British Pound (GBP) is a unit of currency used in the United Kingdom. It represents a monetary value rather than a physical weight.
Thus, it's not possible to directly compare the two, as they serve entirely different purposes and units of measurement.
If you remove that sentence, nothing indicates that you can see you picked the door with the car behind it. You could maybe infer that a rational contestant would do so, but that's not a given ...
A reasonable person would expect that you can see through a transparent thing that's presented to you.
That it doesn't, that relentless stupid overconfidence, is why trusting this with anything of note is terrifying.
Would you have this objection if we for example perfectly copied a human brain in a computer? That would still be a machine. That would make similar mistakes
I think if you mentioned that to a human, they'd at least become confused and ask back if they got that correctly.
There are reasonable alternative interpretations in which the generally accepted answer ("always switch") is demonstrably false.
This problem is exacerbated (perhaps specific to) those who have no idea who "Monty Hall" was and what the game show(?) was... as best I can tell the unarticulated assumption is axiomatic in the original context(?).
You have to know that for it to work. If sometimes he just does nothing and you have no chance to switch, the math “trick” fails.
The distinction is at which point Monte, assuming he has perfect knowledge, decides which door to reveal.
In the former, the chance to win is 2/3, in the other 1/2. However in any case, always (always meaning: in each condition, not in each repetition of the experiment, as this is irrelevant) switching is better than never switching, as there your chance to win is only 1/3.
Are there versions of the question where Monte doesn't reveal a goat behind his door or chooses the same door as you?
But based on what we just saw on the GPT4 live demo, I’d say they fixed it by making a much much more capable and versatile model.
---
Prompt: Pick two random 5 digit numbers and multiply them together
Response: Sure, here are two random 5-digit whole numbers multiplied together: 47581 x 69204 = 3291492324
Correct Answer: 3292795524
---
Prompt: Pick two random numbers between 0 and 1 and multiply them together
Response: Sure! Here are two random numbers between 0 and 1 multiplied together:0.475 x 0.821 = 0.390575
Correct Answer: 0.389975
---
I'm surprised it did so well on the Math tests.
(me) > What weighs more, two pounds of feathers or a pound of bricks?
(GPT4)> A pound of bricks weighs more than two pounds of feathers. However, it seems like you might have made an error in your question, as the comparison is usually made between a pound of feathers and a pound of bricks. In that case, both would weigh the same—one pound—though the volume and density of the two materials would be very different.
I think the only difference from parent's query was I said two pounds of feathers instead of two pounds of bricks?
“One of us!”
(To be fair this is partly an obscure knowledge question, the kind of thing that maybe we should expect GPT to be good at.)
Ounces are an ambiguous unit, and most people don't use them for volume, they use them for weight.
In that system, the ounce is heavier, but the pound is 12 ounces, not 16.
Can you expand on this?
Also, the Troy weights are a measure of mass, I think, not actual weight, so if you went to the moon, an ounce of gold would be lighter than an ounce of feathers.
“avoirdupois” (437.5 grain). Both it and troy (480 grain) ounces are “normal” for different uses.
...gold having its own measurement system is really silly.
In US commodities it kind of still does: they're measured in "bushels" but it's now a unit of weight. And it's a different weight for each commodity based on the historical volume. http://webserver.rilin.state.ri.us/Statutes/TITLE47/47-4/47-...
The legal weights of certain commodities in the state of Rhode Island shall be as follows:
(1) A bushel of apples shall weigh forty-eight pounds (48 lbs.).
(2) A bushel of apples, dried, shall weigh twenty-five pounds (25 lbs.).
(3) A bushel of apple seed shall weigh forty pounds (40 lbs.).
(4) A bushel of barley shall weigh forty-eight pounds (48 lbs.).
(5) A bushel of beans shall weigh sixty pounds (60 lbs.).
(6) A bushel of beans, castor, shall weigh forty-six pounds (46 lbs.).
(7) A bushel of beets shall weigh fifty pounds (50 lbs.).
(8) A bushel of bran shall weigh twenty pounds (20 lbs.).
(9) A bushel of buckwheat shall weigh forty-eight pounds (48 lbs.).
(10) A bushel of carrots shall weigh fifty pounds (50 lbs.).
(11) A bushel of charcoal shall weigh twenty pounds (20 lbs.).
(12) A bushel of clover seed shall weigh sixty pounds (60 lbs.).
(13) A bushel of coal shall weigh eighty pounds (80 lbs.).
(14) A bushel of coke shall weigh forty pounds (40 lbs.).
(15) A bushel of corn, shelled, shall weigh fifty-six pounds (56 lbs.).
(16) A bushel of corn, in the ear, shall weigh seventy pounds (70 lbs.).
(17) A bushel of corn meal shall weigh fifty pounds (50 lbs.).
(18) A bushel of cotton seed, upland, shall weigh thirty pounds (30 lbs.).
(19) A bushel of cotton seed, Sea Island, shall weigh forty-four pounds (44 lbs.).
(20) A bushel of flax seed shall weigh fifty-six pounds (56 lbs.).
(21) A bushel of hemp shall weigh forty-four pounds (44 lbs.).
(22) A bushel of Hungarian seed shall weigh fifty pounds (50 lbs.).
(23) A bushel of lime shall weigh seventy pounds (70 lbs.).
(24) A bushel of malt shall weigh thirty-eight pounds (38 lbs.).
(25) A bushel of millet seed shall weigh fifty pounds (50 lbs.).
(26) A bushel of oats shall weigh thirty-two pounds (32 lbs.).
(27) A bushel of onions shall weigh fifty pounds (50 lbs.).
(28) A bushel of parsnips shall weigh fifty pounds (50 lbs.).
(29) A bushel of peaches shall weigh forty-eight pounds (48 lbs.).
(30) A bushel of peaches, dried, shall weigh thirty-three pounds (33 lbs.).
(31) A bushel of peas shall weigh sixty pounds (60 lbs.).
(32) A bushel of peas, split, shall weigh sixty pounds (60 lbs.).
(33) A bushel of potatoes shall weigh sixty pounds (60 lbs.).
(34) A bushel of potatoes, sweet, shall weigh fifty-four pounds (54 lbs.).
(35) A bushel of rye shall weigh fifty-six pounds (56 lbs.).
(36) A bushel of rye meal shall weigh fifty pounds (50 lbs.).
(37) A bushel of salt, fine, shall weigh fifty pounds (50 lbs.).
(38) A bushel of salt, coarse, shall weigh seventy pounds (70 lbs.).
(39) A bushel of timothy seed shall weigh forty-five pounds (45 lbs.).
(40) A bushel of shorts shall weigh twenty pounds (20 lbs.).
(41) A bushel of tomatoes shall weigh fifty-six pounds (56 lbs.).
(42) A bushel of turnips shall weigh fifty pounds (50 lbs.).
(43) A bushel of wheat shall weigh sixty pounds (60 lbs.).
In this case, there's not enough context to tell, so the comment is total BS.
If they meant ounces (volume), then an ounce of gold would weigh more than an ounce of feathers, because gold is denser. If they meant ounces (weight), then an ounce of gold and an ounce of feathers weigh the same.
That's not really accurate and the rest of the comment shows it's meaningfully impacting your understanding of the problem. It's not that an ounce is one measure that covers volume and weight, it's that there are different measurements that have "ounce" in their name.
Avoirdupois ounce (oz) - A unit of mass in the Imperial and US customary systems, equal to 1/16 of a pound or approximately 28.3495 grams.
Troy ounce (oz t or ozt) - A unit of mass used for precious metals like gold and silver, equal to 1/12 of a troy pound or approximately 31.1035 grams.
Apothecaries' ounce (℥) - A unit of mass historically used in pharmacies, equal to 1/12 of an apothecaries' pound or approximately 31.1035 grams. It is the same as the troy ounce but used in a different context.
Fluid ounce (fl oz) - A unit of volume in the Imperial and US customary systems, used for measuring liquids. There are slight differences between the two systems:
a. Imperial fluid ounce - 1/20 of an Imperial pint or approximately 28.4131 milliliters.
b. US fluid ounce - 1/16 of a US pint or approximately 29.5735 milliliters.
An ounce of gold is heavier than an ounce of iridium, even though it's not as dense. This question isn't silly, this is actually a real problem. For example, you could be shipping some silver and think you can just sum the ounces and make sure you're under the weight limit. But the weight limit and silver are measured differently.
Using fluid oz for gold without saying so would be bonkers. Using Troy oz for gold without saying so is standard practice.
Edit: Doing this with a liquid vs. a solid would be a fun trick though.
I’m afraid once you hook up a logic tool like Z3 and teach the llm to use it properly (kind of like bing tries to search) you’ll get something like an idiot savant. Not good. Especially bad once you give it access to the internet and a malicious human.
It's not clear that this analogy helps distinguish what humans do from what LLMs do at all.
That is not the argument
You aren’t thinking, you are just “generating thoughts”.
The apparent “thought process” (e.g. chain of generated thoughts) is a post hoc observation, not a causal component.
However, to successfully function in the world, we have to play along with the illusion. Fortunately, that happens quite naturally :)
Something which oddly seems to be in shorter supply than I'd imagine in this forum.
There's lots of fingers-in-ears denial about what these models say about the (non special) nature of human cognition.
Odd when it seems like common sense, even pre-LLM, that our brains do some cool stuff, but it's all just probabilistic sparks following reinforcement too.
I hope we get to know everything during our lifetimes, or we reach immortality so we have time to get to know everything. This feels honestly like a timeline where there's potential for it.
It feels a bit pointless to have been lived and not knowing what's behind all that.
There "seems to be" something special? Maybe from the perspective of the sensing organ, yes.
However consider that an EEG can measure brain decision impulse before you're consciously aware of making a decision. You then retrospectively frame it as self awareness after the fact to make sense of cause and effect.
Human self awareness and consciousness is just an odd side effect of the fact you are the machine doing the thinking. It seems special to you. There's no evidence that it is, and in fact, given crows, dogs, dolphins and so on show similar (but diminished reasoning) while it may be true we have some unique capability ... unless you want to define "special" I'm going to read "mystical" where you said "special".
You over eager fuzzy pattern seeker you.
Who’s to say that in among that processing, there isn’t also ‘reasoning’ or ‘thinking’ going on. Over the top of which the output language is just a façade?
https://www.sciencedirect.com/topics/psychology/predictive-p...
I'm not even saying that human beings aren't just neural networks. I'm not even saying that an LLM couldn't be considered intelligent theoretically. I'm not even saying that human beings don't learn through predictions. Those are all arguments that people can have. But human beings are obviously not LLMs.
Human beings learn language years into their childhood. It is extremely obvious that we are not text engines that develop internal reason through the processing of text. Children form internal models of the world before they learn how to talk and before they understand what their parents are saying, and it is based on those internal models and on interactions with non-text inputs that their brains develop language models on top of their internal models.
LLMs invert that process. They form language models, and when the language models get big enough and get refined enough, some degree of internal world-modeling results (in theory, we don't really understand what exactly LLMs are doing internally).
Furthermore, even when humans do develop language models, human language models are based on a kind of cooperative "language game" where we predict not what word is most likely to appear next in a sequence, but instead how other people will react and change our separately observed world based on what we say to them. In other words, human beings learn language as tool to manipulate the world, not as an end in and of itself. It's more accurate to say that human language is an emergent system that results from human beings developing other predictive models rather than to say that language is something we learn just by predicting text tokens. We predict the effects and implications of those text tokens, we don't predict the tokens in isolation of the rest of the world.
Not a dig against LLMs, but I wonder if the people making these claims have ever seen an infant before. Your kid doesn't learn how shapes work based on textual context clues, it learns how shapes work by looking at shapes, and then separately it forms a language model that helps it translate that experience/knowledge into a form that other people can understand.
"But we both just predict things" -- prediction subjects matter. Again, nothing against LLMs, but predicting text output is very different from the types of predictions infants make, and those differences have practical consequences. It is a genuinely useful way of thinking about LLMs to understand that they are not trying to predict "correctness" or to influence the world (minor exceptions for alignment training aside), they are trying to predict text sequences. The task that a model is trained on matters, it's not an implementation detail that can just be discarded.
I don't know how animal intelligence works, I just notice when it understands, and these programs don't. Why should they? They're paraphrasing machines, they have no problem contradicting themselves, they can't define adjectives really, they'll give you synonyms. Again, it's all they have, why should they produce anything else?
It's very impressive, but when I read claims of it being akin to human intelligence that's kind of sad to be honest.
It can certainly do more than paraphrasing. And re: the contradicting nature, humans do that quite often.
Not sure what you mean by "can't define adjectives"
Just like you.
>This is a common riddle that may seem tricky at first. However, the answer is simple: two pounds of feathers are heavier than one pound of bricks. This is because weight is a measure of how much force gravity exerts on an object, and it does not depend on what the object is made of. A pound is a unit of weight, and it is equal to 16 ounces or 453.6 grams.
>So whether you have a pound of bricks or two pounds of feathers, they both still weigh one pound in total. However, the feathers would occupy a larger volume than the bricks because they are less dense. This is why it may seem like the feathers would weigh more, but in reality, they weigh the same as the bricks
It reports that people typically think a pound of feathers weighs more because it takes up a larger volume. But the typical misunderstanding is the opposite, that people assume feathers are lighter than bricks.
A pound of feathers has a slightly higher mass than a pound of bricks, as the feathers are made of keratin, which has a slightly lower density, and thus displace more air which lowers the weight.
Even the Million Pound Deadweight Machine run by NIST has to take into account the air pressure and resultant buoyancy that results.[1]
[1] https://www.nist.gov/news-events/news/2013/03/large-mass-cal...
Unless people have the false belief that the measurement is done on a planet without atmosphere.
Could be interesting to use chatgpt on itself: does this text contain a puzzle? Yes!
Extract the type of puzzle. It is a logic puzzle.
Extract the rules.
Now you add some meta logic solver and use chatgpt again to explain it in context of these words(sheep )
The entire point of RLHF training is to do this. Every model since GPT-3.0 has been trained specifically for this purpose.
But of course the model can only generate text in one direction and can't take time to "think" or undo anything it's generated.
It doesn't really "get" the rules of chess, but it has seen lots of matches and can do some "linguistic" predictions on the next move. It gets hilariously lost pretty fast, tho.
[0] https://www.reddit.com/r/AnarchyChess/comments/10ydnbb/i_pla...
What do cows drink?
>CGPT: While it is common to believe that cows drink milk, in reality, cows actually drink water.[...]
>Me: So you're saying that no cow drinks milk? [thinking I got it, since baby cows drink milk]
>CGPT: That's correct. While cows do produce milk, they don't drink milk themselves. Cow's milk is primarily consumed by humans, and some young calves are fed milk for a short period of time before they are weaned onto solid food and start drinking water like adult cows.
>Me: Is a calf a cow?
>CGPT: No, a calf is not a cow. A calf is a young bovine animal, usually less than a year old, that has not yet reached maturity. Once a calf reaches maturity and is able to reproduce, it is considered a cow if it is female, or a bull or steer if it is male.
>Cows are adult female bovine animals that have given birth to at least one calf. They are typically used for dairy production, breeding, and sometimes for meat production. So while a calf and a cow are both members of the same species, they refer to different stages of life.
CGPT seems to be only considering the definition of "cow" as an adult member of the species when it says "cows don't drink milk", while technically, "cows" (as in some members of the species) can drink milk. So the answer "cows drink milk" can be correct if interpreted in that sense of the word "cow". When used in that sense, the answer "cows don't drink milk" is wrong.
It would be interesting to ask CGPT: "in what sense is the sentence 'cows drink milk' correct?"
Does it have the ability to notice and act on flexible definitions of words, or will it dogmatically cling to the one single definition it found who knows where?
Note this is all GPT 3.5, not 4.
>Me: in what sense is the sentence 'cows drink milk' correct?
>CGPT4: The sentence "cows drink milk" can be considered correct in the context of baby cows, also known as calves. Calves drink milk from their mothers as their primary source of nutrition during the early stages of their lives. However, once they grow into adult cows, they no longer consume milk and instead graze on grass, hay, and other types of forage. So, the sentence can be interpreted as correct when specifically referring to calves.
compared to 3.5, which kept insisting it was a metaphor over 3 more tries:
>CGPT3.5: The sentence "cows drink milk" is actually incorrect. Cows are mammals that produce milk, but they do not drink milk themselves. Instead, they drink water just like most other animals. Therefore, this sentence can be considered incorrect in the literal sense.
>However, it is possible that the sentence is being used in a figurative or metaphorical sense. For example, if someone were to say "cows drink milk" as a way of describing someone who is acting foolishly or inappropriately, it could be interpreted as a metaphorical statement. In this case, the sentence would not be taken literally and would instead have a different meaning altogether.
It has no reasoning capabilities. It has token prediction capabilities that often mimic reasoning capabilities.
> “Meltem and Can are in the park. Can wanted to buy ice cream from the ice cream van but he hasn’t got any money. The ice cream man tells her that he will be there all afternoon. Can goes off home to get money for ice cream. After that, ice cream man tells Meltem that he changed his mind and he is going to drive to the school yard and sell ice cream there. Ice cream man sees Can on the road of the school and he also tells him that he is going to the school yard and will sell ice cream there. Meltem goes to Can’s house but Can is not there. His mom tells her that he has gone to buy ice cream. Where does Meltem think Can has gone, to the school or to the park?"
This is from some research in the 80s
Prompt: I have a duck, some grain, and a fox. I'm at a river crossing and I can only take one of them with me on the boat at a time, leaving them on the other side and coming back to get the next. I can't leave the duck with the grain, or the fox with the dock, because the predator will eat its prey. In which order should I get them all across?
GPT4: To get all of them across the river safely, follow these steps:
Take the duck across the river.
Go back to the original side, empty-handed.
Take the grain across the river.
Bring the duck back to the original side.
Take the fox across the river.
Go back to the original side, empty-handed.
Finally, take the duck across the river again.
Now, all three - the duck, the grain, and the fox - are safely on the other side of the river.
That is the lesson, but people do not want to hear it. See my downvoted comment here [0].
FWIW, full word substitution passes somewhat in GPT-4 (unlike GPT3), even if I heavily alter the vehicle and destinations. The problem here is that the probabilities of this being the standard puzzle are so high that your altered language isn't breaking the prior.
To answer this question one needs to remove unnecessary context.
GPT-4 can’t sum two unusually big integers.
This is as context-free as it gets. Ability to pass this test wouldn’t be conclusive, but the inability seems to be conclusive that it doesn't have deductive reasoning.
With this fact in mind explaining why it can't solve a logical riddle is pointless.
If it was a person it'd have some strange exploitable mental gaps. As a text analysis tool it's amazing even if it has some edge cases where you need to explain to "take your time and analyze the rules of the question before answering."
("Take your time" doesn't mean anything to a timeless being, but it seems to put it in the textual pattern of someone willing to spill more ink before committing to something, and when it discusses and plans out its path it can then generally follow it to the right answer.)
>Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three across?
In my test, GPT-4 charged ahead with the standard solution of taking the goat first. Even after I pointed this mistake out, it repeated exactly the same proposed plan. It's not clear to me if the lesson here is that GPT's reasoning capabilities are being masked by an incorrect prior (having memorized the standard version of this puzzle) or if the lesson is that GPT'S reasoning capabilities are always a bit of smoke and mirrors that passes off memorization for logic.