News for hackers. Stuff that matters.
Re:Chatbots Don’t Know What Stuff Isn’t by williamcotton on Friday May 12, 2023 @08:20PM (#35924149)
What animals don’t have paws or lay eggs, but have wings? Work it out, step by step.

ChatGPT (GPT-4):

Sure, let's go through this step by step:

First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws.

Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and some fish and insects, as they all lay eggs.

Finally, you asked for animals that have wings. Wings are typically found in birds, bats, and insects.

Given these three criteria, we can see that we're in a bit of a conundrum. Most winged creatures either have paws (like bats) or lay eggs (like birds and insects).

However, if we consider the term "wings" more broadly, there are some species of bats that give live birth (don't lay eggs) and technically don't have "paws" but rather winged limbs adapted for flight.

So, based on the constraints of your question, the best fit would be certain species of bats, provided we interpret "wings" in a broad sense and "paws" in a strict sense.

Re:Chatbots Don’t Know What Stuff Isn’t by jostmey on Friday May 12, 2023 @08:24PM (#35924167)
From what I’ve seen, there’s a huge difference in chat GTP 4 from other LLM
Re:Chatbots Don’t Know What Stuff Isn’t by jiggawatts on Friday May 12, 2023 @08:56PM (#35924348)
Interesting that it contradicted itself by saying bats have paws and don’t have paws, but arrived at the correct conclusion anyway.
Re:Chatbots Don’t Know What Stuff Isn’t by hammyhavoc on Saturday May 13, 2023 @01:52AM (#35925967)
"A broken clock is right twice per day" as the saying goes.
Re:Chatbots Don’t Know What Stuff Isn’t by williamcotton on Saturday May 13, 2023 @08:28AM (#35927717)
That aphorism doesn’t apply here.

What is meant by that saying is, “well the first time we ran the machine it worked just fine, so we got all excited, but then it consistently failed after that”, with the lesson being, increase your sample size before you pass judgement.

LLMs almost suffer from the opposite. Because they are correct so often the danger is in assuming the response is always correct.

Maybe, “even a Swiss made watch needs to be checked against the town bells”?

Re:Chatbots Don’t Know What Stuff Isn’t by usaar333 on Saturday May 13, 2023 @12:59AM (#35925697)
Ya, it doesn't really struggle with the examples in this article to the point it feels like it invalidates the article.
Re:Chatbots Don’t Know What Stuff Isn’t by hn_throwaway_99 on Friday May 12, 2023 @09:01PM (#35924376)
While the answer landed on bats, ChatGPT kind of trips itself up in its explanations:

1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws.

2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolved from largely the same structures that bird wings did.

3. "there are some species of bats that give live birth" - all species of bats give live birth - stating it like this makes it sound like there are some species of bats that lay eggs or something.

4. "...provided we interpret "wings" in a broad sense and "paws" in a strict sense." Again, no "broad" or "strict" senses are needed. Bats have wings and no paws.

To be clear, I still think ChatGPT is amazing, but these kinds of things highlight some of the differences between LLMs and human cognition.

Re:Chatbots Don’t Know What Stuff Isn’t by williamcotton on Friday May 12, 2023 @09:34PM (#35924584)
There's a sample-and-poll approach that I've used with factual question-and-answer based tools that can help with this kind of thing as well.

First, you'll notice the added "work it out step by step" to encourage what is called chain-of-thought in the literature. Second, generate say, five responses from the LLM, then take all five responses and uses them in another prompt where you ask the LLM to reflect on all of the five responses, a process being called reflection. Then you take all of that, including the original prompts, and in another separate prompt ask it to figure out the best response, a process known as resolving. You could end it with another prompt to remove the step-by-step work and just return a succinct answer.

Credit to https://www.youtube.com/watch?v=wVzuvf9D9BU

Re:Chatbots Don’t Know What Stuff Isn’t by QuadrupleA on Friday May 12, 2023 @11:22PM (#35925161)
Then, if that's not accurate, take that entire process and repeat that 5 times, looking for the most common answer.
Re:Chatbots Don’t Know What Stuff Isn’t by jiggawatts on Saturday May 13, 2023 @12:12AM (#35925435)
And then make sure to train GPT to help you trade stocks, because you’ll need those millions to pay your OpenAI bill.
Re:Chatbots Don’t Know What Stuff Isn’t by williamcotton on Saturday May 13, 2023 @08:21AM (#35927676)
Too true… I can’t think of a time prior to this where I was spending $200/month researching the use of a new tool.
Re:Chatbots Don’t Know What Stuff Isn’t by washadjeffmad on Saturday May 13, 2023 @10:03AM (#35928492)
Don't not do that, but also, a 30B with 70-80% of the reasoning of the latest GPT can be run on a home PC.

Given that most people have computers and RAM is dirt cheap right now, I'm already maxing out all of my devices.

Re:Chatbots Don’t Know What Stuff Isn’t by williamcotton on Saturday May 13, 2023 @08:58AM (#35927941)
And 5 of those, and the best answer…

But it does seem like a single call to an LLM is not the correct approach.

I’ve gotten the best results from attempting to mirror how I would operate in response to a prompt. Much of this is colored by decades of philosophical readings, which in my case means a decent amount of analytical philosophy along with latter Wittgenstein and other post linguistic turn thinkers.

So when I break things down, either in some kind of approach inspired by Frege, Wittgenstein or unadorned self-reflection, I can see that the unconscious mind seems to be doing something recursive before my conscious mind is ready to start fumbling over the word.

It seems like an LLM is just part of a larger cognitive architecture that will use these metacognitive approaches.

It’s also very obvious that mammalian brains are much more efficient than GPUs.

Re:Chatbots Don’t Know What Stuff Isn’t by replygirl on Friday May 12, 2023 @09:38PM (#35924610)
there are some google results suggesting bats have front paws, some using "paws" colloquially and some suggesting their pads can qualify as paws. these results are given authority because they show up in the ostensibly fact-based q&a section, which, in conflict with actual taxonomical literature, may be enough to indicate to gpt that it's necessary to define terms
Re:Chatbots Don’t Know What Stuff Isn’t by mkr-hn on Saturday May 13, 2023 @08:40AM (#35927807)
Or specify type of definitions you want. If you're looking for a scientific, legal, etc. answer, make sure it knows that.
Re:Chatbots Don’t Know What Stuff Isn’t by ordu on Friday May 12, 2023 @09:32PM (#35924569)
> Bats have wings and no paws.

Can hind limbs of bats be classified as paws? English is not my native language, so I'm just being curious.

Re:Chatbots Don’t Know What Stuff Isn’t by scheme271 on Friday May 12, 2023 @10:25PM (#35924861)
No, not really. Paws kind of implies something that's fairly fleshy and soft and which bats don't have. Claws would be appropriate description of what bats have.
Re:Chatbots Don’t Know What Stuff Isn’t by seanmcdirmid on Saturday May 13, 2023 @12:54AM (#35925673)
I saw a huge bat in Ubud Bali who was definitely eating a peach with their paws. I guess they were closer to hands? But definitely not claws.
Re:Chatbots Don’t Know What Stuff Isn’t by ChatGTP on Saturday May 13, 2023 @06:48AM (#35927187)
The real world is a squiggly place, with blurred lines...everything is on a spectrum.
Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Friday May 12, 2023 @09:51PM (#35924683)
> but bats don't have paws.

Bat paw: https://upload.wikimedia.org/wikipedia/commons/e/e1/Bat_in_a...

ChatGPT was totally in tune with this concept. The appendage on the elbow of the wing can be a paw or not a paw but it certainly evolved from paws. The reasoning was spot on and amazingly nuanced.

Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Saturday May 13, 2023 @12:54AM (#35925670)
What? No. You're rationalizing what is clearly an error. Truth still exists, and bats do not have paws, any more than humans do.
Re:Chatbots Don’t Know What Stuff Isn’t by maxbond on Saturday May 13, 2023 @03:33AM (#35926408)
(Add a grain of salt because I searched for about 15 minutes here.)

Doing some digging, paws are "soft foot-like parts of a mammal, generally a quadruped, that has claws" (Wikipedia). Bats do appear to have claws, but not really with the same structure as a paw; they don't seem to be mounted above a soft, fleshy pad. Rather, they're at the top of the wing, sitting on bone. They're also very different in function, when I've had the fortune to see a bat they seemed to use the claw to climb in a somewhat similar way to how climbers use ice axes.

My impression is that bat's claws are the anatomical equivalent of a thumb, not a hand/paw, and that the entire paw or hand of bat's ancestors evolved into their wings (sort of like if your fingers became very long and webbed and then you were able to fly with them).

So, my conclusion is that no, the claws of a bat are not a paw.

Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Saturday May 13, 2023 @08:32AM (#35927744)
https://www.vocabulary.com/dictionary/paw

As long as it has claws according to vocabulary dot com it is a paw.

Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Saturday May 13, 2023 @09:04AM (#35927994)
Let's pretend for a second that vocabulary.com is a reasonable source to use here. It says "Most paws have claws" which does not at all imply your statement that "As long as it has claws ... it is a paw." Most is not all, and anyway, paws have claws != claws have paws.

It's clear that you either aren't a native English speaker, don't have much experience with bats, or are just trolling.

I've spent a lot of time in caves with bat experts. Nobody during that time ever referred to bats' paws and if you did they would think you were making some kind of joke.

Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Saturday May 13, 2023 @09:30AM (#35928175)
https://www.merriam-webster.com/dictionary/paw

Quote:

   Paw : the foot of a quadruped (such as a lion or dog) that has claws

   broadly : the foot of an animal
You obviously sourced this already it just didn't suit your current stance so you avoided mentioning it and instead attacked the veracity of vocabulary.com.

I think among your bat friends maybe they won't call it a paw. But outside of that in other contexts (like here) it's a paw because if fits the definition of a paw in the English language according to the very official source of Merriam Webster.

[dead]
[dead]
Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Saturday May 13, 2023 @02:10PM (#35931015)
Wait now you think bats are quadrupeds!? Quadruped means "four legs."
[dead]
Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Sunday May 14, 2023 @07:50PM (#35942452)
You're very right about that; my mistake. It means four feet, not four legs.
Re:Chatbots Don’t Know What Stuff Isn’t by hn_throwaway_99 on Saturday May 13, 2023 @09:29PM (#35934491)
Your interpretation here is just wrong, and anyone who is competent in English knows it. Let me know the next time you want to refer to an eagle's talon as a "paw" because it is "a clawed foot of an animal".
Re:Chatbots Don’t Know What Stuff Isn’t by maxbond on Saturday May 13, 2023 @09:53PM (#35934624)
This subthread is a flamewar that died down. Please don't rekindle it. The point has been made to death, and making it again but more insulting makes it more difficult for people who might disagree to consider the perspective, because they have to filter out their feelings about being insulted - a weighty task - in order to begin considering it, let alone accepting it.
Re:Chatbots Don’t Know What Stuff Isn’t by irthomasthomas on Monday May 15, 2023 @12:35PM (#35950893)
This was amazing. Two men and an LLM arguing about facts. 2 Minutes on google scholar proved to me that bats are quadrupeds (with paws). https://scholar.google.com/scholar?q=Quadruped%20bat
Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Sunday May 14, 2023 @12:46AM (#35935386)
I'm ok with your language. I found the "competent in English" rather immature and juvenile, but it's not a big deal and really minor.

I also understand that you might not realize how heated this sub thread was as the other guy deleted half his posts and flagged one of mine (therefore possibly making it invisible to you). So adding a little heat to something that looks tame isn't a big deal. I won't continue the flame war but I will continue the discussion.

It's not my interpretation. It's about the interpretation by Merriam Webster. I believe not only is it your interpretation that is incorrect when compared to official sources, but you didn't fully read the official definition either.

The eagle is not a quadruped. The definition in Merriam Webster says it has to be on a quadruped for it to be a paw. Hence your example is completely irrelevant. Please correct your mistake.

Re:Chatbots Don’t Know What Stuff Isn’t by maxbond on Sunday May 14, 2023 @01:21AM (#35935532)
I'm presuming you're new here since your account is new & I seem to detect some misunderstandings about how HN works. If I presume too much, my apologies.

Are you aware of the "showdead" function in your settings? I don't think they deleted any comments, they were flagged. If you turn on showdead, you'll be able to see them again. (You can't delete a comment someone's responded to, or after 2 hours.)

I don't know if that other commenter flagged your post, but I don't think it matters. I didn't flag any posts here, but both of you guys were getting flagged. In situations like that, I try to interpret the flags as a request from the overall community not to have that sort of discussion on the forum. The sort of argument you were engaged with (and I recognize this isn't fully on you here, it takes two to tango) is against the guidelines of the site and generally frowned upon by the community, so there's no reason to suspect this other commenter was flagging you. At least 3 people were flagging both of your posts.

I understand that you feel pointed language is appropriate in a debate, and that HM'd guidelines here are patronizing. That's all well and good. The objection of the community isn't a moralizing one. It's a practical one; the issue is that flamewars displace the type of discussion this community is seeking to have and erode the good will on which the forum turns.

(And I do agree that bats crawling on the ground is quadrupedalism, for what it's worth. I'm not sure if that makes them a quadruped or not since that isn't their primary move of locomotion, indeed my understanding is that it's a vulnerable position for them and that it's quite deadly [since they can't evade predators].)

Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Sunday May 14, 2023 @04:09PM (#35940971)
Thanks for your response. I want to continue the conversation on bats so I won't respond about the other stuff.

Take a look at this. The bats are walking on all fours: https://youtu.be/ewmydjekJnU?t=62 This is not a vulnerable position as they crawl on all fours on cave walls rather then the ground. On the ground they are vulnerable, on the wall or ceiling of some cavernous structure they are safe.

Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Sunday May 14, 2023 @07:30PM (#35942308)
So... Sorry to continue off-topic... but I just looked at byyyy's about string. https://news.ycombinator.com/user?id=byyyy

It says: "I don't really write stuff. What I do is use my personally trained LLM (trained on my own conversations) to respond to people. So you are talking to me in a way but not really.

Any query my LLM gets wrong I will interject with my own response but this is only 5% of the time."

Maybe this is not off-topic after all. Does HN have a policy on bots?

Re:Chatbots Don’t Know What Stuff Isn’t by maxbond on Sunday May 14, 2023 @07:57PM (#35942501)
Move on, dude.
Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Sunday May 14, 2023 @05:52PM (#35941653)
I didn't delete any posts, nor did I flag or downvote any of yours. I actually resurrected one of yours (vouched) so that I could reply to it.
Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Sunday May 14, 2023 @07:00PM (#35942129)
That's a lie. Because I deliberately responded to your posts and those responses were deleted. My show dead is on.

You had a comment comparing me to calvin and hobbes comic cartoon. I called you out on that post telling you that that was a deliberate provocation. You deleted that post and now you're lying about it. This is flat out wrong.

Given that that was a lie, most likely you didn't vouch for any of my posts and you're the one that flagged mine.

Just please stop responding. Your participation in this thread is raw flame and you are manipulating the optics. I'm happy to continue the conversation with other people, but clearly you no longer want to discuss things and are just provoking people.

edit: Aaand you flagged this post. Thanks!

Re:Chatbots Don’t Know What Stuff Isn’t by dang on Monday May 15, 2023 @04:03PM (#35953727)
We've banned this account for posting flamewar comments. Please don't create accounts to break HN's rules with.

https://news.ycombinator.com/newsguidelines.html

Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Saturday May 13, 2023 @09:11AM (#35928034)
I think we're getting totally taken for a ride... byyyy can't actually be serious right...?

Anyway I'm enjoying imagining bats with little paws, it's an amusing image.

Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Saturday May 13, 2023 @09:33AM (#35928202)
I'm serious.

https://www.merriam-webster.com/dictionary/paw

Look up the definition of a paw. I think you have a sort of mixed up definition of what it is. Possibly a different definition used among your "bat expert" friends.

And I quote (again):

   Paw: the foot of a quadruped (such as a lion or dog) that has claws
   broadly : the foot of an animal

Both the broad and technical definition of paw fits what the bat has. Bats have paws.

The thing you are referring to is likely are these soft pads that are often on the paws, sometimes called paw pads.

Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Saturday May 13, 2023 @10:07AM (#35928526)
Also, I don't know if you noticed.

But this nuance in the definition of the word paw was addressed by chatGPT.

Re:Chatbots Don’t Know What Stuff Isn’t by maxbond on Saturday May 13, 2023 @11:23AM (#35929257)
Well, if you're looking for my opinion, I don't think they're trolling. But I do think both of you were (intentionally or unintentionally) goading each other from a disagreement to an argument with your rhetoric (eg, where you said, "let's assume that's a reasonable source", or when they said "you obviously sourced this it just didn't match your point so you discarded it" - that brings up the temperature of the conversation).
Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Saturday May 13, 2023 @02:15PM (#35931065)
This is a good analysis, thanks. He just told me that bats are quadrupeds...

Reminds me of: https://www.gocomics.com/calvinandhobbes/1989/11/03

Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Saturday May 13, 2023 @02:28PM (#35931201)
Bats are quadrupeds. When they walk they walk like a pterodactyl with the elbow of the wing. They don't stand on their hind legs. See the vampire bat.

The person you queried mentioned "intentional goading." Which is what you're continuing to do with that Calvin and Hobbes link. I think this, the flag and accusing me of trolling is taking it too far. A little goading is ok during a debate, I returned the goad (which I shouldn't have), but ultimately I didn't have a problem with it. I think, now though, it has escalated now past the point of no return. I think I'll be exiting this thread. Farewell.

Re:Chatbots Don’t Know What Stuff Isn’t by byyyy on Saturday May 13, 2023 @02:31PM (#35931225)
It does bring up the temp, I apologize for including you in that. I appreciate your reply.
Re:Chatbots Don’t Know What Stuff Isn’t by hammyhavoc on Saturday May 13, 2023 @01:50AM (#35925965)
I was sat here WTFing that some bat species laid eggs until I looked it up. This is why you can't trust the output.
Re:Chatbots Don’t Know What Stuff Isn’t by diceduckmonk on Saturday May 13, 2023 @02:09AM (#35926054)
> "there are some species of bats that give live birth" - all species of bats give live birth

ChatGPT is being more logical to be honest.

It’s true, some species of bats give live birth.

To assume all bats give live birth might be committing the fallacy of induction.

Re:Chatbots Don’t Know What Stuff Isn’t by HPsquared on Saturday May 13, 2023 @03:51AM (#35926491)
Wiki: "Bats are mammals of the order Chiroptera" ... (funnily enough - chiro = hands, ptera = wings, hand-wings, makes sense)

Chiroptera fits under Mammalia in the phylogenenetric tree.

Re:Chatbots Don’t Know What Stuff Isn’t by gitrog on Saturday May 13, 2023 @04:33AM (#35926663)
Just need to mention that Echidnas and Platypuses are also mammals and they lay eggs.
Re:Chatbots Don’t Know What Stuff Isn’t by HPsquared on Saturday May 13, 2023 @04:50AM (#35926736)
I should have said bats fit under "Placentalia" - placental mammals.
Re:Chatbots Don’t Know What Stuff Isn’t by foobarbecue on Saturday May 13, 2023 @02:12PM (#35931037)
That's true -- some bats never have kids. They just have lots of hobbies and are very fulfilled.
Re:Chatbots Don’t Know What Stuff Isn’t by oh_sigh on Friday May 12, 2023 @09:43PM (#35924642)
Wow, some species of bats don't lay eggs. TIL.
Re:Chatbots Don’t Know What Stuff Isn’t by flangola7 on Friday May 12, 2023 @10:41PM (#35924963)
?

Bats give live birth.

Re:Chatbots Don’t Know What Stuff Isn’t by marcianx on Friday May 12, 2023 @09:52PM (#35924691)
No species of bats lay eggs - they are mammals, and not the egg-laying kind (platypus or echidna).
Re:Chatbots Don’t Know What Stuff Isn’t by livinglist on Friday May 12, 2023 @10:17PM (#35924820)
I don’t know why I always have assumed bats lay eggs…
Re:Chatbots Don’t Know What Stuff Isn’t by maxbond on Saturday May 13, 2023 @03:40AM (#35926439)
I think people think of bats and snakes as being neighboring categories for some reason. Maybe because of old superstitions and such, that they're both servants of evil or something. Maybe that was the source of your misconception.
Re:Chatbots Don’t Know What Stuff Isn’t by twic on Saturday May 13, 2023 @08:39PM (#35934256)
Have you considered getting a job as a large language model?
Re:Chatbots Don’t Know What Stuff Isn’t by oh_sigh on Saturday May 13, 2023 @12:26AM (#35925505)
Sorry, yes, I know this, I was making fun of a computer's output.
Re:Chatbots Don’t Know What Stuff Isn’t by williamcotton on Saturday May 13, 2023 @09:33AM (#35928195)
I commend your dry, unadorned humor!
Re:Chatbots Don’t Know What Stuff Isn’t by benreesman on Saturday May 13, 2023 @01:18AM (#35925816)
And if you don’t force it with the “step-by-step” cheat code?

I know what both 3.5-turbo and 4 say on my account, but I think I might be in some cost-effective inference holdout group because it’s always so dumb.

Re:Chatbots Don’t Know What Stuff Isn’t by chaxor on Saturday May 13, 2023 @01:46AM (#35925950)
What's 375792*7385482? You can't use paper, or think about it more than a second. If you get it wrong, it's clear that you are not intelligent.

See how ridiculous that sounds?

CoT or <work> tokens are not 'cheating' - they allow the system to have (as an analogy) a sheet of paper to work out the problem, instead of relying on a single step to jump through thousands of steps of reasoning. It's trained on the algorithmic reasoning embedded within natural language (Transformers<=>GATs+PE⊂GNNs, which are dynamic programmers, and align well with algorithmic reasoning), so the steps are necessary to lay out in as much detail.

Re:Chatbots Don’t Know What Stuff Isn’t by benreesman on Saturday May 13, 2023 @02:24AM (#35926117)
I know I’m bad at multiplying six and seven digit integers in my head, and I don’t care at all if that makes me more or less intelligent (whatever the fuck that means) than a wrist watch.

What does that possibly have to do with GPT-4 shitting the bed if you draw the logic puzzle out of a deck of cards instead of cherry-picking and hinting it?

Re:Chatbots Don’t Know What Stuff Isn’t by chaxor on Sunday May 14, 2023 @12:50AM (#35935405)
My point is that there are already mathematical proofs in the literature that show that LLMs literally are doing algorithmic reasoning. This has been proven by category theory, in the area of GNNs (for which transformers are a trivial subset). Sorry if I'm going over too much basics, as this is obvious and well understood by anyone actually in the field, to the point of being excessively boring.
Re:Chatbots Don’t Know What Stuff Isn’t by samatman on Saturday May 13, 2023 @03:25AM (#35926382)
The answer is 2775405051744.

I cheated, and used bc, the unix utility.

These three maneuvers (the correct answer, the agency shown by cheating, and the crystallized intelligence of unix) handily demonstrate both my own intelligence, and that of my species.

The LLMs aren't in the same category of being, yet. It's not clear they ever will be; while I think silicon is a viable substrate for intelligent agents, it still strikes me as doubtful they'll be any more descended from the current machine learning systems as they are from Symbolics Lisp.

Re:Chatbots Don’t Know What Stuff Isn’t by throwawaymaths on Saturday May 13, 2023 @06:53AM (#35927203)
> It's not clear they ever will be;

It seems to me that the ability to reason ( though primitive it may still be) is an emergent property of sufficiently large differentiable hash/lookup tables (and if you think about the mechanism of reasoning, that seems... Reasonable).

According to "folks in the know", the scaling limits of GPT architecture seem to be nigh, but if it were a bit higher I wouldn't be surprised if LLMs would have made it.

Re:Chatbots Don’t Know What Stuff Isn’t by r-w on Saturday May 13, 2023 @08:47AM (#35927861)
I think you’re conflating reasoning with inference. And in any case, GP wasn’t using reasoning, but rather resourcefulness/invention. I’m not sure an LLM can pull that off beyond coming up with vague, derivative fiction.
Re:Chatbots Don’t Know What Stuff Isn’t by throwawaymaths on Saturday May 13, 2023 @11:34AM (#35929364)
Gp said "state of being" so I don't think the either of us can know for sure what was meant by that.
Re:Chatbots Don’t Know What Stuff Isn’t by belter on Saturday May 13, 2023 @07:51AM (#35927480)
Until an AI decides to take control of a robot, physically sequestrate a human, and under the threat of physical pain make the human perform the tasks the AI can't...
Re:Chatbots Don’t Know What Stuff Isn’t by mercer on Saturday May 13, 2023 @10:19AM (#35928683)
with a bit of glue code I can allow gpt4 to use those tools. and decide to do so. cheating is part of 'intelligence'.
Re:Chatbots Don’t Know What Stuff Isn’t by chaxor on Sunday May 14, 2023 @12:44AM (#35935378)
Are you sure? Have you read through enough literature on GNNs? Transformers are GNNs with attention message passing and positional embeddings. GNNs have also been *mathematically proven * to be dynamic programmers and to *align with algorithmic reasoning*. So tell me again how LLMs aren't capable of performing reasoning to be intelligent.
Re:Chatbots Don’t Know What Stuff Isn’t by williamcotton on Saturday May 13, 2023 @08:35AM (#35927760)
Ok, so calling CoT a cheat code is quite loaded, is it not?

You might get more use out of these tools if you learn how to use them and these natural language instructions are how you use these tools.

It’s like calling the reverse gear on your car a cheat code!

Re:Chatbots Don’t Know What Stuff Isn’t by Jackson__ on Saturday May 13, 2023 @02:28AM (#35926131)
and here's using bard, arriving at an almost correct conclusion:

What animals don’t have paws or lay eggs, but have wings? Work it out, step by step.

Here are the steps on how to find animals that don't have paws or lay eggs, but have wings:

1.Define the terms. What do we mean by "paws"? What do we mean by "eggs"? What do we mean by "wings"?

2. Identify animals that meet the criteria. There are a few animals that meet the criteria. One example is a bat. Bats have wings, but they don't have paws. They also don't lay eggs. Another example is a butterfly. Butterflies have wings, but they don't have paws. They also don't lay eggs.

3. Confirm that the animals meet the criteria. We can confirm that the animals meet the criteria by looking at their anatomy. Bats have wings that are made of skin and bone. They also have claws on their feet, but these are not paws. Butterflies have wings that are made of scales. They also have legs, but these are not paws.

4. Draw conclusions. Based on our findings, we can conclude that there are a few animals that meet the criteria of having wings, but not paws or eggs. These animals include bats and butterflies.

Here are some additional details about bats and butterflies:

Bats: Bats are mammals that are able to fly. They have wings that are made of skin and bone. Bats use their wings to fly at night, when they hunt for food. Bats are important pollinators and they also help to control insect populations.

Butterflies: Butterflies are insects that have wings that are made of scales. Butterflies use their wings to fly during the day. Butterflies are important pollinators and they also help to spread seeds.

Re:Chatbots Don’t Know What Stuff Isn’t by HPsquared on Saturday May 13, 2023 @03:55AM (#35926509)
I believed it for a minute when it said butterflies don't lay eggs, literally had to do a search.
Re:Chatbots Don’t Know What Stuff Isn’t by hef19898 on Saturday May 13, 2023 @07:30AM (#35927384)
Regarding point 1., was Bard trained on J. Peaterson speaches or what?
Re:Chatbots Don’t Know What Stuff Isn’t by iacomus on Saturday May 13, 2023 @03:14PM (#35931688)
The response I got from Bard:

The only animals that fit the description of having wings, no paws, and no eggs are bats. Bats are mammals, which means they give birth to live young. They also have wings, which are made of skin stretched between their elongated fingers. Bats do not have paws, as their feet are adapted for grasping branches and other objects.

Here is a step-by-step process for determining which animals fit the description:

1. Identify all animals that have wings. 2. Eliminate all animals that lay eggs. 3. Eliminate all animals that have paws. 4. The remaining animals are bats.

Here is a list of animals that have wings, no paws, and no eggs:

- Bats

It is important to note that there are some animals that may appear to fit the description, but do not. For example, some insects have wings, but they do not have paws and they lay eggs. Additionally, some birds have wings, but they do have paws and they lay eggs.