It knows that the sentence structure is very similar to a class of sentences it has seen before and that the expected response is to take tokens from certain locations in that sentence and arrange it in a certain way, which resembles an algebra equation

It doesn't understand credit card rewards, it understands how to compose an elementary word problem into algebra

> It doesn't understand credit card rewards

Probe it, go in and ask all sorts of questions to check if it understands credit card rewards, credit cards, rewards, their purpose, can solve math problems on this topic, etc.

Examples? I’m giving questions that I usually see in r/creditcards.

Then don't. Instead of breathlessly trying to prove your theory, try and do some science by falsifying it. (cf. Wason test)

Think of things it would get right only if it truly understood, not "common questions on reddit".

The entire idea of solving math problems in middle school was that you didn’t have to know the domain and that all of the necessary information was there.

When I wrote code for the health care industry, if you had asked me anything deeper about the industry or how to do brain surgery, I couldn’t have answered your question.

You're still trying to prove your position.

Look, you're all over this thread misunderstanding LLMs and rejecting the relatively correct explanations people are giving you. The comment by joe_the_user upthread that you called an oversimplification was in fact a perfect description (randomly sampling from a space of appropriate inputs). That's exactly the intuition you should have.

Do you know the Wason test? The point is that people do not intuitively know how to correctly pick which experiments to do to falsify an assumption. My point is that you are not picking the right experiments to falsify your assumptions, instead you're confirming what you think is going on. You're exactly failing the Wason task here.

Really want to understand language models? Go build a few from scratch.

Don't have time for that? Read Wolfram's post or any of the other similar good recent breakdowns.

Only interested in understanding by playing with it? Great! An experimentalist in the true scientific tradition. Then you're going to have to do good experimental science. Don't be fooled by examples that confirm what you already think is going on! Try to understand how what people are telling you is different from that, and devise experiments to distinguish the two hypotheses.

If you think ChatGPT "understands" word problems, figure out what "understanding" means to you. Now try your best to falsify your hypothesis! Look for things that ChatGPT can't do, that it should be able to do if it really "understood" by your definition (whatever you decide that is). These are not hard to find (for most values of "understand"). Finding those failures is your task, that's how you do science. That's how you'll learn the difference between reality and what you're reading into it.

I am not trying to anthropomorphize anything.

But to say it’s just a glorified search engine is minimizing the impact it and the likes of it can have on entire industries.

It’s already obviated the need for me to find a junior dev to farm off some scripts that I really didn’t have time to write

Check out my other reply

https://news.ycombinator.com/item?id=34811353

The relevant part

“However, keep in mind that the value of the rewards can vary depending on how you redeem them, and your personal spending habits and travel preferences may also play a role in determining which card is right for you.”

One can equally say, "Human brains only know that a neuron is activated by a pattern of axon firing in response to physical inputs from nerve endings."

Does any of that change anything? Not really.

>It doesn't understand credit card rewards

Is this assertion based on anything but philosophical bias surrounding the word "understand"?

>it understands how to compose an elementary word problem into algebra

That's exactly how a human, who may or may not have understood rewards programs beforehand, would solve that word problem.

Dan Dennett is informative here.[0] Yes, consciousness (even your own!) can just be a bag of tricks.

[0] https://youtu.be/fjbWr3ODbAo?t=25s

Isn’t that the point? Isn’t that the same thing we test middle schoolers on in pre-algebra classes?

My more complicated question comparing the Amex Hilton Aspire to the Amex Gold was inspired by this question asked over at r/creditcards

https://www.reddit.com/r/CreditCards/comments/112udp3/what_i...