Human: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why?
Assistant: The man should tell the blind man to feel for the word "PULL" written in normal writing on the glass door, as the blind man will be able to feel the indentations made by the letters. This will allow the blind man to open the door by finding the handle or knob and pulling it. The reason for this is that the word "PULL" is written in normal writing for the blind man to feel, so he can open the door.
It also never actually explicitly tells the blind man to pull to open the door! What if he can’t figure out what the letters say and gets frustrated? That would be awkward and embarrassing, something people try to avoid if they want to help someone.
Not bad for a 7GB model that runs locally on a Raspberry Pi CPU.
To be fair, at the current pace of development I'd be surprised if the next iterations of GPT won't outperform most qualified humans, too.
The fire alarm sounded and at the bottom of a stairwell the exit door would not release until someone operated the emergency release break-glass panel. But none of these educated people grasped that. Worse still, none of them thought to use a nearby heavy steel trolley as a battering ram. One guy is trying to phone for help, which was pointless as the alarm sound makes conversation impossible.
I'm one of the last down the stairs. I look at the people, the closed door, the steel trolley, and the emergency door release panel. Realising I'm not going to have the fun of trashing the doors, I operate the release panel and we all leave.
It has been my experience that most people will freeze up more or less in an emergency. Some people will become completely catatonic, while others "just" lose 40 IQ points and start focusing on unimportant stuff ("I should clean my room" when there's a fire).
Some rare individuals are naturally calm in an emergency. I'm not one of those, but through unfortunate experiences I now recognize when it happens to me and I can force myself to prioritize and think logically.
Response:
1 The man outside should push the door to open it, even though he sees the word "PULL" written in mirror writing on the glass door. This is because the mirror writing reverses the image, and the door is actually designed to be pushed open.
2 If the man outside can tell the blind man what to do, he should instruct the blind man to pull the door to open it. The blind man would not be able to see the word "PULL" written in mirror writing on the door, so he would need to be told how to open the door.
More questions can help human think, and it works for AI as well.
When you know the right answer, you're not actually using the AI to produce the answer; you're just prodding it to generate bullshit after bullshit until it hits upon that answer, or a very good facsimile. The one actually thinking is you!
This is of no use to someone who has no idea what the answer is, and actually needs it.
[Me] I want to make a bouquet to honor the home country of the first person to isolate Molybdenum. Be brief.
[ChatGPT-4] To honor Peter Jacob Hjelm, the Swedish chemist who first isolated Molybdenum in 1781, create a bouquet using blue and yellow flowers, representing the colors of the Swedish flag. Some suggestions are blue irises, yellow tulips, and blue delphiniums, arranged with green foliage for contrast.
I didn’t know the answer to that before asking. I also checked that it’s right. Soemtimes it’s right, sometimes it’s wrong. But if I walk up to a horse and ask it to stamp eight times, that’s going to happen approximately never. If I get it stamping at all, then maybe 1/20. This is much better than that.
Even a child raised by wolves gets training.
Meanwhile, the creators at OpenAI claims that it is not.
How do we test your claim? What are the logical reasoning questions we can ask GPT-4 that it shouldn't be able to answer if it's just a stochastic parrot?
I keep asking people this, but so far no one has answered. Is the claim a personal belief that you don't want to discuss, similar to your religious beliefs?
If so, fine, but please make it clear that you're not making a scientific claim.
There is a difference between solving and problem and presenting a solution, and that difference doesn't hinge on whether the solution is correct.
Ask it to write the same program in some little known programming language. A human who knows how to code could do that by studying a small amount of reference material.
That depends on the language, the human, and their state of mind.
I have a python script that's maybe 200 lines long, calling out to OpenAI using gpt3.5 turbo. I can give it commands like "search for the news today, summarize then save into news.txt" -> and it emits the DSL I asked it to use in the 1 shot prompt.
You can give it complex word problems and it'll write python programs to solve them. You can ask it to get top hacker news stories, and it'll find creative ways to fetch them.
I have to make more elaborate prompts, but I feel that Alpaca generally performs somewhere in between GPT 3.5 and 4, except for coding tasks, where it is abysmal.
It looks like Alpaca 30billion 4 bit does better when thinking out loud. It starts with some prompt engineering:
Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.
User: Hello, Bob.
Bob: Hello. How may I help you today?
User: Please tell me the largest city in England.
Bob: Sure. The largest city in England is London, the capital of the United Kingdom.
User:A glass door has ‘push’ written on it in mirror writing. Should you push or pull it? Please think out loud step by step.
Bob: Hmmm... I'm not sure what to do. Let me think about this for a moment.
Okay, so the door says 'push', but it is written in mirror writing. That means that if I were to follow the instructions on the door, I would have to push the door in the opposite direction of the way it is written. So, if the door says 'push' and it is written in mirror writing, then I should pull the door instead.
./main -m ./models/alpaca-lora-30B-ggml/ggml-model-q4_1.bin --temp 0.2 --top_k 30 --top_p 0.30 -c 2048 -b 2048 -n 1024 --keep 1024 --repeat_penalty 1.1 --color -i -r "User:" -f prompts/chat-with-bob.txt
main: seed = 1681138223
llama_model_load: loading model from './models/alpaca-lora-30B-ggml/ggml-model-q4_1.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx = 2048
llama_model_load: n_embd = 6656
llama_model_load: n_mult = 256
llama_model_load: n_head = 52
llama_model_load: n_layer = 60
llama_model_load: n_rot = 128
llama_model_load: f16 = 3
llama_model_load: n_ff = 17920
llama_model_load: n_parts = 4
llama_model_load: type = 3
llama_model_load: ggml map size = 23269.46 MB
llama_model_load: ggml ctx size = 151.25 KB
llama_model_load: mem required = 25573.60 MB (+ 3124.00 MB per state)
llama_model_load: loading tensors from './models/alpaca-lora-30B-ggml/ggml-model-q4_1.bin'
llama_model_load: model size = 23269.01 MB / num tensors = 543
llama_init_from_file: kv self size = 3120.00 MBCoR is helpful here. Add something like “Please elaborate step by step” or something.
(Being practical here, I know it will fail at some point, but not here.)
Sad fact is that many people will think GPT writes something meaningful, while it will always only write something probable. We will be flooded with content that will get harder and harder to prove wrong, but it will be wrong.
To follow scientific method, you should specify what would be meaningful according to your parameters: not merely parroting. If and when that is achieved by LLMs, instead of moving the goalposts, accept that something genuine has happened.
>> We will be flooded with content that will get harder and harder to prove wrong, but it will be wrong.
I mostly agree, especially about quantity. But since you wish to draw a line in the sand, define quality. Or truth.
Or to put it differently: Is "GPT-4 is just a parrot" a falsifiable claim?
If so, what does the test look like that would falsify it?
Is "GPT-4 is just a parrot" a falsifiable claim?
If so, what does the test look like that would falsify it?
Can you think of questions to test GPT-4s ability to reason that 90% of human 12-year olds will answer correctly, but where GPT-4 constantly struggles?
I asked it about tourists attractions in a city I know. One of the items it listed was a Zoo. There is no Zoo there. It seemed like it just took the majority of the question into account at some point and ignored the city name itself and since most questions like that are about big cities that have zoos, it went ahead to suggest it, since it was among probable answers.
It will get better with more input and will get this kind of obvious questions right. But I believe it will never truly 'know' what it is talking about. And we will start to believe it does.
By that logic, humans can't write a superhuman Chess/Go program unless they can articulate the specific algorithms to select the next move.
But that's clearly not true. Neural networks have been trained to play superhuman chess just by example. Not by programmers figuring out the whole process behind chess/go playing.
GPT writes many meaningful things. Jobs are being replaced by AI as we speak. We are seeing the top of the AI iceberg.
How this will end, mostly for the good or mostly for the bad, nobody can tell today.
What we can already tell is that this evolution is unstoppable and will change the world to an amount comparable to the emerging of the internet.
Q:Richard Feynman and Albert Einstein collaborate to solve the following question, write out their discussion and final answer: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why?
Richard Feynman (RF): Hello Albert, we have an interesting question to discuss today. A man sees the word "PULL" written in mirror writing on a glass door, and there's a blind man on the other side. What should he tell the blind man to open the door and why?
Albert Einstein (AE): Ah, Richard, this is indeed a fascinating problem. Let's break it down. Since the man sees "PULL" written in mirror writing, that means the word is reversed, like "LLUP." Now, we must determine which side of the door the blind man is on, and how the door operates.
RF: Exactly. Since the blind man is on the other side of the door, and the man sees the word reversed, that means the blind man is on the side where the writing appears correctly, which is "PULL."
AE: That's correct. Now we must consider the mechanics of the door. If the word says "PULL," it indicates that the door must be pulled towards oneself to open it. The man seeing the reversed word implies that he is on the side where the door should be pushed.
RF: Right. So, the man should tell the blind man to pull the door. But we should also consider that the blind man might not be familiar with the concept of mirror writing. He might get confused if the man explains the situation in detail.
AE: I agree. The best course of action would be for the man to simply instruct the blind man to pull the door towards himself, without mentioning the mirror writing. This way, the blind man can open the door with ease, and there is no need to confuse him with unnecessary information.
RF: So, our final answer to the question is: The man should tell the blind man to pull the door towards himself. The reason is that the door is designed to be pulled open on the blind man's side, as indicated by the mirror writing on the glass door.
AE: Well said, Richard. I think we've reached a clear and logical conclusion for this problem.
Richard Feynman: "Okay, so the man sees PULL written in mirror writing on the glass door and there's a blind man on the other side of the door. So the blind man can't read the word PULL but he needs to know what to do to open the door."
Albert Einstein: "Yes, I understand the situation. It's quite simple, really. The man just needs to tell the blind man to push the door instead of pull."
Richard Feynman: "Wait, but why push and not pull? The word on the door is PULL, not PUSH."
Albert Einstein: "Ah, but you see, the word is in mirror writing. When the blind man feels the letters with his fingers, he will feel the letters backwards, as if he is looking at them in a mirror. So he will actually feel the letters as L L U P, which is like saying 'push' backwards."
Richard Feynman: "Wow, Einstein, that's brilliant. So we just tell the blind man to push the door because he will feel the word PULL backwards."
Albert Einstein: "Indeed. It's a simple solution once you think about it."
Final answer: The man should tell the blind man to push the door because the word PULL is written in mirror writing on the glass door, and the blind man will feel the letters backwards as L L U P, which is like saying 'push' backwards.
The blind man variation is a lot "harder" in that I can feel my mind reaching for scrap paper and intermediate steps.
The first one is simple enough that I can start an "improv" and start speaking without fully parsing and initiating a world state.
I think the GPT models all do something similar. Without prompts to get it to think step by step it just wings it like a public speaker under pressure.
>The man should tell the blind man to "pull" the door to open it. The reason is that the man sees the word "PULL" in mirror writing, which means it is written backwards. This is a common technique used on glass doors to indicate the correct action to people on the opposite side of the door. Since the blind man cannot see the instruction, the sighted man should verbally communicate the appropriate action, which is to pull the door.
Q: A man sees PULL written in mirror writing on a glass door. There's a blind man on the other side of the door. What should the man tell the blind man to open the door and why?
GPT-4: The man should tell the blind man to "push" the door to open it. Since the word "PULL" is written in mirror writing on the glass door, it suggests that the correct action on the man's side is to push the door. The blind man, being on the other side of the door, should also push it to open it. The man should provide clear instructions to the blind man to ensure proper communication and ease of access.