Re:Natural language is the lazy user interface
by AlotOfReading on Saturday January 28, 2023 @12:39AM
(#34555493)
It's just doing really good pattern recognition, not running the computation in any meaningful way. We can test this by finding an attractor in the latent space and asking the model to compute something "near" it. Here's an example that ChatGPT consistently gets wrong: computing the number of years between dates.
The exact result I get for this varies, but it's consistently off by 1 or 2 depending on year 0 handling. Here's a representative example of the code:
If you run this with arguments, you'll get the correctly computed, incorrect results. A key point to note here is that ChatGPT is very stubborn about incorrectly handling year 0. We can abuse that by asking ChatGPT to calculate the timespan using a corrected function. If it's "understanding", it should compute the correct result. If it's operating as a stochastic parrot, it will give us an incorrect one.
Hopefully that serves to demonstrate the point. As an interesting aside, the subsequent plaintext explanation of the code actually computes the right answer.
[ Parent
| on HN ]
There are no comments on this comment yet.