News for hackers. Stuff that matters.

A comment on GPT-4.5 or GPT-5 being tested on LMSYS?.

Re:GPT-4.5 or GPT-5 being tested on LMSYS? by vincelt on Monday April 29, 2024 @01:55PM (#40201686)
Still fails to solve this one:

Prompt:

> Complete the following Python program:

```

len, print = print, len

def print_len(x):

     "Print the length of x"
```

gpt2-chatbot:

```

len, print = print, len # Swap the functionalities of print and len

def print_len(x): "Print the length of x"

    len(x)  # This 'len' is actually 'print' because of the swap above
print_len("Hello, world!") # This will output: 13

```

[ Parent | on HN ]
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by rany_ on Monday April 29, 2024 @04:10PM (#40203530)
That's the same output as gpt-4-turbo for me, so no real improvement for that particular prompt.
[ Parent | on HN ]
Re:GPT-4.5 or GPT-5 being tested on LMSYS? by hgomersall on Monday April 29, 2024 @02:53PM (#40202396)
That's not bad though!
[ Parent | on HN ]