Comparing 60 LLMs with a set of 20 prompts

Re:Comparing 60 LLMs with a set of 20 prompts by smilingemoji on Saturday September 09, 2023 @11:16AM (#37446353)

I took at the look at the SVG smilies, only GPT gets it right (https://benchmarks.llmonitor.com/svg). You can preview the SVGs drawn by the LLMs by pasting this into the browser console

document.querySelectorAll("td pre").forEach((node) => { let code = node.textContent; node.insertAdjacentHTML('afterend', code) })

Or take a look at my screenshot: https://i.ibb.co/Kw0kp58/Screenshot-2023-09-09-at-17-15-20-h...

[ Parent | on HN ]

Re:Comparing 60 LLMs with a set of 20 prompts by ComputerGuru on Saturday September 09, 2023 @12:05PM (#37446931)

Clever use of the dev tools console. I think you’re being a little uncharitable, the prompt said “smiley” and not “smiling emoji” and “smiley” was once upon a time used for the whole slew of emoji. With that in mind, several others also pass.

[ Parent | on HN ]

Re:Comparing 60 LLMs with a set of 20 prompts by dr_kiszonka on Saturday September 09, 2023 @01:08PM (#37447608)

Good point, but you may be too charitable. At least according to Wikipedia, a smiley is supposed to be smiling. https://en.wikipedia.org/wiki/Smiley

[ Parent | on HN ]

Re:Comparing 60 LLMs with a set of 20 prompts by ComputerGuru on Saturday September 09, 2023 @07:29PM (#37451338)

I don’t know about Wikipedia but I’m speaking from loved historical experience.

Here’s a throwback for you: https://www.phpbb.com/customise/db/styles/smilies-13

(All “emoji” images were just called smilies in a lot of software.)

[ Parent | on HN ]

Re:Comparing 60 LLMs with a set of 20 prompts by vincelt on Saturday September 09, 2023 @01:05PM (#37447576)

Oh neat, thanks for sharing, wanted to add an interpreter to that test

[ Parent | on HN ]