News for hackers. Stuff that matters.

A comment on Anthropic Claude performance depends heavily on prompt style. Parent comment

Re:Anthropic Claude performance depends heavily on prompt style by peyton on Thursday December 07, 2023 @02:12AM (#38553718)
The appropriations bill example also looks right—the insertion doesn’t stylistically match the rest of the document. I’m much more skeptical of evaluations if this is how the sausage gets made. Feels like bullshit artistry.
[ Parent | on HN ]
Re:Anthropic Claude performance depends heavily on prompt style by jafitc on Thursday December 07, 2023 @06:38AM (#38555316)
These are not actual tests they used for themselves.

Some third party did these tests first (in article and spread on social) to which the makers of Claude are responding.

I knew it’s a weird test right when I first encountered it.

Interesting that the Claude team felt like it’s worth responding to.

[ Parent | on HN ]