News for hackers. Stuff that matters.

A comment on ESPN AI-generated recap of retiring player's final match fails to mention them. Parent comment

Re:ESPN AI-generated recap of retiring player's final match ... by jetrink on Monday September 09, 2024 @12:40PM (#41490330)
I got nerd-snipped by this. I transcribed the match using the whisper-small.en and then asked ChatGPT to create a summary using a neutral prompt:

Here is a transcript of a soccer match. In the style of an experienced professional sports reporter, please write a 200 word article about the match.

Its summary starts, "In her final professional match, Alex Morgan delivered a performance filled with emotion and resilience, though her San Diego Wave fell short in a 3-1 loss to North Carolina Courage. The game at Snapdragon Stadium in San Diego was more than just a contest; it was a tribute to one of soccer’s most iconic figures."

It did get the score wrong. Here's the rest:

1. https://chatgpt.com/share/de8c60d1-69ab-4291-99dc-d4d95af3d3...

[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by regretaverse on Monday September 09, 2024 @12:54PM (#41490526)
OpenAI transcription & LLM is likely more costly than what they're willing to splurge on this. Care to link soccer.txt, so I can try with Llama 3 / 3.1?
[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by jetrink on Monday September 09, 2024 @01:17PM (#41490873)
Here you go: https://pastebin.com/QXzg95AV

One thing to try is only using the post-match commentary, which starts after the line containing 'final whistle.' When I did this, the result was more factually accurate, while still focusing on Morgan.

https://chatgpt.com/share/9c122702-b46f-4e5e-bd05-e063a74126...

[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by whimsicalism on Monday September 09, 2024 @01:05PM (#41490700)
too costly? it’s probably a few cents total
[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by brewdad on Monday September 09, 2024 @01:31PM (#41491054)
And it got the most basic detail, the score of the game, wrong. Even at the cost of only a few cents it's worthless.
[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by whimsicalism on Monday September 09, 2024 @02:24PM (#41491720)
seems like this is a solvable problem with just a bit more engineering effort, but yeah... totally worthless
[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by acchow on Monday September 09, 2024 @03:46PM (#41492727)
You should just need a better prompt. I think everyone would benefit from using a standardized prompt which asks the model to think through its work between `<thought>` tags before writing its response, and also reflecting on the response between `<reflection>` tags, and then outputting the final response afterwards
[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by tantalor on Monday September 09, 2024 @04:28PM (#41493322)
Easy, just instruct the LLM to not hallucinate the score, problem solved.
[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by low_tech_love on Monday September 09, 2024 @12:56PM (#41490545)
That’s a very good write up, so good it’s depressing. Am I the only one who is utterly uninterested in reading anything that an AI writes?
[ Parent | on HN ]
Re:ESPN AI-generated recap of retiring player's final match ... by iainmerrick on Monday September 09, 2024 @01:14PM (#41490824)
If it got the score wrong, it’s not a very good writeup.
[ Parent | on HN ]