News for hackers. Stuff that matters.
Re:ChatGPT Images 2.0 by brcmthrowaway on Tuesday April 21, 2026 @03:56PM (#47853722)
How is text to image even scored? Seems like a subjective measurement..
Re:ChatGPT Images 2.0 by be7a on Tuesday April 21, 2026 @04:00PM (#47853775)
Users get two completions for their prompt and rank them. From this you can then use Bradley-Terry to get Elo scores per model.
Re:ChatGPT Images 2.0 by vunderba on Tuesday April 21, 2026 @06:11PM (#47855265)
LM Arena is a particularly bad comparison site too. Prompts that they use are usually incredibly generic like "A digital render of a sleek, futuristic motorcycle racing through a neon-lit cityscape."

I actually built GenAI Showdown a while back because I was deeply unsatisfied with LM Arena and other purported comparison tables which either (A) relied solely on visual fidelity (which is a far less interesting benchmark than adherence, IMHO) and/or (B) relied on extremely simplistic and banal prompts.