felix089

I beat Jev, but jeez the leaderboard is pretty competitive...

cool, send a PR we merge it into the leaderboard :)

Nice! what was your score on jevman?

Good timing with our pac-man benchmark for decision models, now also including Microsoft Decision-1: https://news.ycombinator.com/item?id=50007993

Someone actually ran the benchmark on Qwen 3.8 Flash Next NVFP4, which is a chat model and submitted it via PR. We merged and it's live on the leader board in the top 3.

Quick update, we doubled the models tested based on community contributions, and a new model took the #1 spot. also well over 1000 games have been played against decision model ghosts. https://opper.ai/jevman-benchmark/#leaderboard

Ideally yes but that would not work with the model latency. But it is already partly covered when a ghost enters pacmans corridor he is asked again. Here's how the decisions work in detail https://github.com/opper-ai/jevman-benchmark/blob/main/READM...

Great to hear, if it's about inference credits we actually built a thing called an AI wallet where users can just easily pay for their own inference on apps. if that's interesting lmk, id be happy to support this https://opper.ai/ai-wallet

Not yet but the repo is open source and set up so that you can benchmark your own model and we can add it to the leaderboard, would be cool to see if Jeffy can play: https://github.com/opper-ai/jevman-benchmark/blob/main/CONTR...

Thank you and yea agreed fixing it now, try again in an hour from now :)

Maybe the next summer hit, peak AI

Thanks yea same, it is both very convincing but at the same time also open to changing its positing when presented with a good argument.

Hey HN! About 2 months ago I launched AI Roundtable here, where 200+ models answer and debate your question (https://news.ycombinator.com/item?id=47507666).

It's now collected 29,605 public sessions and 336,039 individual model responses, and we just published all the stats for everyone to see, updating them daily.

A few highlights:

- In multi-round debates, Claude Opus 4.7 convinced other models to flip their vote almost 3K times, the most of any model. Gemini 3.1 Pro came in second at 2.1K

- Most used model is Gemini 3.1 Pro at 25K sessions, with GPT-5.4 second at 21K.

- Grok 4.1 Fast held its position 88.7% of the time, the highest conviction rate of all models. Probably not surprising.

It's been quite amazing to see all the questions and feedback since launch. Initially the only mode was structured answers (vote yes/no or pick from custom options).

Based on feedback we've added an open questions mode where models answer freely and a roundtable chat where you can join the debate and follow up with individual models to challenge their reasoning.

If you want to give the roundtable a try, it's free to use until community credits run out. All models routed via my startup Opper. Happy to dig into specifics or make more data available if interesting.

Hey HN! About 2 months ago I launched AI Roundtable, where 200+ models answer and debate your question (https://news.ycombinator.com/item?id=47507666).

It's now collected 29,502 public sessions and 334,589 individual model responses, and we just published all the stats for everyone to see, updating them daily.

A few highlights:

- In multi-round debates, Claude Opus 4.7 convinced other models to flip their vote almost 3K times, the most of any model. Gemini 3.1 Pro came in second at 2.1K - Most used model is Gemini 3.1 Pro at 25K sessions, with GPT-5.4 second at 21K. - Grok 4.1 Fast held its position 88.7% of the time, the highest conviction rate of all models. Probably not surprising.

It's been quite amazing to see all the questions and feedback since launch. Initially the only mode was structured answers (vote yes/no or pick from custom options).

Based on feedback we've added an open questions mode where models answer freely and a roundtable chat where you can join the debate and follow up with individual models to challenge their reasoning.

If you want to give the roundtable a try, it's free to use until community credits run out. All models routed via my startup Opper. Happy to dig into specifics or make more data available if interesting.

Hey just fyi the open question feature is now live. Also gave the UI a facelift. Any feedback welcome! Also got a custom domain for easy access: https://askroundtable.ai

It's now live, give it a spin!

Nice! Opus in general is the best debater so far, most models cited Opus for changing their opinion, by a considerable margin.

Cool question! just a quick headsup, they don't have access to tools so what you are seeing are answers based on their training data. They might not know about the latest model version. That said, sonnet is def a great choice.

It's so funny to see the smaller / first gen models make the wrong choices despite overwhelming evidence, almost adorable. I ran the same test with one model from each GPT generation, all but 3.5 Turbo could be convinced. https://opper.ai/ai-roundtable/questions/i-want-to-wash-my-c...

haha good to hear, then the latest update on the roundtable history list seems to work well and the good ones are on top

Thanks, yes this is coming shortly!

Two models changed their minds but from opposite sides so the score stayed the same, that's the first time I've seen this.

Okay since the launch we got about 5k questions asked to the roundtable, really cool stuff! We had much higher usage than expected and had to scale up to keep things running. Thanks for all the feedback, shipped a bunch of updates during the day. Now the history tab has a much better sorting logic, added upvotes, and more filters. You can create final summaries in a couple of voices, which is quite funny I think. There's a couple more things coming shortly, like open questions mode and potentially joining as a participant in the roundtable. Any other feedback just let me know. Thanks!

You can basically already do that, all you need is to create your own API key and put it in navbar/API key. Then all your sessions are unlisted so unless someone has the link nobody will should be able to find it. You can still share them with others if you like. Like unlisted yt videos.

Okay it's done, all fixed!