Jeeves. Reasoning improves Jev-like decision models

Re:Jeeves. Reasoning improves Jev-like decision models by TN1ck on Tuesday September 29, 2026 @09:39AM (#49893066)

I just did a run with a benchmark I just used to test other models against. (It's about detecting irony in german soccer tweets). On my M5 Pro with 48GB it took over 30min to decide on just 100 tweets, the thinking definitely takes long.

It performed quite below Jev, but above other open decision models I tested (68 correct vs 79 correct for Jev - see [1]). I'm running it for the moderation benchmark as well, but that will probably take a few hours on my machine.

[1] https://tn1ck.com/blog/jevdit

[ Parent | on HN ]

Re:Jeeves. Reasoning improves Jev-like decision models by TN1ck on Tuesday September 29, 2026 @01:57PM (#49897575)

Update: Jeeves took about 2 hours to moderate 394 data points and performed really well. It’s not as good as Jev, but it’s super close! In general, it’s super cool that you can tune how strict you want content moderation to be with these models.

[ Parent | on HN ]

Re:Jeeves. Reasoning improves Jev-like decision models by nicowaltz on Tuesday September 29, 2026 @02:06PM (#49897760)

cool to see!

[ Parent | on HN ]

Re:Jeeves. Reasoning improves Jev-like decision models by robbie-c on Wednesday September 30, 2026 @05:06AM (#49906269)

One of Nico's colleagues here, I'm working on an MPS port https://github.com/PostHog/jeeves/pull/1 which should speed up M5 Pro performance quite a bit

[ Parent | on HN ]