News for hackers. Stuff that matters.
Re:We’re releasing Jev 1.14.0 along with experimental featur... by ranger_danger on Wednesday October 14, 2026 @05:37PM (#50095707)
Which others?
Re:We’re releasing Jev 1.14.0 along with experimental featur... by benaduggan on Wednesday October 14, 2026 @05:46PM (#50095822)
There are several public benchmarks for decision models where they don't top every single one. https://benchmarkheaven.com/jev-models/all

I guess their cheeky "Warning: AI Benchmarks, don't trust them" is their way out of that one.

Re:We’re releasing Jev 1.14.0 along with experimental featur... by woadwarrior01 on Wednesday October 14, 2026 @06:59PM (#50096733)
The way that site charges a fee[1] for benchmarking models, makes any benchmarks on it very sus, IMO. TBF, the benchmarking site is probably more profitable than TypeSafe, just like the joke about how WinRAR is more profitable than OpenAI[2]. :)

[1]: https://benchmarkheaven.com/submit

[2]: https://news.ycombinator.com/item?id=49797811

Re:We’re releasing Jev 1.14.0 along with experimental featur... by Maxious on Wednesday October 14, 2026 @10:27PM (#50098404)
https://huggingface.co/spaces/multimodalart/jev-decision-ind... is also ranking just for the love of the game.

Given that benchmarks are useful, why not give them economic value?

Re:We’re releasing Jev 1.14.0 along with experimental featur... by woadwarrior01 on Thursday October 15, 2026 @04:36AM (#50100909)
There's nothing wrong with them wanting to capture some economic value, but the way they're monetizing it is a slippery slope of incentives.

OTOH, I trust the benchmarks from Apolinario's HF space that you've linked to.