declanjackson
Re:Perplexity Decider v1.3: Training, eval, and use cases
on Friday November 13, 2026 @06:01PM (#50491378)
But isn't there still a meaningful spread of difficulty, risk tolerance, etc. in decision model relevant tasks? Especially at high volumes I think there would still be value in being able to make intelligence/cost/speed/calibration tradeoffs between models
Re:Perplexity Decider v1.3: Training, eval, and use cases
on Friday November 13, 2026 @05:49PM (#50491233)
Artificial Analysis has an Index which also includes Cost per Task, Time per Task, accuracy, calibration etc. here: https://artificialanalysis.ai/evaluations/artificial-analysi...
(caveat, I work at AA and helped build this Index)
Re:Gemini 4 Hydrogen
on Thursday November 12, 2026 @08:16PM (#50476847)
This is based on what we measure from the API - we see different endpoints depending on how they are served can have different cache hit rates (cache hit rate is defined as the % of cacheable tokens that actually come back as cache hits)
Re:Gemini 4 Hydrogen
on Thursday November 12, 2026 @05:49PM (#50475524)
Hi there - AA employee here. This is due to 3.8 Flash having a much lower cache hit rate compared to Gemini 4 Hydrogen
Re:Claude Haiku 5.5
on Wednesday October 07, 2026 @03:19PM (#49997533)
According to AA benchmarks, it uses 162k output tokens per task (with max reasoning) - over double GLM-5.3 Flash for similar level of Intelligence
Re:Qwen3.8-27B
on Friday August 14, 2026 @05:54PM (#49305058)
With max reasoning, Luna is actually less than half the actual cost to run compared to DeepSeek V4 Flash 0731 with updated prices (based on Artificial Analysis Cost per Task)