News for hackers. Stuff that matters.

A comment on Show HN: YPerf – Monitor LLM Inference API Performance.

Re:Show HN: YPerf – Monitor LLM Inference API Performance by Oras on Monday December 23, 2024 @04:22AM (#42493049)
Nice one. It would be great to use filtering. For example, I want to check the TPS of Llama 3.3 across multiple providers.
[ Parent | on HN ]
Re:Show HN: YPerf – Monitor LLM Inference API Performance by xjconlyme on Monday December 23, 2024 @06:11AM (#42493552)
[Updated] I've added the filtering and the multiple provider comparison!

---

Great suggestion!

I currently pick the fastest TPS among providers, and you can see a detail performance list by clicking the <Learn More> icon at the last column. For example, here's the detailed OR page of Llama 3.3: https://openrouter.ai/meta-llama/llama-3.3-70b-instruct

I'll add the filtering soon.

[ Parent | on HN ]