News for hackers. Stuff that matters.
Re:Kimi-K3 Technical Report [pdf] by FuriouslyAdrift on Monday July 27, 2026 @02:36PM (#49073838)
We're running Kimi 2.8 on a $107k server and getting around 50k tokens per second or better on most things.

We've already saved money compared to last years token cost on Claude/Gemini

Re:Kimi-K3 Technical Report [pdf] by logicallee on Monday July 27, 2026 @04:18PM (#49075019)
Are you developing software? Is most of it used on a coding agent? (Like Claude Code or ChatGPT Codex?) If so, what coding agent do you use? If you're not developing software what do you use it for (roughly)?
Re:Kimi-K3 Technical Report [pdf] by FuriouslyAdrift on Tuesday July 28, 2026 @05:11PM (#49089959)
Our use cases are data analysis, software development, general AI chat with RAG so far.
Re:Kimi-K3 Technical Report [pdf] by mdp2021 on Monday July 27, 2026 @03:57PM (#49074808)
> We're running Kimi 2.8 on a $107k server

Equipped with what? Is it CPU based inference, a mix...?

Re:Kimi-K3 Technical Report [pdf] by wongarsu on Monday July 27, 2026 @05:01PM (#49075477)
Not GP, but my educated guess is that they are running a system with between 4 and 6 MI325 or MI355x or similar AMD GPUs. With the 50k tps as the total figure for all parallel requests. Those cards have a lot of memory for their price, allowing you to push to really high batch sizes while still having a large context size for each request
Re:Kimi-K3 Technical Report [pdf] by FuriouslyAdrift on Tuesday July 28, 2026 @05:10PM (#49089952)
4x MI300A in a Gigabyte server
Re:Kimi-K3 Technical Report [pdf] by freediddy on Monday July 27, 2026 @04:21PM (#49075042)
how many requests per second can the server take?
Re:Kimi-K3 Technical Report [pdf] by FuriouslyAdrift on Tuesday July 28, 2026 @05:13PM (#49089979)
That's about it across the whole server. Kimi 2.8 is about as big as we can go. Kimi 3 is massive.
Re:Kimi-K3 Technical Report [pdf] by ajju on Monday July 27, 2026 @04:24PM (#49075077)
What is your config?
Re:Kimi-K3 Technical Report [pdf] by retinaros on Tuesday July 28, 2026 @10:14AM (#49084214)
on 1 paralel request maybe
Re:Kimi-K3 Technical Report [pdf] by FuriouslyAdrift on Tuesday July 28, 2026 @05:12PM (#49089972)
Correct but even with 30 simultaneous users, we're not seeing any noticable lag compared to AIaaS even with agentic workloads.