NanoChat – The best ChatGPT that $100 can buy

Re:NanoChat – The best ChatGPT that $100 can buy by echelon on Monday October 13, 2025 @05:13PM (#45573391)

8xH100 is pretty wild for a single inference node.

Is this what production frontier LLMs are running inference with, or do they consume even more VRAM/compute?

At ~$8/hr, assuming a request takes 5 seconds to fulfill, you can service roughly 700ish requests. About $0.01 per request.

Is my math wrong?

[ Parent | on HN ]

Re:NanoChat – The best ChatGPT that $100 can buy by vessenes on Monday October 13, 2025 @05:14PM (#45573403)

This is the spec for a training node. The inference requires 80GB of VRAM, so significantly less compute.

[ Parent | on HN ]

Re:NanoChat – The best ChatGPT that $100 can buy by andai on Tuesday October 14, 2025 @06:07AM (#45578195)

The default model is ~0.5B params right?

[ Parent | on HN ]

Re:NanoChat – The best ChatGPT that $100 can buy by Tepix on Monday October 13, 2025 @05:32PM (#45573568)

As vessenes wrote, that‘s for training. But a H100 can also process many requests in parallel.

[ Parent | on HN ]