News for hackers. Stuff that matters.

A comment on Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?. Parent comment

Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by michaelt on Friday August 08, 2025 @05:02PM (#44841654)
Why stop at 80 H100s for a mere 6.4 terabytes of GPU memory?

Supermicro will sell you a full rack loaded with servers [1] providing 13.4 TB of GPU memory.

And with 132kW of power output, you can heat an olympic-sized swimming pool by 1°C every day with that rack alone. That's almost as much power consumption as 10 mid-sized cars cruising at 50 mph.

[1] https://www.supermicro.com/en/products/system/gpu/48u/srs-gb...

[ Parent | on HN ]
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by handfuloflight on Friday August 08, 2025 @07:26PM (#44842739)
What about https://www.cerebras.ai/system?
[ Parent | on HN ]
Re:Ask HN: How can ChatGPT serve 700M users when I can't run... by procaryote on Saturday August 09, 2025 @04:11AM (#44844860)
> as much power consumption as 10 mid-sized cars cruising at 50 mph

Imperial units are so weird

[ Parent | on HN ]