News for hackers. Stuff that matters.

A comment on Cerebras-GPT: A Family of Open, Compute-Efficient, Large Language Models. Parent comment

Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ... by sbierwagen on Tuesday March 28, 2023 @02:16PM (#35345461)
CS-1 cost "$2-3 million", CS-2 costs "several" million.

A single Nvidia H100 costs somewhere around $30,000 each, so a GPU server with every slot populated costs about $300,000.

[ Parent | on HN ]
Re:Cerebras-GPT: A Family of Open, Compute-Efficient, Large ... by brucethemoose2 on Tuesday March 28, 2023 @02:42PM (#35345875)
ServeTheHome claims "HGX A100 platforms, when they are sold as single servers are generally in the $130K-$180K even leaving a very healthy margin for OEMs/ resellers"

https://www.servethehome.com/graphcore-celebrates-a-stunning...

Not sure about the H100, but it seems to be more supply constrained (hence pricier) atm.

Now, the real question is how many HGX nodes "equals" a single CS2 node. The math here is extremely fuzzy, as the benefit to such extreme node consolidation depends on the workload, and the CS-2 takes up less space, but the HGX cluster will have more directly accessible RAM and better turnkey support for stuff since its Nvidia.

[ Parent | on HN ]