News for hackers. Stuff that matters.

A comment on JetMoE: Reaching LLaMA2 performance with 0.1M dollars.

Re:JetMoE: Reaching LLaMA2 performance with 0.1M dollars by patrick-fitz on Thursday April 04, 2024 @05:09PM (#39935836)
Out of curiosity, looking at the cheapest price for a H100 that I could find online.

Lambda Reserved Cloud [1] starts at $1.89 per H100 per hour.

It could be possible to get the cost down to a lower amount:

$1.89 * 96GPUs * 24hours * 14days = ~$61k

1 - https://lambdalabs.com/deep-learning/servers/hyperplane

[ Parent | on HN ]
Re:JetMoE: Reaching LLaMA2 performance with 0.1M dollars by avrionov on Thursday April 04, 2024 @06:30PM (#39936602)
This is the price of training if nothing fails.
[ Parent | on HN ]
Re:JetMoE: Reaching LLaMA2 performance with 0.1M dollars by logicchains on Friday April 05, 2024 @02:51AM (#39939407)
It also depends on the interconnect speed. If you don't have fast enough interconnect between the machines, you won't get linear speedup with N gpus.
[ Parent | on HN ]