News for hackers. Stuff that matters.
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by hnfong on Wednesday October 11, 2023 @09:28PM (#37852436)
Never seen a hardware failure, I see.
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by ninkendo on Thursday October 12, 2023 @06:31AM (#37855433)
The snark is getting a bit tiring. No, plenty of hardware failures but we had redundancy. Our availability wasn’t dependent on how fast I cold drive to the data center at 3am. Drive dies, who cares, there are plenty of hot spares, we’ll deal with it during business hours. Server dies, who cares, there are lots of them. We have remote hands too, so simple hardware replacement is something you can get cheap onsite labor to do.

If your operations halt until some poor sysadmin has to drive to the colo, you are absolutely doing it wrong.