Meta is inviting researchers to pick apart the flaws in its version of GPT-3

Re:Meta is inviting researchers to pick apart the flaws in i... by zubspace on Monday June 27, 2022 @02:46PM (#31898909)

I have no knowledge of such things, but it seems they run Cuda jobs on about 150 nodes?

But why do they have so many problems to keep this cluster stable? Network failures? Bad GPU's? Bad drivers? Bad software?

Running fixmycloud and going after all those cryptic errors every day seems like a nightmare to me...

[ Parent | on HN ]

Re:Meta is inviting researchers to pick apart the flaws in i... by semi-extrinsic on Tuesday June 28, 2022 @03:32AM (#31905141)

Seems kind of par for the course for an HPC cluster, no?

It makes sense to think of these things like Formula 1 cars, they are trying to eke out the absolute maximum performance, and reliability suffers because of that.

"Ordinary" cloud is more like a Toyota where you optimize for fuel economy and low maintenance.

[ Parent | on HN ]