News for hackers. Stuff that matters.

A comment on Could you train a ChatGPT-beating model for $85,000 and run it in a browser?. Parent comment

Re:Could you train a ChatGPT-beating model for $85,000 and r... by KeplerBoy on Friday March 31, 2023 @03:24PM (#35392124)
Nvidia happily sells what you're describing. They call it "DGX Station A100", it has 4 80GB A100 and retails for 80k. Not sure i believe their claimed noise level of <37 dB though.

Of course that's still a very small system when talking LLM training, the only reason why i would not put that in a regular office is it's extreme price. Do you really want something worth 80k in a form factor that could be casually carried through the door?

[ Parent | on HN ]
Re:Could you train a ChatGPT-beating model for $85,000 and r... by amluto on Friday March 31, 2023 @05:08PM (#35393692)
If you live near an inexpensive datacenter, you can park it there. Throw in a storage machine or two (TrueNAS MINI R looks like a credible low-effort option). If your workload is to run a year long computation on it and otherwise mostly ignore it, then your operational costs will be quite low.

Most people who rent cloud servers are not doing this type of workload.

[ Parent | on HN ]