Could you train a ChatGPT-beating model for $85,000 and run it in a browser?

Re:Could you train a ChatGPT-beating model for $85,000 and r... by dekhn on Friday March 31, 2023 @03:06PM (#35391832)

it's hard to tell from what you're saying: you're planning on putting an ML infrastructure training server on a regular 15A circuit, not in a data center or machine room? And power is paid for by somebody else?

My thinking about pricing doesn't include that option because I wouldn't just hook a server like that up to a regular outlet in an office and use it for production work. If that works for you- you can happily ignore my comments. But if you go ahead and build such a thing and operate it for a year, please let us know if there were any costs- either dollar or in suffering- associated with your decision

[edit: adding in that the value of this machine also suggests it cannot live unattended in an insecure location, like an office]

signed, person who used to build closet clusters at universities

[ Parent | on HN ]

Re:Could you train a ChatGPT-beating model for $85,000 and r... by KeplerBoy on Friday March 31, 2023 @03:24PM (#35392124)

Nvidia happily sells what you're describing. They call it "DGX Station A100", it has 4 80GB A100 and retails for 80k. Not sure i believe their claimed noise level of <37 dB though.

Of course that's still a very small system when talking LLM training, the only reason why i would not put that in a regular office is it's extreme price. Do you really want something worth 80k in a form factor that could be casually carried through the door?

[ Parent | on HN ]

Re:Could you train a ChatGPT-beating model for $85,000 and r... by amluto on Friday March 31, 2023 @05:08PM (#35393692)

If you live near an inexpensive datacenter, you can park it there. Throw in a storage machine or two (TrueNAS MINI R looks like a credible low-effort option). If your workload is to run a year long computation on it and otherwise mostly ignore it, then your operational costs will be quite low.

Most people who rent cloud servers are not doing this type of workload.

[ Parent | on HN ]

Re:Could you train a ChatGPT-beating model for $85,000 and r... by PeterisP on Friday March 31, 2023 @08:45PM (#35395787)

Why couldn't a 75k machine live unattended in an office? If the same office has just a hundred employee workstations, those in total are worth much more than that, heck, Apple offered a Mac Pro configuration that was $50k for a single workstation.

[ Parent | on HN ]