News for hackers. Stuff that matters.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by exikyut on Thursday February 18, 2021 @12:23AM (#26176229)
Use x1e.32xlarge on AWS with 3TB of RAM. Just $12,742/mo - https://calculator.aws/#/estimate?id=7428fa81192c57087ac8cdf...

Alternatively order something like the HP Z8 with 3TB RAM configured, which is only $75k - https://zworkstations.com/configurations/2040422/

It's interesting. It would take ~six years for the Z8 to break even compared to AWS, but traffic into and out of the machine would be $0, and I don't think you're running directly on the metal with AWS, so performance would probably be a bit higher. And then there's storage - I configured, uhh, 120TB of a mixture of SSDs and HDDs. I'm not even going to try and ask AWS for a comparible quote there.

I may or may not have added dual Xeon Platinum 8280s to the Z8 as well. :P

Re:Google Open-Sources Trillion-Parameter AI Language Model ... by tjbiddle on Thursday February 18, 2021 @03:42AM (#26177445)
When you're spending that kind of money on a machine, there's no way you're paying retail price. Sales reps would give you a significant discount.

Also - think you meant 6 months, not 6 years anyhow :)

Re:Google Open-Sources Trillion-Parameter AI Language Model ... by exikyut on Thursday February 18, 2021 @05:33AM (#26178223)
Interesting. I'm very curious... 20%? 35%?

And I did mean 6 months, woops. Didn't even notice...

Re:Google Open-Sources Trillion-Parameter AI Language Model ... by jsnell on Thursday February 18, 2021 @12:53AM (#26176439)
> It would take ~six years for the Z8 to break even

Do you mean six months?

Re:Google Open-Sources Trillion-Parameter AI Language Model ... by exikyut on Thursday February 18, 2021 @05:29AM (#26178193)
Oh *dear*. I definitely tripped over there, and I didn't even notice.

Yup.

Re:Google Open-Sources Trillion-Parameter AI Language Model ... by high_byte on Thursday February 18, 2021 @06:50AM (#26178764)
Z8 sounds like fun. but I might just buy two teslas (roadster and X, or a cybertruck) and a gaming PC. :D
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by visarga on Thursday February 18, 2021 @01:00AM (#26176482)
hate to break the party but this model only loads a small part of itself in RAM when inferencing
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by exikyut on Thursday February 18, 2021 @05:31AM (#26178203)
That's a good thing. Less completely means more energy for interestingness, and less expense means more accessibility.