News for hackers. Stuff that matters.
Re:Estimating PaLM's Training Cost by axg11 on Sunday April 10, 2022 @01:13PM (#30979196)
There are 67 authors on the PaLM paper. Assuming average salary of $150k (probably a large underestimate for Google employees) that's $10M+/year in salary alone. Compute is a big cost factor for this project but it's not clear if it even dominates the total cost.

In the long run this type of research will more than pay itself back in benefits to Google. NLP underpins everything they do. It would be interesting to see how OpenAI API (GPT-3) is doing in terms of revenue. They're going for the more direct method of seeking value from a trained large language model.

Re:Estimating PaLM's Training Cost by the8472 on Sunday April 10, 2022 @01:29PM (#30979401)
The model itself didn't take all those authors a full-time work year to build from scratch. They build other models and release other papers throughout the year. Of course those works build on each other, but that turns it into a much more complicated question how we account for the value and costs of prior work.
Re:Estimating PaLM's Training Cost by axg11 on Sunday April 10, 2022 @02:05PM (#30979831)
To counter that, there were likely other people with smaller contributions not listed as authors. Some of the leadership are likely earning 2-3x my estimate salary.
Re:Estimating PaLM's Training Cost by platers on Sunday April 10, 2022 @03:23PM (#30980725)
Everyone on the paper is likely making at least 300k. And double that again for taxes, healthcare, and perks.
Re:Estimating PaLM's Training Cost by hansvm on Sunday April 10, 2022 @06:35PM (#30982409)
Double is a good rule of thumb for a median liquidatable income, but healthcare, food, gyms, 401k matching, ... are all bounded by reasonable constants, and taxes only add <10% from the employer's side of things. I'd be surprised if the fully loaded cost were more than an extra 50%.
Re:Estimating PaLM's Training Cost by marczoid on Sunday April 10, 2022 @03:17PM (#30980667)
The fact that there are 67 people on the paper doesn’t mean they all worked on it full time. In fact, I suppose less than 5 did. I am quite certain most people only spent a tiny fraction of their time on this project. (Not saying the outcome isn’t impressive, it’s just how these projects tend to go.)
Re:Estimating PaLM's Training Cost by modeless on Sunday April 10, 2022 @04:36PM (#30981439)
Large underestimate is an understatement. The average fully loaded cost of these employees may be an order of magnitude higher than your guess. Jeff Dean alone probably costs Google $10m per year.