News for hackers. Stuff that matters.
Re:Running a 180B parameter LLM on a single Apple M2 Ultra by adam_arthur on Thursday September 07, 2023 @01:55PM (#37422898)
GPT-4 cost on the order of $100 million, per Sam Altman.

This is orders of magnitude lower than many companies and government R&D budgets. It's easily financeable by 1000s of independently wealthy people and organizations. It's easily financeable by VC money. This is far cheaper than many other startups or product initiatives that have been tried. There are very likely to be many organizations that build models for the specific purpose of open sourcing the resulting model... the Falcon and Llama models are already proof enough of this

Costs to train equivalent models may increase in the short term due to race towards GPU consumption raising costs... but compute will get cheaper in aggregate over time due to improving compute tech.

And once the model is built it is largely a sunk cost, yes. All that needs to happen is for a single SoTA model to be made open to completely negate any advantage a competitor has. Monetization from LLMs will be driven by focused application of the models, not from providing an interface to a general model. High quality data holds more value than the resulting model

Not every query requires timeliness of data. Incorporating new data into an existing model is likely to be cheaper than retraining the model from scratch, but just speculation on my end.

Re:Running a 180B parameter LLM on a single Apple M2 Ultra by foobiekr on Friday September 08, 2023 @12:46AM (#37429443)
I think you are overestimating R&D budgets for companies. Very few tech companies - even large ones - have R&D budgets in the $10B+ range, let alone $100B. Most of the fortune 100 isn't even $10B.
Re:Running a 180B parameter LLM on a single Apple M2 Ultra by danielbln on Friday September 08, 2023 @02:12AM (#37430039)
Where do you get $100B from?
Re:Running a 180B parameter LLM on a single Apple M2 Ultra by foobiekr on Friday September 08, 2023 @12:43PM (#37435993)
"orders of magnitude"