News for hackers. Stuff that matters.

A comment on OpenAI’s CEO Says the Age of Giant AI Models Is Already Over. Parent comment

Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by MichaelZuo on Monday April 17, 2023 @09:41PM (#35608967)
Exponential scaling for a presumable GPT-5 suggests it's response time will be unusably long for the vast majority of use cases, and probably cost multiple dollars USD per query.

Not to mention there doesn't actually exist enough English text data in the world to even double GPT-4's training set.

[ Parent | on HN ]
Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by epups on Monday April 17, 2023 @09:57PM (#35609076)
Compute will also scale exponentially in coming years. The data source limitation seems to be a harder barrier, I think many companies are experimenting with AI generated content for training at this point.
[ Parent | on HN ]
Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by MichaelZuo on Monday April 17, 2023 @10:45PM (#35609364)
> Compute will also scale exponentially in coming years.

Cost per transistor scaling has already plateaued or perhaps even inverted with TSMC's latest and greatest.

And the new chips, even after 25 layers of EUV lithography, more than doubling the previous record, and an extra year of fine tuning, has total SRAM size scaling of -5% and logic scaling of -42%.

These are numbers verified by experienced semi people.

[ Parent | on HN ]