OpenAI’s CEO Says the Age of Giant AI Models Is Already Over

Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by narrator on Monday April 17, 2023 @06:47PM (#35607672)

"Altman said there are also physical limits to how many data centers the company can build and how quickly it can build them."

An H100 has 80GB of VRAM. The Highest end system I can find is 8xH100. Is a 640GB model is the biggest model you can run on a single system? Already GPT-4 is throttled and has a waiting list and they haven't even released the image processing or integrations to a wide audience. Maybe they are just unable to do inference in a cost-efficient manner and at an acceptable speed on anything bigger than GPT-4?

[ Parent | on HN ]

Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by moralestapia on Monday April 17, 2023 @07:42PM (#35608126)

>Maybe they are just unable to do inference in a cost-efficient manner and at an acceptable speed on anything bigger than GPT-4?

Yes, that's the whole thing. As others have pointed out, GPT-4 seems like an optimum point that balances cost, ROI, etc... it won't improve much just by throwing more data at it.

[ Parent | on HN ]

Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by forrest2 on Monday April 17, 2023 @07:52PM (#35608205)

You might not see it available via self serve interfaces, but I think the h100 cards can do 256x to a single host.

[ Parent | on HN ]

Re:OpenAI’s CEO Says the Age of Giant AI Models Is Already Over by narrator on Tuesday April 18, 2023 @07:31AM (#35612564)

You're right! You can nvlink 32 systems with 8 h100s! That's 20 terabytes of VRAM total!

[ Parent | on HN ]