News for hackers. Stuff that matters.
Re:NanoGPT by londons_explore on Wednesday January 11, 2023 @07:34AM (#34337687)
> very little money in optimizing deep learning

Oh - there are a lot of people working on optimizing AI. Amongst hobbyists, academia, and corporations alike.

The thing is, if you come up with a neat optimization that saves 30% of compute for the same results, typically instead of reducing your compute budget 30%, you instead increase your model/data size 30% and get better results.

Re:NanoGPT by narrator on Wednesday January 11, 2023 @11:12AM (#34340015)
Jevon's paradox of data and AI. The more efficiently data is used, the more demand their is for data.
Re:NanoGPT by antognini on Wednesday January 11, 2023 @12:11PM (#34341004)
Any state of the art model takes about three weeks to train.
Re:NanoGPT by visarga on Wednesday January 11, 2023 @03:50PM (#34344535)
More an indication of human patience than task difficulty.