I think there will be a trend where model's size will shrink due to better optimization / compression while hardware specs keep increasing.

You can already see this with Chinchilla:

https://towardsdatascience.com/a-new-ai-trend-chinchilla-70b...