I disagree - larger models are significantly more capable, likely as a function of their size, and the cost to inference them is only going to go down.

Maybe for niche use cases like handling customer support requests small models will do well, but GPT4 is not going to be condensed to a few billion anytime soon imo.