Every model and ecosystem has different compromises, and obviously small models hit diminishing returns quick.

But the big models that are in fashion now... GPUs are basically always memory capacity limited to some extent.