There seems to be a lot of potential for efficiency gains in models where a much smaller model can achieve the same results. On the other end computing hardware always gets faster and GPU size/performance has been growing exponentially for a good long while.

Local inference is quite inevitable.