Yeah, we will find ways of training these things faster, and we will have faster hardware. We will have better, more parameter efficient models. This will inevitably be something we can do locally. You can already trade time for money. Like, right now you can buy hardware for less than what a house costs, train using the hardware for a year and you basically have something as capable as GPT3.5 - this is already possible and needs to be done only once (and you can still sell the hardware if you want) and a single dedicated person is capable of doing it by themselves. It will only get faster, and cheaper from here. Same goes for inference, still expensive to run but will get cheaper.