On the other hand, once you have a large model, you can use various distillation techniques to train smaller models faster and with better results. Meta seems to be very successful doing this with Llama, in particular.