There have been a lot of papers on techniques for training in low precision, even as low as binary. But the largest models that would benefit most aren't using those techniques, which indicates to me that they don't work as well as their paper abstracts suggest.

My experience with quantization is that a little is easy, and a lot takes work. Quantization aware training helps, but it's finicky and adds extra steps to the training process, which in turn slows down iteration speed.