Thank you for bringing it up. I added BERT intentionally to draw attention on this fact that I'm unable to convey properly yet: I was okay with the use of previous generation of models, so it's not that I'm against LLMs since BERT-like architectures were indeed used previously.

What worries me about the current generation of models is privacy. In another comment, I tried to draw the line where models start to worry me regarding it and discussing it with whimsicalism, we noticed that it's not the generative aspect in particular; model size is probably the main factor. For instance, a larger encoder could make its transform almost lossless and thus more prone to data leakage again, although a decoder-like makes it more susceptible.