Yes, but there is a whole field of artificial intelligence called unsupervised learning that tries to identify labels without pre-defined labels. At the extreme end there are no externally imposed / defined labels and artificial labels are determined by empirical clusters or some orthogonal data pattern or algorithm. Unsupervised learning is much less effective and not as mature as supervised learning. In the case of LLMs the label is "next words" and it's inferred from a corpus of text.