If the task requires labels yes. Alternatively, an auxiliary model (not part of the training) can generate pseudo labels and use those during training. A more general approach (for which a few works in FL have proven to work pretty well) would do instead un/semi-supervised training.