If you have to classify user input as they’re inputting it to provide a response — so it can’t be batched - 2 seconds could potentially be really slow.

Though LLMs sure have made creating training data to train old school models for those cases a lot easier.