You do not need a prompt engineer to create: “answer with just "service" or "product"”

Most classification prompts can be extremely easy and intuitive. The idea you have to hire a completely different prompt engineer is kind of funny. In fact you might be able to get the llm itself to help revise the prompt.

All software engineers are (or can be) prompt engineers, at least to the level of trivial jobs like this. It's just an API call and a one-liner instruction. Odds are very good at most companies that they have someone on staff who can knock this out in short order. No specialized hiring required.

> ..and validating the output?

You glossed over the meat of the question.

Your validation approach doesn't really change based on the classification method (LLM vs NLP).

At that volume you're going to use automated tests with known correct answers + random sampling for human validation.

Prompt engineering is less and less of an issue the simpler the job is and the more powerful the model is. You also don't need someone with deep nlp knowledge to measure and understand the output.

>less and less of an issue the simpler the job

Correct, everything is easy and simple if you make it simple and easy…

Plenty of simple jobs required people with deeper knowledge of AI in the past, now for many tasks in businesses you can skip over a lot of that and use a llm.

Simple things were not always easy. Many of them are, now.