I also think there's some exaggeration. Annotating files with a feature tag system is both manual and not scabale. Custom prompting for each commit or feature a lot more so. You do a decent bit of specialized work here.

And I think he left out the most important part, was the answer actually right? The real value of any good dev at all is that he can provide reasonably accurate analysis with logic and examples. "Could have an error" is more like a compiler warning than the output of a good engineer.

Side note: "broke the benchmark script?" If you have an automated way to qualitatively evaluate the output of an LLM in a reasonably broad context like code reading, that's far bigger a story.