Sometimes this works! But it's not guaranteed - this isn't their core strength, especially once you get into really deep knowledge of complex APIs.
They are MUCH more useful when you use them for transformation tasks: feed in examples of the APIs you need to work with, then have them write new code based on that.
Working effectively with LLMs for writing code is an extremely deep topic. Most people who think they aren't useful for code have been mislead into believing that the LLMs will just work - and that they don't first need to learn a whole bunch of unintuitive stuff in order to take advantage of the technology.
There is a space for learning materials here. I would love to see books/trainings/courses on how to use AI effectively. I am more and more interested in this instead of learning new programming language of the week.
When I see claims like this I suspect that either people around me somehow 10x better at promoting or they use different models.