Spot on. People need to say what they are actually using the models for and not just "coding".
I mostly use it to make react/javascript front ends to a python/fastapi backend and chatGPT4 is great at that.
I tried to write a piece of music though in the old Csound programming language and it barely even works.
It will be interesting to see how the context plays out because I have noticed that I can often give it extra context that I think will be helpful but end up causing it to go down a wrong path. I might even say my best results have been from the most precise instructions inside the smallest possible context.
I have them working with mostly C++ and Clojure, a bit of Python, and Vimscript every once in a while. Both models are much better at Python and fairly bad at Vimscript. Clojure failure cases are mostly from invented functions and being bad at modifying existing code. I can't pick out a strong pattern in how they fail with C++, but there have been a few times where GPT4 ends up looping between the same couple unworkable solutions (maybe this indicates a poor understanding of prior context?).