Yup! I don't have a lot to say about LLMs for coding. There are places where I'm certain they're useful and that's where I use them. I don't think "generate a react app from scratch" helps me, but things like "take a CPU profile and write it to /tmp/pprof.out" have worked well. I know how to do the latter, but would need to look at the docs for the exact function name to call, and the LLM just knows and checks the error on opening the file and all that tedium. It's helpful.

At my last job I spent a lot of time on cleanups and refactoring and never got the LLM to help me in any way. This is the thing that I try every few months and see what's changed, because one day it will be able to do the tedious things I need to get done and spare me the tedium.

Something I should try again is having the LLM follow a spec and see how it does. A long time ago I wrote some code to handle HTTP conditional requests. I pasted the standard into my code, and wrote each chunk of code in the same order as the spec. I bet the LLM could just do that for me; not a lot of knowledge of code outside that file was required, so you don't need many tokens of context to get a good result. But alas the code is already written and works. Maybe if I tried doing that today the LLM would just paste in the code I already wrote and it was trained on ;)