For research I'm enjoying asking ChatGPT to annotate its responses with sources and reading those; in some cases I've found SIGGRAPH papers that I wouldn't have stumbled upon otherwise, and it's nice to get them all in a response.
ChatGPT (4o, if it's of any interest) is very knowledgeable about DirectX12 (which we switched to just this week) and I've gained tons of peripheral knowledge with regards to the things I've been battling with, but only one out of four times has it been able to actually diagnose directly what the issue was; three separate times it's been something it didn't really bring up or note in any meaningful regard. What helped was really just me writing about it, thinking about everything around it and for that it's been very helpful.
Realistically, if someone let an agent running on this stuff loose on our code base it would likely end up wasting days of time and still not fix the issue. Even worse, the results would have to be tested on a specific GPU to even trigger the issue to begin with.
It seems to me that fancy auto-complete is likely the best this would be able to do still, and I actually like it for that. I don't use LLM-assisted auto-complete anymore, but I used to use GitHub Copilot back in 2022 and it was more productive than my brief tests of agents.
If I were to regularly use LLMs for actual programmit it would most likely be just for tab-completion of "rest of expressions" or 1 line at a time, but probably with local LLMs.
You have any example questions where o3 failed to be helpful?
I use it pretty similarly to you, only resorting to it to unblock myself basically, otherwise I'm mostly the one doing the actual work, LLMs help with specific functions or specific blockers, or exploring new "spaces". But almost all the times I've gotten stuck, o3 (and o3-pro mode) managed to unstuck me, once I've figured out the right way to ask the question, even when my own searching and reading didn't help.