Early on, I realized that you have to start a new "chat" after so many messages or the LLM will become incoherent. I've found that gpt-4.1 has a much lower threshold for this than o3. Maybe that's affecting your workflow and you're not realizing it?
I am on full-rework iteration three, learning as I go on what works best, and this is definitely the way. I'm going to be making a presentation to my team about how to use AI to accelerate and extend their day-to-day for things like this and here's my general outline:
1. Tell the LLM your overall goal and have it craft a thoughtful product plan from start to finish.
2. Take that plan and tell it to break each of the parts into many different parts that are well-planned and thoroughly documented, and then tell it to give you a plan on how to best execute it with LLMs.
3. Then go piece by piece, refining as you go.
The tool sets up an environment, gets the data from the warehouse, models it, and visualizes it in great detail. It took me about 22 hours of total time and roughly 2 hours of active time.
It's beautiful, fast, and fully featured. I am honestly BLOWN AWAY by what it did and I can't wait to see what others on my team do w/this. We could have all done the setup, data ingestion, and modeling, no question; the visualization platform it built for me we absolutely could NOT have done w/the expertise we have on staff--but the time it took? The first three pieces probably were a few days of time, but the last part, I have no idea. Weeks? Months?
Amazing.
This has worked extremely well for me.