When I worked on supercomputer simulations, I always landed on just dumping intermediate results and looking at them over figuring out how to compile something TotalView could look at.

When the latter was possible it saved time but it was always breaking under me and I eventually said fuck it