And also at the results and the intermediate steps. Visualize everything. Don't just look at the final evaluation metric but dive in and see where things go wrong.

Do just a few bad predictions skew your score? What does the best prediction look like? What does the worst look like?

Are all your results just shifted by 2 pixels to the left due to some bug? Are there mislabeled examples in the test set? Etc. etc.