But reading this I do have a thought: chain of thought, or guided thinking processes, really do help. I haven't been explicit in doing that for the image itself.

For a problem like this I can imagine instructions like:

"The attached image describes the problem. Begin by extracting any relevant information from the image, such as measurements, the names of angles or sides, etc. Then determine how these relate to each other and the problem statement."

Maybe there's more, or cases where I want it to do more "collection" before it does "determination". In some sense that's what chain-of-thought does: tell the model not to come to a conclusion before it's analyzed information. And perhaps go further: don't analyze until you've collected the information. Not unlike how we'd tell a student to attack a problem.