The buttons on a touch interface absolutely can change positions. What button is currently in scope? Everything is contextual in my experience. With physical interfaces, I don't have to worry that my radio's volume knob is going to wind up on the ceiling after I had just turned on the AC.

Even if you design the logical interface such that all 2-d screen mappings are consistent regardless of context, there is still the problem of tactile feedback. Physical affordances can provide varying shapes, materials and actuation methods which can immediately distinguish them from each other without the need for visual confirmation.