There can be ISA-level differences: Itanium’s VLIW structure meant a lot of memory was storing NOPs filling unused slots unless your work aligned perfectly with the CPU and you had a brilliant compiler.

These days, I think you’re right with the possible exception of density: if you have comparable resources, vector extensions, etc. an ISA which uses memory more efficiently is going to get a little more out of its cache and memory bandwidth, but that’s probably not going to be huge.