That's the way of getting the address itself, that's unrelated to how fast the actual memory read/write is.

Stack is fast because it is frequently "touched" staying in cache. If you were to continuously read write a small segment of the heap, I don't think it would fair any worse than "the stack". This was my point