NAT requires each and every packet to be rewritten with different source (or destination, or both, depending on the flavor) IP addressing information, so it isn't just a simple copy.

Drivers and sanity checking what's going in/out of the VM probably account for the rest. Hardware may be fast but adding layers of complexity will have a performance impact.