It does, although barely noticeable. More noticeable is the cost to ISPs for carrier-grade NAT equipment.
I have even found that real world IPv6 addresses are not quite as bad as they look. We have all seen the auto-generated ones and they look awful but you don't actually use those as such. For example you are allocated a /48 prefix: 2001:db8:1234:: in general you might think of the ISP as 2001:db8:: and your site as 2001:db8:1234::. Your first subnet might be say 000a or simply "a" for VLAN 10 because you decide not to allocate IPv6 to your default VLAN 1 and start with your current PC VLAN which is 10.123.10/24. In IPv4 you have 10.123.10.1 as the default gateway (VRRP) with .2 and .3 for the physical routers. Hence your PC VLAN routers get given 2001:db8:1234:a::2 and 3 and ::1 for the VRRP address. You can play games with 1337 addresses using 0-9a-f eg face:b00c sigh.
Your comment brings a whole new angle to IPv6 I hadn't before considered. Having each packet routable straight to a TCP/UDP port number eliminates the complexities of NAT. Since the {hard,soft}ware that handles NAT wouldn't be needed, perhaps a shift to IPv6 could also give throughput gains.