Are you familiar with the joke where the patient says about some awkward improbable activity, "Doctor, it hurts every time I do it" and the doctor responds "Then don't do that!"

The principle here is the same: then don't do that!

I will mention though that although pointers occupy 64 bits, the virtual address space on current x64 machines is limited to 48-bit (256 TB). This means that if you are careful, you can get away with stashing an extra 16-bits of data in the high two bytes of your pointers. Or if you are nefarious (as described in the recently discussed http://zinascii.com/2016/the-illumos-syscall-handler.html) you might figure out a way to utilize the "non-canonical" addresses for privilege escalation.

More restrictively, current CPUs have an even smaller range for physical RAM addresses. This varies by CPU, but the Skylake processor I just checked on (cat /proc/cpuinfo | grep address) shows that it only has 39 bits physical address space, which is only 512GB. There are servers with considerably more capacity, but I don't know if any actually get to the full 48 yet.

I mention these limits to say that if you are planning to handle 3.16 x 10^17 8B items on your system any time soon, I think you may hit some other blocking issues before you need to worry about the suitability of the double ended vector.