Re:State of the OpenVMS Port to x86_64 [pdf]
by loph on Wednesday October 12, 2016 @01:30PM
(#12694874)
DEC had a tool called VEST which was used to take native VAX binaries and convert them to run on Alpha AXP. It was very cool technology. A fair amount of the system utilities for OpenVMS on AXP were VESTed VAX binaries. I'm pretty sure the EDT editor was one of these.
The VEST technology could possibly be used to convert native VAX binaries to run on X86_64, I don't know for sure how the endian-ness would be dealt with.
He also likes to talk about the binary compatibility on VMS; e.g., VAX binaries that run unchanged on Itanium. Unfortunately, that commitment to compatibility has wavered a bit since HP offshored VMS maintenance. We've had to work around some breaking changes recently.
If I'm understanding these slides correctly, AMD64 VMS would require recompiling VAX, Alpha, and Itanium applications from source. I kind of see a chicken-and-egg problem: application vendors won't port without demand, and customers won't adopt without applications.