Also, you're making a slight mistake. About a decade ago (?), x264 was changed so that even 2-pass mode is really just using CRF under the hood. It's not "achieving the best quality", it's just figuring out what quantizers you need to hit your average bitrate target exactly. Some more information about that here: https://trac.ffmpeg.org/wiki/Encode/H.264
> normal 1-pass CBR encoding
You may be confusing constant bitrate (which no one should ever use) with CRF (constant rate factor), which varies the bitrate from frame to frame and scene to scene, without having to do a 2 pass encode.
This really is their core business and somewhere they can have a competitive edge, though. Even if you think it's unethical or that it might make business sense to contribute back to the upstream projects, you can see why they might want to keep some of that secret sauce in-house.
Is that still the case? They switched from x265 to beamr for HEVC. Not sure if they are still on x264 on recent encodes.
The best way to understand why this is used is to think of a movie — when there are shots that are totally, absolutely black, like scene changes, normal 1-pass CBR encoding uses the exact same amount of data to that part as it uses for complex action scene. But by using VBR and multipass, encoder “knows” that this piece is OK with lower bitrate and that bitrate can be then used for more complex scenes, thus creating better quality for those scenes that require more bitrate.