Yes this not that hard if you are statically linking as much as possible. The magic of stable syscalls. Variance between glibc is the biggest headache but musl libc solves a lot of the problems.

Have you ever actually tried that last step that you're suggesting? It's actually really time consuming and expensive to maintain that infrastructure due to oddities between distros, like glibc or unsupported compiler versions. Statically linking is easier than redoing all the work of setting up and tearing down developer environments just because one platform has a different #define in libc. It's also cheaper when your images are not small and you're paying for your bandwidth/storage.