> But the Rust compiler should be doing this automatically

This will become a lot easier once the actual full const-generics implementation lands. Because then even your "non-generic" implementations will be enabled to depend on compile-time-defined features of the instantiated type (such as size, alignment, value bounds etc.) without any sort of duplicate code generation. Of course, these in turn should delegate to truly non-generic code when feasible to minimize bloat.