All you need is a language that actually incorporates units of measure into the type system - i.e. you can define units orthogonal to types (including relationships between units), and then you can specify both the unit and the underlying numeric type in a declaration.
https://docs.microsoft.com/en-us/dotnet/fsharp/language-refe...
This can also be pulled off with somewhat less pleasant syntax on top of a sufficiently flexible parametrized type system - e.g. C++ templates.
And yet people actually do manage meaningful unit systems that don't allow this.
C++11's std::duration manages just fine without that much scope bloat. operator* for two durations is simpley not defined so will lead to a compile error.
Company i worked for 20 years ago had commercial engineering modelling program that did all physical types correctly and did scaling for user but it was fairly unique in scope and like you say almost all programming languages fall short here and it had its own quirks.
Every implementation of units is secretly trying to be Moment.js, basically.