Beyond that, the decision whether a number is signed or unsigned is only needed for string formatting (e.g. it's like Schroedingers integers: whether a number is signed or unsigned only matters when you actually look at the number).

But in any case: with 'sign-agnostic' integer types high level languages would simply need separate signed vs unsigned mul/div operators. Not a big thing when modern languages already have different operators for wraparound vs overflow-checked arithmetic (for instance + vs +% in Zig).