So you can still have sight-reading. You also get fully arbitrary sight-transposition, which today is only possible for the very most elite musicians.
Except that's not how a multi-staff partition works. Each staff has a clef that determines it's offset; they can be the same or wildly different. They also usually signify different "voices" or instruments, further encoding musical nuance beyond pitch and tone.
Transposing instruments only even exist because sight-transposing is so difficult. There is nothing fundamental about an instrument that makes it transposing. We only use that kludge because of the difficulty sight-transposing.
Suppose you do have a transposing instrument though. It's in Bb, and you'd like to play music written for an instrument in F. Very few musicians can do that without hesitation.
The same applies when there is a desire to shift the entire piece of music up or down. Perhaps you are using a hecklephone instead of a lupophone, so you shift everything up. Perhaps you just lost a trumpet player with an unusually high range, and thus need to shift everything down. It would be great to be able to expect all players of ordinary skill to be able to do this without hesitation.
BTW, I'm aware that one needs to avoid troublesome notes and things like impossible trombone glissandos. This doesn't change the general truth of the above.
This doesn't work - the staff becomes too large, or the registral range it covers becomes way too small. The current staff has evolved to be readable without expressly counting the lines or spaces, via visual subitizing. If you add more lines, you lose this property and sight-reading becomes infeasible. Plus, diatonic pitch is actually meaningful in a musical sense, and it's useful to have a notation that references it. That's why we use for instance C, D, E, F, G and not <0>, <2>, <4>, <5>, <7>.
(This also describes the very real problems w/ Klavarskribo, BTW.)