You should have a version, but given the (admittedly unrealistic) choice between versioning and extensibility I’ll take extensibility every time. There are plenty of formats where there’s a version tag and it’s never been bumped past “1”.

Early compressed audio / video formats were generally a total mess, with a couple exceptions like MOV, so I’m not surprised that MP3 is horrible.