What actually matters is the packet format.
You cannot make it backwards compatible because it is impossible to expand the IP header's address space without it becoming unparseable to existing network equipment.
"Ah" you say, "but we could put the extra bits after the end of what is basically an IPv4 packet"
Well no, you can't do that either because it would require executing a global seizure of existing IPv4 space and reorganising the entirety of the IPv4 internet into a hierarchical manner where your "IPv7" extra bits can be processed prior to the final destination.
IPv6 works fine and is well-designed. Hardware and software implementations have been in-production for decades now; Google alone see 30% of their total traffic over v6.
Fundamentally, support of any network protocol is subject to Internet participants having compatible kit; x86-64 wasn't immune to this; if you didn't buy the chips, you couldn't use the software. The reason it actually won is that the Itanium architecture was completely different and resorted to translating x86 instructions which, unsurprisingly, ran like treacle and never made it to the consumer market either.
The idea of inventing yet another internet protocol is patently absurd when you consider it would require yet more upgrades across the globe to hardware and software in order to support it.
"The reviewers generally felt that the most important thing that TUBA has offers is that it is based on CLNP and there is significant deployment of CLNP-capable routers throughout the Internet." [0]
[0] https://www.potaroo.net/ietf/idref/draft-ipng-recommendation...
Of course these days nobody remembers either TUBA or CNLP...
I'm sure Dart would be absolutely over the moon to have 33% adoption.[0]
You can't just slap some zeros onto the start of a v4 address and come up with a bigger address space that somehow works with v4 networking code. The problem isn't backwards compatibility, it's forwards compatibility: v4 isn't forwards compatible with larger address spaces, so it's not possible to do what you're suggesting.
There are various ways of making a bigger address space with backwards compatibility to v4 though -- and all of the ways which are actually possible and work are already implemented by v6.
The majority of packets on the internet pass through fairly hard-coded switching chips that probably couldn't be updated to handle this. They can handle IPv6, though.
If it were made truly backwards compatible
It can't be made backward compatible. Every switch on the internet needs to understand your new extended address scheme otherwise they'll send your packets to a dead-zone somewhere in Uzbekistan. Backward compatibility was never an option. As soon as you add new bits to the address, old routers won't send them to the right place.
"But that's so hacky", you say. "We need to fix the old standard." This is what Intel said with its Intel 64 Itanium processor. Break with the old instruction set and make it better. AMD came out with Opteron[1], a hacky extension of the x86 ISA but for 64 bit. Backwards compatible. Swept the market. It's why your binaries are compiled to "amd64" now.
Yes, IPv4 has quirks, and those quirks are well understood and lots of code exists now to deal with them. Let's not find out all the quirks of a new system the hard way for seemingly little benefit.
1: https://www.eweek.com/pc-hardware/64-bit-intel-vs.-amd