You could have just extended the IPv4 header in such a way that the routers in between you and the server didn't need to understand the extended protocol, and could continue to route IPv4 packets as-is, since they only see the upper 32 bit of the address. Then network inside LAN and the server need of course to understand IPv4-extended (let's call it IPv5) packets, but still you only need to update the endpoint devices, and not the core of the network (fare easy, since that evolves more quickly!). You could also implemented a mechanism "similar" to NAT, where the router did for outgoing traffic compose the extended "IPv5" packet by combining the local IPv4 and the external IPv4, and did the opposite operation for incoming packets. That way really only the router needed to be updated, and not individual devices in your house. Thus being backward compatible in both ways.

Also adopting IPv6 these days would require to maintain still IPv4, this means that for example firewall rules must be made both for IPv4 and IPv6, something that adds a lot of work to IT technicians (and they thus disable the protocol that if you disable the internet still works, that is IPv6). Extending IPv4 would mean that the same firewall rules can apply, just considering again the upper 32 bits of the address.

Finally IPv6 is in general more complex, it has some obscure things like SLAAC or the fact that an interface can have multiple IPs, that not all IPs are routable, etc, even simply the notation for IPv6 addresses that is not even consistent among software (in some software you have to put it between [] to disambiguate with port numbers, in some other not), DHCPv6 implementations that are still these days quite broken, etc. Compared to IPv4 is quite a complication...

I'm the opinion that we are slow to adopt IPv6 since it's too different from IPv4, and requires maintaining a network with two protocols, since everyone completes the migrations, this is unlikely to succeed in the next years (for example in my country, Italy, most ISP, even the most important one, don't even bother to provide you an IPv6 address! I think that because it does increment the problems and thus the requests to handle in the customer service, thus they decided that you just get IPv4 that works reliably).