Of course the port layout is going to be a fiasco. It's just not physically possible any other way, unless you start shoving everything into a single USB-C port and force the user into using adapters and docks, at which point you'd just be removing actual ports like laptop makers have been doing for the past years.
Obviously port layout is harder than just "just move it" but aside from the physical board foot print, mounting hole locations, and GPIO, there doesn't seem to be some holy thou-shalt-not directive that ports can be moved around, considering the USB and Ethernet ports have flip flopped a few times, the ports change shape, get removed, etc. So, move the most commonly used ports next to the required ones, and put them all on one side of the board so this can actually be neatly slotted underneath a monitor or on a shelf or rack mounted without having to buy a bunch of right angle adapters, PoE hats and switches, or a $50 Argon One that moves the power and display ports to a more convenient location on the back of the Pi by means of a PCB.
Not to undermine your concerns, but in case you are curious the Raspberry Pi Compute Module 4 (CM4) is lovely and allows you to choose from a wide variety of carrier boards which might better fit your use case. I use them in my robots but I have a few carrier boards for home that make them like a regular Pi for random stuff around the house.
The current form-factor is based on the B+ (the B had ports on every single side, if you count the GPIO pins, while the B+ moved the power round to the same side as the HDMI), and I suspect part of the reason for sticking with the form factor is the HAT specification (which specifies physical layout as well as electrical and software requirements), which I think was published at the same time as the B+ was released (my memory's fuzzy on that point, though).
I get that the board is small, but, put the most commonly used ports on one edge, and extra/accessory ports on another. Power, network, two USB, and one display on one edge, that's all I want, so I can neatly tuck this into a shelf or rack, or at the very least orient the thing on my workbench so all of the cables are coming out one way. It's not like I can put my Pi 5 in a Pi 3 case, so what's the path dependency on footprint and layout? This form factor frankly sucks, and I can't see a good reason why they've stuck with it other than "that's the way the B has always been laid out".