Speaking as someone who recently blew through a 4-way stop with other cars waiting, because the setting sun made it hard to see the stop sign, yeah, there are no shortage of human factors at play.
Stop signs are the shape they are so you can recognize them from the reverse for this purpose
Not the main point but I never liked how traffic signs aren't affirmative sometimes. In the case of the 4-way usually the stopping direction have a stop sign with a missing "All Way" sign below it. But thats not required in all jurisdictions. Nor is the non-stopping direction 100% sure their stop sign is still visible (shrubs, weather, etc) and can proceed at regular speed.
I'm sure there are a ton of reasons for the way things are, I just think the outcome is you can't really ever be 100% sure what the correct thing to do is without slowing down and observing.