It only seems like it would be a problem at businesses with bad staff.
I've also noticed the acquired habit in more experienced retail staff, when confronted with a queue of indeterminate order, to say, "I can help the next customer in line" and leaving implementation of the next method as an exercise for the customers themselves.
FWIW, I find the best arrangement is a Fry's style mega-queue with boundaries clearly defined by racks of candy, magazines, or other impulse-purchase crap that feeds into multiple registers.
Teaching customers how to use them, on the other hand.....
And for some situations, like a movie theater box office, a single long line actually drives people away (on non-blockbuster-opening-nights) because it looks like a big crowd, compared to two small lines. Even if it means they'd get through it faster. .... at least, before the age where everyone just buys their ticket on fandango.
The less obvious reason is that after a certain number of back-and-forths, you can interrupt the other person saying "aft-", cut them in line sideways so you both sort of swoop through the doorway, which then leads to a warp zone.
My reasoning is a little different (and a little more charitable I suppose in its generalization about human nature): I don't want to go step in front of the register with no one waiting in it because I don't want to look like a sociopathic jerk. I also assume that is why most people haven't done so.
At the same time, I know as soon as someone does step up and do it, people will grumble but the single ad hoc queue will redistribute itself into 3 more balanced queues. I'm usually waiting for someone else to be that sociopathic jerk.
What I wish is that the manager of the store gave a little more thought to this issue in the first place.