As someone who grew up watching Ally McBeal, I am pretty sure that noone of my generation really has any problem with unisex toilets. But where the confusion comes from is that I think that the concept of a "bathroom" has somehow been conflated with that of a "changing room", e.g. at a gym or something. I have found that people who are passionately against them, with gently probed, actually mean the latter.
In college, my freshman-year dorm bathroom was unisex. It initially created huge complaints from everyone. The women complained that the toilets were always messy and the men complained that the toilets were always occupied. Those may be stereotypes, but they're also frequently true. We eventually resolved our issues by designating 1 stall as "standing only" and the other 2 as "sitting only" to ensure that men could always pee without a wait and women could always sit without having to clean first.
But that only worked because we could impose those kinds of rules. A public bathroom is a different matter and I'll bet both genders get annoyed with having to share with the other absent those kinds of imposed rules.
Note that this kind of thing can be cultural as well. At my last job, the non-Asian women complained about some of their Asian counterparts' bathroom use. It seems that women who grew up with squat toilets liked to stand on the seats so they could use their preferred squatting technique. This resulted in much the same sort of messes that result from men standing when they pee.
I've worked in a gym and bars, back when I was a student, so I have been in women's changing rooms/bathrooms regularly, when we closed up, to clean. Not everywhere has dedicated cleaning staff; sometimes the lifeguards or the bartenders do it at the end of their (our) shift. It's true that men tend to miss and pee on the floor. Alot. But good God, the state of the women's...
But I'm curious as to the arguments against all gender bathrooms? I've visited tons of restaurants over the years that just had a single restroom, and it's never been a problem. When you have people over they might use your bathroom, and that's also not considered a problem. Heck, at all the doctor's offices I recall visiting they always had a single bathroom. I always find it slightly frustrating when I try using the restroom and it's occupied and I'm forced to wait, even though there's a unoccupied bathroom mere feet away. What are people's concerns with shared bathrooms?