In competitive rental markets, there will always be another tenant who will not be "difficult" in the eyes of the agent who will snatch the place up from you if you start going off the beaten path.

Before I got my place there were 3 other apartments in the same complex I verbally agreed to and then agent called back to say it had been taken. Not sure if you've rented in likes of London, NY or SF before but the competition can be intense.

Look it sucks that I had to do that, in an ideal world we could have done what you said, but I valued getting the place over my reservations of handing over bank statements.

So often this is the case. Someone asks something unreasonable, and pushing back gets you marked as difficult. It acts as a filter, even if done unintentionally

Because nobody wants to rent to a difficult tenant.