I suspect that this has something to do with the fact that a portion of the administrative work is offloaded to the broker.
I never met anyone from the lender when I got my mortgage. All of the background investigation, income/employment verification and paperwork was completed with the broker.
Conclusion: lenders are irrational. Surprise!