That's the reason right there. The rates are totally set by regulation. The number of medallions is similarly set. If medallion rental cost were to go up or down by 20% this wouldn't affect whether the marginal ride would be worthwhile for the driver.
Likewise, there are mental and affordability factors that influence passengers. When I was still going out to downtown clubs, I lived in a close suburb of my mid-sized city. Cabs were regulated by the city, but unregulated outside. That meant that predatory cab companies would charge as much as $50 for a 5 mile ride, and something like $10 for a 4.8 mile ride within the city.
In NYC, historically this has meant that many cabs are kept in motion 24x7, and are concentrated in specific areas in Manhattan to maximize the number of fares.
For a cab it's similar, except you have labor cost and medallion cost as well. With rates fixed by regulation, the only way to increase profit is to drive more.