Mining difficulty roughly maps to a financial incentive for a miner to operate. As that line rises, the number of miners who can afford to mine in any significant amount would naturally decrease, right?

Unless there's some other angle to convince miners to continue mining unprofitably, which feels like defeats the whole purpose of a crypto-maximal economy to me.

Small miners typically join a pool to smooth the variance.

Also, hash rate and reward value don't always correlate - mining difficulty can increase during periods of flat or declining coin price.