If difficulty adjustment didn't exist then mining would still be as easy as it was in the satoshi days. The current ASICs would have already finished mining all 21M bitcoins several times over.
Just to clarify a bit further, this is true right now but the amount minted with each block isn't constant and decreases over time. More info on block rewards: https://www.bitcoinblockhalf.com/
Other coins have solved this, in fact bitcoin cash had to solve it as soon as the fork happened because it would have taken too long to hit an adjustment period.
When the blocks being mined approximately every 10 minutes is valued higher in USD, naturally more people want more of those blocks. So the miners expand simply to compete, since what they compete for is worth more.
Ideally market powers would offset this by people with spare CPU time beginning to mine to pick up the slack, since profitability is up. However due to the development of specialized mining hardware, mining isn't elastic anymore and so it doesn't happen.
Because of that the fees will not decrease until the next difficulty adjustment which is every two weeks, or until the miners come back online.