The cost of mining only adjusts after a set number of blocks. It's possible for there to be a sudden drop in price, meaning the majority of miners shut down unprofitable rigs, and then it takes essentially infinite time to mine the required N blocks to adjust the difficulty.