Without this ability to adjust on-the-fly, the exchange rate of Bitcoin will continue to be very volatile. There's a reason we don't use gold as the basis of our currency anymore: every time the value of gold rose, waves of people went out looking to dig it out of the ground. They were successful, and suddenly there's a lot more money floating around. But not any more bread, oil or cloth, so a loaf of bread that used to cost $1 now costs $2.
With a fiat currency, you have a store of value that is created by decree, not by the efforts of any individual in the system. So now, if there's an gold rush, the value of gold (denominated in dollars) will rise. People will probably go mine more gold and the value of gold will decrease again, but the value of wheat (as denominated in dollars) will stay more of less stable. If you can take $1 million worth of servers, electricity, etc. and are able to mine $5 million worth of Bitcoin, you get the same effect.
Instead the activity of Mining should be thought of as minting new coins and miners are people trying to get lucky; similar to how someone mining for gold. Remember even the worthless paper dollar needed to start somewhere.