It significantly increases the risk of network forks for people who don't have completely correct clocks. If a merchant has a clock that is off by 1 hour (they didn't adjust properly for daylight savings perhaps, or maybe they just have a really crappy clock), then it's possible they will not even be seeing the most recent 6 blocks, as all of them are 2 hours in the future (3 hours as seen by the merchant).

There are consequences to having blocks that are permanently set forwards beyond a one-time difficulty decrease of 0.6%.