It depends on how much other code is depending on the code you're changing. The more that depends on it, the higher the stakes for making large modifications on it, and the higher the degree of implicit assumptions which are very hard to fully capture in tests and specifications.

But if you have solid tests and spec for the area, I fully agree: be bold (I wrote earlier: http://blog.barrkel.com/2008/07/be-bold.html). But it's not always possible, and you need balance when the issue requires more responsibility.