It's all a balance, but one thing to consider is that future maintenance or new development may require changes to that software. At that time, having good tests will help you avoid breaking things in subtle ways. The problem is that when you reach that point, nobody may remember enough about that code to write the tests, so it's preferable to have already written them.
If a piece of software is trivial or isn't core to your business, though, obviously this doesn't apply.
Which is exactly how testing should be done. If you have a piece of software that you test so frequently by hand that you could get actual ROI by automating the testing, awesome. If you don't, then what exactly are you getting out of all the automated testing of code that often never changes with tests that essentially assure you that 2 + 2 is still 4?