I mostly agree with you. I've found unit tests shine in refactors simply because you are often refactoring for performance and/or bug extraction so the overall functionality of a given unit of code is largely identical to the previous version.

However, outside of refactoring, I've never found them to be cost effective time wise compared to other automated testing options.