No one is advocating releasing broken stuff, but whats the point of releasing something perfectly tested that no one wants? I think the answer lies somewhere in the continuum. And in my own startup (wello) we do go back and write tests, because testing everything manually does hurt once the feature is working well and users actually want what you are building. So we do carve out time and write tests and often times then build it right. You could argue that this is inefficient, but I could also argue that for many features we removed because users did not want them, we saved the time needed in upfront TDD.
Regression tests should get refactored at the same time by the same tool that refactors the rest. Code changes should be handled the same way.
So, the code provides a list of numbers 1, 2, 5. Non-TDD would code a test that verifies the list is indeed 1, 2, 5.
However, the business wanted the list in reverse order: 5, 2, 1. TDD would force the developer to state this up-front. A non-TDD developer is more reluctant to change the code, since "it passes the test!"
I've done this, although I like to think I'm quite happy to rewrite something if it's not good enough (or just wrong).