For me, it definitely affects the quality of the tests. Once I've written the production code, I know how it works, so its faults are harder for me to see. I also find it's easier to leave coverage gaps. I think I could get some of the same benefit by bringing in somebody else to review tests without looking at the production implementation, but I haven't tried.
The other thing I think TDD gets for me is cleaner API design, because my orientation begins (and mainly stays) outside the thing I'm working on. And cleaner design definitely makes refactoring easier.
Of course, if people have tried it both ways and feel they are getting the same results with some other technique, that's great. But one can only measure long-term maintainability by maintaining a code base for a while, so I think regardless my point on needing experience to judge appropriateness of TDD stands.
The ability to refactor without fear comes from having adequate test coverage. Whether you write the tests before you write the code (TDD) or after the code is completed doesn't seem to affect your ability to refactor at a later date.