I'm pretty much in agreement with everything you wrote here, but I want reply to one thing you mentioned...
You somewhat disagreed with, "we know that most developers do not have great design/refactoring skills", because developers may have little chance to refactor. Totally on board with this; lack of understanding of technical debt and how it impacts a team's achievement is one of the biggest problems with software management.
But my point - which I clearly didn't make well enough - was that refactoring and design skills are really the same thing, and I've seen enough modified code to understand what the design skills of most developers are.
The thing is, many refactoring efforts make code worse, not better. Refuctoring is something I've observed many times over, by people who consider themselves quite skilled at the art. The end result of their efforts is that the code is fancier or trickier, not more robust or easier to understand.
So, from a manager's perspective, how are you to discern between a programmer whose refactoring efforts save time down the road, and one whose efforts lose time down the road. Of course, normally you have technical leadership that can make this distinction and makes sure those people get a cold hard reality check, but many organizations discourage technical people from gaining enough power that they can stop other technical people in their tracks before they mess things up.
Just put it in the estimate for that new feature. When your boss starts to needle you about why such a "simple" feature is taking so long, that is when you throw the bad code under the bus. You get to demonstrate a real need, rather than just a theoretical one.
The older I get, the more I believe "Big Bang" code is a bad idea, no matter what the bang actually entails. Gotta make a branch to make a feature? Gotta rewrite a large part of the underpinnings of the app to make things able to move forward? Please don't, please develop it in parallel in the mainline.
In a way, refactoring on its own violates TDD. You are supposed to write failing tests first, then write code that passes the test. Refactoring as it is described by the most zealous of TDD advocates is just a hand-waiving over writing code without testing. If we're being strict about it, write a test that proves the code needs to be changed, then you can change the code. If we're not being strict about it, what is even the point?
Regression is some of the biggest harm that can come to users, because it violates their ingrained expectations. Greenfield defects might prevent the adoption of a new feature, in which case the status quo is maintained, but regression is forward progress that is physically lost.
Out of inspiration from Python docstrings, I have been occasionally working on a JavaScript structured documentation format that I call Pliny: https://github.com/capnmidnight/pliny
It's all about creating a database of documentation that can be rendered to arbitrary formats. The documentation pages on the website for my big project are all completely generated by Pliny.
I made it executable code rather than comments because I could iterate faster on the idea if there were a simple syntax to construct the database structure at the same time as writing the documentation, as well as by including the right setup script, I could have the database live-queryable in the REPL without any additional processing step.
Unfortunately, there's nothing to verify that the documentation matches the code. There is a lot of type information there that would be super useful to exploit to statically analyze the code, but that'd be a big project to add that. It'd almost be easier to convert all of the code to TypeScript and recreate Pliny to extend a generated .d.ts typings file, so the doc gen could use just the typings file as the database.
You have to remember there a lot programmers out there, just implementing business rules into code.
I somewhat disagree with the statement he makes, "we know that most developers do not have great design/refactoring skills". I've certainly worked at places that all but ordered me to never refactor code, and I suspect my experience is very far from unique. They thought "refactoring" was a made-up word that programmers used to cover up dicking around and wasting time. From a non-programmer's perspective, all they see is the programmer spending several hours with the end result being that nothing has visibly changed. Mind you, no visible changes is the ideal case for refactoring, so you have no easy means to explain why that is wrong.
He says that TDD only works if the programmer is good, but I think it's more the case that test-first only works in cases where we have very well-defined problems. For these TDD trainers that he's talking about, they've gone through the same problem set over and over again. For an experienced programmer, if you're encountering the same sorts of problems, you know what the pitfalls are and can look ahead, even if you aren't violating YAGNI, you can still do yourself some favors.
To put it to a specific metric, if you have a specification from which you're working, you basically have an answer sheet for your code. You can then write code that automatically verifies that your code matches that answer sheet. The more like "having a spec" your problem is, the more likely TDD will work very well for you.
But I don't do that kind of work very frequently. Most of my work is extremely experimental, fluid, and in a lot of ways, its behavior is arbitrary. Does a padding of 5px or 10px look better? Should the surface of a polygon be more or less smooth? Should running over the ground be 3m/s or 5m/s? Should the user have two hand tools that are identical, or should they be different tools? A lot of it has to be visually verified, because it would otherwise require a computer vision algorithm of some kind to know that the code I wrote to go from text to screen pixels worked correctly.
It's much more design than development, just that design is expressed in code rather than in Photoshop PSDs.
As a result, I focus much more on REPLs, saved REPL sessions, demo code, and making the iteration time between code change and test run as narrow as possible. What I think TDD provides (poorly) for people in this situation is this REPL-like behavior, in languages that don't particularly provide a good REPL (let us consider RoR a separate "language" from Ruby, in this case, as the expectations tend to be completely separate).
So if I'm writing an implementation of a rope data structure as part of a syntax highlighting text editor that renders into a WebGL texture, yeah, I have tests for that. Ropes and text editors are well-understood data structures, there's lots of information available about them, and there are right and wrong answers to what operations each should have and how they should behave. But that's not where the work ends. I still need to figure out how to let people interact with that text editor, be it with touch gestures or motion controllers or a keyboard or a gamepad or what have you, and that has no easy-mode.