That's true; I think it's one of the reasons Zyvex's designs didn't include nanobot-sized constructors, instead preferring "convergent assembly" using nanometer-sized parts but not nanometer-sized replicators. Instead, they envisioned that the minimal self-replicating unit would be the size of a laser printer. A person armed with a rock could disable a warehouse full of such devices in a few hours if that became necessary.
So, self-replicating nanotechnology doesn't imply a risk of gray goo.
As a more useful point of reference than space opera and Drexler's thought-provoking speculations, Linux and GCC are capable of replicating themselves, but they don't pose the same risks as the Morris worm.
> that is a completely different threat model from any machine we have ever seen.
Even without the bogus gray-goo threat, that's true of self-replicating machines, but so were my other examples at the time. Well, maybe the PEPCON disaster wasn't so different from the items in https://en.wikipedia.org/wiki/List_of_ammonium_nitrate_disas... and https://en.wikipedia.org/wiki/Largest_artificial_non-nuclear..., in particular https://en.wikipedia.org/wiki/Wanggongchang_Explosion in 01626, which arguably brought about the end of the Ming 18 years later. Nothing like the Wanggongchang explosion had ever happened before, though, and nothing like it would happen again until the 20th century; quite possibly it was the largest human-made explosion before the US started the first nuclear war in 01945, though there are a couple of other candidates.
If we ever build unsupervised self-replicating machines, we will get grey-goo like problems, that is a completely different threat model from any machine we have ever seen. And if the machines are any fast or any far from us, logic dictates that none of the normal safety measures we apply elsewhere will apply.
(So, I guess the lesson is "don't build unsupervised self-replicating machines". Even our body limits unsupervised self-replication by limiting how many generations a cell can create. But it is surely not "people can misuse any tool", "use it wisely" or "control the consequences".)