"He won't move an inch," or "He won't move a centimeter."
"She won't quit until she's six feet under," or "She won't quit until she's two meters under."
"I'll go the whole nine yards," or "I'll go the whole nine meters."
"You can see for miles and miles," or "You can see for kilometers and kilometers."
Changing your scientific measuring system doesn't necessitate a change to your colloquialisms; commonly used phrases don't change.
And we have expressions including other old units that you don't have (lieu, arpent, toise, etc.).
Heck, most people using them don't even have a rough idea of how much they originally meant, and if you were to press them to give you a number for the sake of an experiment, they might be 2 orders of magnitude off.
https://en.wikipedia.org/Obsolete_Russian_units_of_measureme...
The heuristics at play that the human brain would use is likely: "5 miles is way more than I'd normally want to walk on my feet, since the trip would take me about 2 hours."
Honestly your comment, assuming it is meant to be taken seriously, strikes me as bizarre, and I suspect you're misinterpreting your American friends' reactions.
Ah!
Note: Americans hate it when I do this.