That the metric system is just as arbitrary as the customary units. Things like the meter and (kilo)gram were based on arbitrary objects rather than anything objective. They've since been redefined using physical constants to come close enough to the old reference objects. (And the US customary units are officially defined as exact fractions from the SI units -- making the whole world happy to have exact measurements regardless of the system you use.)
At least the customary units have nice divisors. Just saying.
There was some debate at the time about using a pendulum with a 1s period to define the metre, but it was known to be less accurately reproducible than the geodetic version.
Their aim was to define a system that could in principle be recreated from scratch anywhere on Earth without necessarily relying on access to the physical prototypes. In practice the relative imprecision of the original measurements compared to the later demands of science and engineering meant that the prototypes became the definition.
It’s interesting to compare with the British Imperial units reform in the 1820s. Part of that act specifies how to recreate the prototypes from physical properties of the units. When the Palace of Westminster burned down in the 1830s the ancient prototypes were destroyed. But the replacements were recreated from copies, not from the specification in the act. And not much later, the official definition was based on the metric system because the metric standards were manufactured to much greater quality.
No-one claims it isn't arbitrary. Any choice of units is arbitrary. "Not being arbitrary" just isn't a desideratum when considering what makes a good system of units.
The main desirable property is coherence [0], which SI has.
But I don't think anyone would claim SI is perfect. It's not; it's got at least a couple of warts. One is that "kg" is a base unit and is prefixed. But that's about the worst of it.
CGS is also coherent, but has a similar problem with a prefixed base unit (in this case, cm).
US Customary units is not only incoherent, but does things like conflate mass and force (lb is a technically a measure of force, so is the long/short ton, and slug is the proper unit for mass) and has far too many derived units that measure the same thing (e.g. ft lb/s, hp and BTU/h, vs just W).
[0]: https://en.wikipedia.org/wiki/Coherence_(units_of_measuremen...
There was an issue with the standard for the Kilogram, which was recognised then corrected by introducing a definition which is based on physically measurable phenomenon. The new approach allows independent experiments to derive the value of the Kilogram.
That seems to me like a process that works, and I struggle to think of a better outcome.