There are multiple markets and varying degrees of regulating them. But the universal constant is that the less regulated a market is the better for consumers and society.

That's a bold statement and by no means a universal constant. If your rivers are on fire because there is no environmental regulation, then that is not good for society.

Good example! Environmental regulation managed to drive manufacturing out of developed countries making them vulnerable to less scrupulous, unfriendly regimes while merely shifting the environmental impact elsewhere.

The correct solution was, of course, taxing externalities like pollution and CO2. That would've allowed the free market to search for solutions naturally and locally.

That is my main qualm with regulations: they are a brute force imposed solution which solves the perceived problem on the short term while doing damage and 2nd order effects on the long term.

How is taxing externalities not a measure of regulation?

Taxation uses market mechanisms and thus encourages market solutions. Regulations impose politician-thought solutions directly on market participants.