The problem is that it is really, really expensive to lay brand new fiber in most places.
If we had complete deregulation, the companies that already have the fiber laid would dominate. A competitor might look at a market, and see the price that the incumbent is charging and think they could provide the service for cheaper and still recover the cost of laying fiber. However, if they decided to actually start laying fiber, the incumbent would just lower prices and the new company would be unable to compete (since they have to recover the cost of laying fiber while the incumbent doesn't).
The only way for true competition in this market is to do what a lot of countries do... have regulations that say whoever lays fiber has to agree to rent it to competitors at reasonable prices. That way, you don't have to keep laying redundant fiber just to compete.
The free market would just lead to monopolies.
Do regulations force ISPs to make people pays more when there are no competitors around? Do regulations set the the price? Aren't duopolies a possible consequence of a free market?
Note that there are natural monopolies/oligopolies, which are the result of the nature of the market (eg need for scale) rather than legal lobbying.
The idea of "free market" giving the best outcome for buyers requires:
1) virtually infinite suppliers
2) virtually infinite customers
3) fully fungible product/service
4) fully rational and informed customers
In markets like ISP, real estate, water/electricity supply, wastewater/garbage/mail/roads the product or service is in no way fungible because is based on your location.
E.g. you cannot pay for an ISP in Germany and use it from San Francisco.
On top of that it's practically impossible to have a large numbers of suppliers in such markets.
Finally, providing mail/electricity/water/Internet to small and remote town is rarely profitable. You can have so-called "market failures" where market equilibrium does not serve customers at all, or at unaffordable prices.
This is why such services are provided by local governments in most countries or heavily regulated companies.
I am struggling to think of anything more fungible than electricity, and water, mail, and internet access follow closely.
Rather, the reason that competition is problematic in these areas is because they are natural monopolies:
As a home consumer you cannot switch from your current electricity provider to a cheaper one... that operates in a different continent.
The link you provided is spot on and does not contradict in any way what I wrote.
"Natural monopolies were recognized as potential sources of market failure as early as the 19th century"
Another good page: https://en.wikipedia.org/wiki/Captive_market