This argument has been shown to be wrong on many, many occasions.

Many users do not have a choice which browser they use, do not know they even have a choice or do not know how to exercise that choice i.e. download and install a competing browser. Finally you're then left with the massive problem of sites only working on certain browsers.

The situation where the user doesn't know about and/or is incapable of installing another browser is pretty common with default browsers like IE and Safari, but not nearly as common with Chrome.

Google have been bundling Chrome with widely-used software like Adobe Flash for years now, there's probably many people who've installed it and set it as their default browser without intending to.

Might. It only took ... what - five years? Ten years? for people to start ditching IE after it turned Evil (IE5+). The years between "Browser X is the best thing since sliced bread" and "Browser X Considered Harmful" became known as the Browser Dark Ages. Not looking forward to a repetition thereof.

From 0 to 1 is a lot different than from 1 to 2, or from 1 to n.