I agree. Browser speed does not affect consumer choice directly. Very few people chose Chrome over Firefox because it was slightly faster or slightly more stable. The vast majority of users chose Chrome because it felt better.
Also, Google's pushing it helped. Otherwise consider how Chrome has occupied an IE like market share despite the quality Delta between Chrome and IE/Edge, Firefox or Safari is significantly lesser than the Delta between Firefox and IE was a decade ago, yet Firefox peaked at a much lower marketshare than Chrome's today.
It would be like using Tor all the time. You know it's better, but the difference in speed is so great, that you have to make a conscious choice. I think I speak for a lot of people when I say that I want the benefits _without_ compromising speed.