Yes, it is a feature of the default browser stylesheet, but many websites override it. DuckDuckGo, for instance, keeps all of its links black and uses the :visited pseudo class to show the checkmark.

Even if it's default browser behavior, keeping it around is a specific design decision.

Well, it's a CSS thing these days (a:visited), and something that an awful lot of other sites these days sacrifice in favor of looking more shiny and uniform.