In some parts of the US, notably coastal California, the taste, smell, and color(!) of the natural water supply can be quite poor. It has more to do with geology than tap water safety and remediating the natural chemistry is cost prohibitive in extreme cases. For example, in the western volcanic regions of the US, pristine water sometimes has an unpleasant sulfur taste -- perfectly safe to use but no one wants to drink that because it tastes awful due to natural water chemistry.

Many parts of the US have excellent tasting tap water, and in those places everyone drinks it. California is one exception, and many parts of it have infamously discolored and poor tasting/smelling water. When I lived in California I drank bottled water, most everywhere else I drank tap water. California water is safe, it just tastes terrible and is sometimes discolored because of where it comes from.