This is very location dependant. In southern Europe there are places where most normal people actually would refill their water at the fountain when going through a town, and I have even faced small queues when the fountain is small but a slightly larger group is refilling bottles. In many small towns you don't even have a shop so buying bottles is not even an option. There's sometimes a local bar / restaurant but if you want to just get water that's a bit overkill.