I have literally seen that happen, although the house was electric blue and not purple. I did in fact put my money where my mouth is and moved to a farm without an HOA.

They would hate to live in Tobermory I suppose: https://commons.wikimedia.org/wiki/File:Tobermory_Main_Stree...

Or indeed, Boston: https://www.istockphoto.com/photo/victorian-houses-in-cambri...

Tangentially, Victorian houses were frequently very colorful. The lack of color photography leads people to think of them as drab, depressing shades of gray. But nothing could be further from the truth. HOAs that restrict colors to a boring beige are doing a disservice to their neighborhoods.

If I lived in one of those houses, would I be able to paint it whatever color I wanted (e.g. pink and yellow stripes)? Or is there some sort of coordination to make sure they all look appealing as a set?

You can choose whatever colour you want. It seems they used to me more sombre but around 50 years ago the owner of the hotel decided randomly to paint it bright yellow and neighbouring houses along the front thought it was a great idea and started choosing their own outlandish colours.