Mostly, yes. but there were clearly common people in history that were described as 'somewhat eccentric' but still managed a normal-ish life otherwise. The chances are most of those were people who would today be labelled 'high functioning autistic' or 20 years ago would have been labelled as having aspergers.
What money/class-status afforded was the ability to be more eccentric without being pushed into an institution.
That's the meme, and it probably did go that way (crazy -> eccentric), but it's also true that someone who was just a little bit off would have been called eccentric whatever their social class.
It would not have been a very effective euphemism for a crazy rich person if it didn't also have the nicer meaning for everyone, rich or poor.
The classical difference between them is money. When a powerful person acted "crazy" the term used was "eccentricity".