You don't need to suspect it. You can read the social science literature and have some confidence that hard-working researchers have confirmed your suspicion.
Here's a good place to start: Income Inequality and Social Dysfunction by Wilkerson and Pickett.
I assume someone downvoted me for the typo (sorry! the name is supposed to be Wilkinson), and not because they find it uncomfortable when science supports ideas that run counter to their own.
Here's another good resource, this time from Stanford - http://www.stanford.edu/group/scspi/
enjoy!
I honestly suspect that shrinking the pie but making it more evenly distributed would be best for all for the increased social harmony and stability.