I suspect that whatever wealth they did have was carefully passed to the next generation and, further, that people who didn’t do that were less likely to have their DNA represented in the modern human pool.