The impact of poverty is more than just being poor, it can also affect your state of mind. There are studies that show people who have grown up in poverty have (on average) less self-control than those who grew up without experiencing poverty. I'll dig up the study so you can see it for yourself.

EDIT: There are a bunch of studies related to poverty and self-control in this article:

https://www.psychologytoday.com/blog/the-motivated-brain/201...