Money is just a proxy for power, so “poorer” means not able to do something. Note that you are referring to “we” being poorer, and a country’s ability to do (or get) something is not the same as an individual within the country’s ability to do something.