EU budget is a “puddly” 150b/year. Still, with the vast difference in economic development it adds up to a 500+$ per person per year funding differences between several country’s. Which as you say is not going to change things dramatically in a given year, but it’s still meaningful over decades.
Of note, England negotiated a rebate before joining. So, it’s a demonstrated issue for them.
EU budget always had a kind of political and strategical value attached to it — e.g. to protect agriculture, which is sound strategically (to be able to feed yourself in isolation if it comes to that), but the deeper symbol is that Europe is very afraid of famine. In the 1940s, it all began with cooperation around coal, energy. The EU, at least originally, is built to ensure individual countries can't F with the really important stuff (between each other, nor shoot themselves in the foot).
Evidently, the mission has been met with some difficulties.