You're right. The wealth transfers are fairly small. The EU had a budget of €118 billion for 2016. There's only so much you can do with that money for 500 million people. Note that a large chunk of it goes back into the big countries as agricultural subsidies.