While I don't think that you're advocating the same type of "help" that a colonialist would, you must be able to see the irony in calling someone a colonialist while also saying the exact same thing that all colonialists said — that "we must help them". The implication, of course, is that they are unable to help themselves.

You believe that — through your own luck and with your own privilege — you are able, and therefore should be willing to help people who — through no fault of their own — require some help.

A colonialist would say that — due to their innate genetic superiority — they are able, and therefore should be willing to help people who — due to their own innate inferiority, primal culture, and lack of intelligence — require some help.