If your country is a signatory to the UN Convention on Refugees then yes, it is your problem whether or not you feel happy about that.

And what will the UN do if we chose not to comply because the original intent of said convention is now being abused as an excuse for mass third-world immigration?

No, I'm serious, what will the UN, a cabal of unelected bureaucrats who I have never voted for, do to my country if we don't comply with said convention?

Treaties like are predicated on the will of the signatories to sanction each other for non-compliance, which can take subtle or overt forms. While that by itself is no guarantee of change, it may be that you have underestimated the depth of others' political feelings on the topic.

In other words, the worse that will happen is strongly-worded letter from timid bureaucrats.