Yes. How can I be assured that their mechanism has not, and cannot be compromised? I can obtain updates through their website. I should be able to register for notification when updates are available. Why would I want to allow remote access (even to my ISP) which I cannot monitor? Moreover, why would I want to (potentially) allow (whoever has access) to install whatever they want on my router?
If you don't trust their update mechanism to remain uncompromised, why do you trust their website to remain uncompromised?
I could buy a Firewall, but why should I have to?
Smart.
If you're really that paranoid, I suggest you open your wallet and buy a firewall to put between the LAN port on your router and the rest of your internal devices.