The problem gets worse when thing aren't just routers, but modems from your ISP. Unlike routers, which you are free to replace it (or install OpenWrt), using a modem is mandatory, and it's very likely that there is no alternative. Even if the modem uses a standard protocol like DOCSIS internally, it either has some vendor-lockins or proprietary/customized software by ISPs.

A few years ago, I got my optical cable modem from my ISP, and I immediately found that the real admin account was locked from me, only accessible by the ISP. So I couldn't just PPPoE and gets my connection, but I had to use the NAT on the modem. After a quick websearch, I realized there are plenty of ISP/OEM management interfaces and backdoors in the modem. Although the documented backdoor has been disabled, I eventually found my way to a root shell, so I could change the mode to bridged w/ PPPoE.

Meanwhile, I always use my own router behind the modem, with an additional firewall blocking all the incoming connections from the modem's IP address. I absolutely don't have the slightest trust on these modems. If I cannot change the modem, at least I can isolate the problem away from my routers (and use encrypted protocols as much as possible). Attackers are free to execute arbitrary code on my modem, I'll simply treat the modem as part of the untrusted Internet, which can be as dangerous as a hacked modem.