Owning the modem only gives them access to the link layer. It didn’t give them any access to my router or to my https traffic

Do you think root access on your router gives access to your unencrypted https traffic?

It gives them access to the LAN so they can, for instance, figure out how many internet gadgets your house has and sell that information to advertisers, or do even worse than than.

depends

Sorry, but your answer is all over the place. First you talk about the other end and then you talk about the modem. I don't care about DOCSIS, it's slowly dying here, almost all people have fiber to home in my country (last time I checked, it's getting close to 90%).

So, let's talk about fiber. So:

The ISP controls whatever the other end of that fiber is plugged into. It doesn't matter if the medium is fiber, or copper, or a piece of string. The ISP always has control of the other side of the customer interface. It doesn't matter if the box physically resides in your home or not.

Sure, the ISP owns the other end, but what's your point? By using my own router (and my own SFP+ module, which is less important), the ISP does not have a device (backdoor) on my network and cannot control my router. Sure, they could capture traffic on their end, but at that point it's pretty much all encrypted anyway. If I don't trust my provider knowing to what individual hosts I connect, I could set my router to tunnel all traffic to another host/VPN/whatever.

At any rate, using your own router + maybe modem removes the worst backdoor when it comes to providers.