I sleep in mine, except for once a week when I take them out to disinfect overnight or replace (I have 2-week disposables). I've had a mix of opinions from doctors on this. My current doctor says it's fine if my eyes aren't getting dry/irritated (in which case I should take them out), though I've had past doctors who were against it.

The tradeoff in the current doctor's opinion is that taking them out nightly reduces infection risk theoretically, through nightly disinfection, better access to the eye surface for your immune system, and lower risk of corneal abrasions. But handling the contacts daily for removal/reinsertion provides a new route for infection, as a lot of infections are introduced by fingers, lens cases, etc. So the added handling might offset the benefits of nightly removal enough to make it a net loss, if you're one of the people for whom sleeping in contacts doesn't produce dry eyes or irritation. Real-world data seems not good enough to compare the magnitude of those two effects.

An anecdotal word of warning: I wore bi-weekly disposables as well for quite a few years and would repeatedly sleep in them. Never had dry eyes or irritation, however after a few years I developed tiny capillaries growing in my eye (corneal neovascularization), as it turned out they were not getting enough oxygen.

Supposedly if they continue to grow I could lose my eyesight, and they won't actually go away even if I stopped wearing lenses.

Yeah, that's a good point. I've had that mentioned to me, but I get annual checkups, and have been told that they'd show up in the annual checkup if I were having that problem, at which point I could change my habits (it's not an acute condition that crops up in a month or something). Supposedly that complication has also gotten less common in the past 5-10 years as newer lens materials are much more oxygen-permeable than older ones were.