A black hole doesn't have to be very dense. The larger it is, the less dense it needs to be to prevent light from escaping. The supermassive black hole at the center of our galaxy may be less dense than water[1].

For black holes closer to a single solar mass, then yes, they will be super dense, and the degeneracy pressure helps keep it from becoming a black hole. You have to add enough mass to overcome the degeneracy pressure for it to develop the event horizon that makes it a black hole.

But not all black holes do that. Others can simply keep acquiring mass without that violent sudden event. They just gradually cross the point where no light can escape, but you might not even notice looking at it. The light just keeps gradually shifting redder and redder until the wavelength becomes infinite.

[1] https://en.wikipedia.org/wiki/Sagittarius_A*