I used to think this as well. However, all orbiting mass can shed energy via gravity waves, and by interaction that fling some mass out at high kinetic velocity while other mass sinks deeper, which is a primary method of star formation afaik. Basically, radiating each other instead of photons.

There is also the possibility of dark matter only forces and fields that don't affect normal matter, but allow dark matter to self interact to some extent.