They interact with normal matter and themselves via the force of gravity...

But gravity is (effectively) a conservative force.

For the most part a dark matter particle attracted to another particle will just convert all the kinetic energy it gained during the attraction back to potential energy as it whizzes past and away from the other particle.

Normal matter particles can radiate away photons when they hit each other, ie friction, leading to a non-conservative interaction. This allows them to shed their energy and clump in a way dark matter can't.

This is why there is still a lot of atomic hydrogen in interstellar space. Thermodynamics suggests the atoms should bind together and form molecular hydrogen, but since a hydrogen molecule has no dipole moment, two hydrogen atoms alone in space cannot easily radiate away the energy released when they would bind. So they don't bind.

I used to think this as well. However, all orbiting mass can shed energy via gravity waves, and by interaction that fling some mass out at high kinetic velocity while other mass sinks deeper, which is a primary method of star formation afaik. Basically, radiating each other instead of photons.

There is also the possibility of dark matter only forces and fields that don't affect normal matter, but allow dark matter to self interact to some extent.

No, gravity is different to all other forces. First it's not really a force, then it's attracting and distant. We don't even have a proper theory for it yet.

Sure gravity is different from the other forces. That changes nothing about what I said, because we do have a proper theory for it: General Relativity. General Relativity matches a vast number observations with stunning precision.

Of course General Relativity is not the final answer, we know that. But that does not mean we know nothing about gravity, or that we somehow can't use our current theory to make predictions.

Dark matter as a concept assumes General Relativity is good enough, and takes it as-is. Without modifying gravity we need some other way to explain the discrepancies like galactic rotation curves, and so for dark matter we add in some new, so-far unseen, particles. We also know they have to interact very weakly with normal matter besides gravity, otherwise we'd would already have noticed them in existing observations and experiments. Based on this we can make predictions.

What would happen if we have some matter that only interacts gravitationally buzzing around?

Well as mentioned in my previous post such matter can't cool down and clump like normal matter, so instead it would be spread out in diffuse blobs around galaxies. And when we model galaxies with such dark matter halos we find that indeed, that might explain galaxy rotation curves. It also makes other predictions which also seem to match other observations. So overall dark matter looks like a pretty good candidate.

The alternative of course is to assume there aren't dark matter particles, but rather that GR has to be replaced or modified in some non-trivial way. There are many people working on that.

However they're so far struggling, because it turns out to be very difficult to change or replace GR without failing to match existing observations. So that seems to indicate that GR is a pretty darn good theory of gravity.