They'd also have fallen to the center of the galaxy, probably following roughly the same mass distribution as visible matter.

What I'm suggesting is just that there might be multiple dark matter particles, some of which might have structure. At present that is basically untestable, of course.