Except that dark matter doesn't even interact with itself. Particles of dark matter just pass right through each other rather than clumping together to form complex molecules, rocks, planets, etc.

A Dyson swarm emits approximately same energy as its host star, just in infrared. Dark Matter could, I guess, be Matryoshka brains organised carefully to not hit each other by their keepers, so long as they are also actively "gardening" what we can see so they never ever get Dyson-d.