The point being that Dark Matter doesn't even seem to interact with matter, so why call it any kind of matter.

Matter isn't matter unless collisions prove it's occupancy of space. That's pretty much why matter is considered anything at all. You can't gloss over a significant lack of collisions.