Take out Earth, Mars, and Venus for good measure.
Whatever scattered remnants that are left of humanity hanging out in the asteroid belt are going to have a hell of a time finding food to eat or oxygen to breathe.
Why?
Do you think they'll be reliant on planets for food an oxygen? That's daft — they will be growing their own food and mining their own oxygen.
Also, I doubt they will be "scattered remnants" — there's likely to eventually be a far larger population off-planet than on.
Much harder to wipe out a civilisation that's dispersed among hundreds or thousands of smaller space colonies. Especially if many of those colonies are hidden in an asteroid belt of millions of rocks.