Cyclists, especially in a group should stay two or three abreast as it significantly reduces the passing length for overtaking cars. The only reason people complain about cyclists two abreast is that they want to overtake when it's not really safe to do so.

Does the law say anything about having to get into single file?

(In any case, the law doesn't provide for this case of vigilantism. And two wrong wouldn't make a right.)

Of course not. The law typically requires slow traffic to pull over when safe and allow faster traffic to pass when there are more than a few cars backed up behind, but that's a bit different. But naturally the lack of legal standing doesn't stop drivers who think they have the right to pass all bicyclists as soon as possible.