Why do we say it is mostly empty just because of 2/3 of these things don't interact? Disproving the plum-pudding model was significant but saying the atom is 99% empty space is misleading. We know there are billions of virtual particles appearing and disappearing all the time within the atom and space, they just cancel out.
Unless you think Sagan would have been surprised by the presence of an electric field (mediated by virtual particles…) between electrons and protons in an atom it’s quite likely you’re choosing an obtuse understanding of the term.
A solid wood table is not mostly empty by any common contextual definition. Photons do not pass through it freely, your hand won't pass through it freely, and the electron clouds of the carbon atoms in the wood are physical in almost every common sense of the term. They very much push back on any other electron cloud that comes near enough and are generally the only part of the atom that does the interacting.
While it might be true that almost all the mass is concentrated in the center of the atom, that's not what people mean by empty. Houses aren't considered empty just because most of the mass is in the foundation.
Anyway, for atoms we mostly care about colisions with charged particles. I'd say that they are almost empty.