> It irks me that this is used so much for comparison, since combinations of things are so overwhelmingly larger than collections of things that it feels like a category error to compare them.

I think this is the core of the issue. "Atoms in the universe" is only such a relatively 'small' number if you assume that the way they're arranged is unimportant.