I'm sure it matters on some sites. I decided it didn't matter on my site, so I turned virtually all of them off. Might as well save people the download, and on some pages the font size dwarfed the content size.

I like the compromise of naming the desired fonts in CSS so if the user happens to have them installed, they'll see them.