> I believe it's impossible to tell "Google Voice" usage from bona fide Fi usage where it's really the users' only phone number

These are definitely distinguishable, and some auth service treat them differently. Google Voice numbers come from a different carrier (the one named Bandwidth) than most Fi numbers.