Some problems with that argument:

* If the variance is high enough, the average hardly matters. You're just getting slightly better noise. Example: a friend is into a sport, so does that mean you want to watch that sport? A friend watches a lot of Fox News, so does that mean you want to watch it?

* The "people I trust" graph isn't captured by "people I follow." You can follow people you don't trust, for reasons like entertainment or keeping track of what groups you dislike are up to.

* "Trust" is an abstract term that isn't well defined out of context. Trust for what purpose?

* Arguing in terms of an abstract social graph obscures whether we are talking about people or organizations and how we construct any actual graph. People follow others for whimsical reasons and often don't maintain their following list. If you ask them why they are following someone, they might not even know.

For the limited purpose of finding interesting people to follow it can be okay, but I don't see it getting automated in a way that would work for web search or finding people with a common interest. For example, Reddit often works better because you're looking for something that can't be found by asking people you know. The people are out there but you're not connected.