It's not insignificant. Humans are pretty good in determining whether someone is making eye contact or not. At 5 meters, we can easily detect a 10cm deviation in gaze direction [1].

Also, I'd like to be (or feel like I am) much closer than 5 meters when talking with someone.

[1] http://tachilab.org/content/files/publication/tp/imai200604P...