Anyone knows a good document on when/which browser actually sends referer header?
0: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Or...
1: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Re...
Stripping everything but the domain should be mostly ok for the client though, so if I come from a.b.com/foo, it just sends b.com as referrer? Both "a" and "foo" can hold any amount of data so those would have to go. Sending b.com should be enough to provide traffic statistics?
Although tracking down fraud is usually as it relates fraudulent traffic which mostly effects advertising so I imagine a lot of the commenters here would say that is not a good reason.
HTTP Auth, on the other hand, is actual password protection which cannot be spoofed by a malicious client. Very different from robots.txt.