News for hackers. Stuff that matters.
Re:What one may find in robots.txt by laumars on Monday May 18, 2015 @12:14PM (#9565228)
You can't trust the referrer header for anything security related.
Re:What one may find in robots.txt by Yengas on Monday May 18, 2015 @02:59PM (#9566214)
Re:What one may find in robots.txt by laumars on Tuesday May 19, 2015 @04:27AM (#9568695)
Your citation does reitterate my point. I quote "However, checking the referer is considered to be a weaker from of CSRF protection."

The referrer header can be subject to all sorts of subtle edge cases such as switching between secure and unsecure content (or is it the other way around, I can't recall off hand?) which many broswers will then refuse to send a referrer header. So while checking the referrer might work most of the time, it's really not robust enough to be considered trustworthy for anything security related.