Re:More than 1MM Facebook accounts exposed
by cleverjake on Friday November 02, 2012 @01:50AM
(#4731416)
I don't understand how these pages could have been crawled - could someone enlighten?
[ Parent
| on HN ]
Re:More than 1MM Facebook accounts exposed
by nico-roddz on Friday November 02, 2012 @01:55AM
(#4731424)
It's seems that Facebook uses robots.txt to block this pages
https://www.facebook.com/robots.txt
But, depending of the amount of inbound links, Google will index the urls anyway.
It's a common issue.
Re:More than 1MM Facebook accounts exposed
by cleverjake on Friday November 02, 2012 @02:17AM
(#4731488)
Google ignores robots.txt if the number of inbound links is > N?
Also - any speculation as to how so many sites were lining to peoples login pages?