News for hackers. Stuff that matters.
Re:Wanna have zillions of google profiles? by pierrefar on Thursday July 22, 2010 @10:15AM (#1538142)
The odd thing, is that getting a list of all profiles correctly doesn't wor. If you go to Google's robots.txt and scroll to the bottom to find the profiles sitemap is located at:

http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xml

Which 404s.

Luckily, it's quite easy to reverse engineer. Start with:

http://www.gstatic.com/s2/sitemaps/sitemap-000.txt

and increment 000 till you 404. From quick testing, it's less than 3500 but more than 3200.

Re:Wanna have zillions of google profiles? by blueberry on Thursday July 22, 2010 @10:21AM (#1538167)
As of this comment it's exactly 3278. I wonder why you didn't bother doing 3-4 more trials before writing the comment :) http://www.gstatic.com/s2/sitemaps/sitemap-3278.txt
Re:Wanna have zillions of google profiles? by pierrefar on Thursday July 22, 2010 @11:18AM (#1538375)
Because I spent too much time trying to figure out why the index sitemap XML file was returning a 404. I'm really surprised that Google, one of the inventors of sitemaps and big proponent of using them, can have such a high-profile error. I dug around to see if I could find what's going on.

By the time I got to figuring out how many text files there are, I was getting bored.