News for hackers. Stuff that matters.
Re:Deprecating our AJAX crawling scheme by a2tech on Wednesday October 14, 2015 @03:49PM (#10388953)
Do you know if they sent Google a sitemap? Our client is insisting on a sitemap that has pointers to every-single-product. Something on the order of 2MM+ product pages. It seems like a bit much to me
Re:Deprecating our AJAX crawling scheme by MichaelApproved on Wednesday October 14, 2015 @04:06PM (#10389066)
Keep this in mind https://support.google.com/webmasters/answer/183668?hl=en&to...

> Break up large sitemaps into a smaller sitemaps to prevent your server from being overloaded if Google requests your sitemap frequently. A sitemap file can't contain more than 50,000 URLs and must be no larger than 50 MB uncompressed.

Re:Deprecating our AJAX crawling scheme by iwilliams on Wednesday October 14, 2015 @05:18PM (#10389614)
The site is built on Tumblr which automatically generates a sitemap for the individual posts, but not any other pages on the site. For example the "about" page is not in the sitemap, but is still indexed.