Thanks for the details on this! We dug into our Wordpress and realized that W3 Total Cache was configured to block anything that had the word 'bot' as part of its user agent string (sigh). That's now fixed and live.
As to the redirect - that's actually a bit of devious magic on our end. Since jquery.it is hotlinking to our JavaScript and CSS we just added a bit of JavaScript to automatically redirect them to the right jQuery.com page:
if ( /jquery\.it/.test( window.location.hostname ) ) {
window.location = window.location.href.replace('y.it', 'y.com');
}
Thanks again for your pointers - looks like the jQuery 1.6.2 blog post is already showing up in the search results and jQuery.it is not in the first page of results (although "jquery 1.6.1" hasn't updated yet and jQuery.it is still there - I assume that it'll be remedied as the Google spider makes more progress). Thanks again!Yep, that'll do it :) Glad you found it and fixed it quickly!
One follow-up thought for you and anyone in this situation: Set up a Google Webmaster Tools account ( http://www.google.com/webmasters/tools/ ) and make reviewing it part of the webmasters' daily routine. In this particular case, the Crawl Errors page in the Diagnostics section would have flagged this problem very quickly.
A couple of pro tips for Webmaster Tools:
1. Be sure to look at the "date detected" column because it is accurate and somehow people miss it.
2. Set up email forwarding for messages Webmaster Tools sends to you: http://www.google.com/support/webmasters/bin/answer.py?answe...
This is the real problem here, because people do have a lot of trust that very highly-ranked results on Google won't hurt them.
That said, I find it funny that there is so much vitriol against Google in this case when, as has been noted above, it's likely a misconfiguration on jquery.com's part.
I don't think your explanation holds. Pagerank is based on reputation (created by links and other means which Google isn't very specific about), more than on contents. jquery.com has a pagerank of 8, so it can't be all inaccessible to Google. The GP says that blog.jquery.com is inaccessible. So how does jquery.it get a pagerank of 8? Your explanation would look correct if jquery.com had a pagerank of 1 (due to misconfiguration) and jquery.it sneaked in with a rank of 2. But this isn't the pattern.
However, this still doesn't explain why jquery.it has such a high pagerank. And blog.jquery.it has a pagerank of only 1. I can't find any interesting-looking links to jquery.it that would explain the high rank. It's also strange that Google returns blog.jquery.it 1st for the search "jquery 1.6.2", since those strings are found at lots of highly ranked blogs (pagerank>1 for sure). We could be dealing, once again, with Google's preference for finding search terms in the domain name. But this puzzle is still not coming together for me. This is why I'd like wiser persons than me to try to get to the bottom of this.
EDIT: 1 more datapoint: bing doesn't return anything from *.jquery.it for the search in question.
I work at Google as a Webmaster Trends Analyst to help webmasters with issues like this one.
Looking into this, the first thing I noticed is that the blog.jquery.com seems to be blocking Googlebot from fetching its pages, but the site responds normally for web browsers: it returns an HTTP 500 error headers for requests using a Googlebot user agent. You can see this yourself using a public tool like Web Sniffer to fetch the page spoofed as Googlebot ( http://web-sniffer.net/?url=http://blog.jquery.com/2011/06/3... ) or using Firefox with the User Agent Switcher and Live HTTP Headers addons.
Unfortunately this is a very common problem we see. Most of the time it's a mis-configured firewall that blocks Googlebot, and sometimes it's a server-side code issue, perhaps the content management system.
Separately from that, I also notice that the blog.jquery.it URL is redirecting to the blog.jquery.com, suggesting they are fixing it on their end too.
If an jquery.com admins want more help, please post on our forums ( http://www.google.com/support/forum/p/Webmasters?hl=en ).
Cheers,
Pierre