Re:ITunes is Anti-Web
by Tichy on Thursday July 10, 2008 @11:06AM
(#242459)
I think you meant "scan", not "ban"? I was wondering about that, because the marketing of my iPhone app would not be very far reaching if it could not even be found by Google.
[ Parent
| on HN ]
I mean "ban". Google requires that all pages make no distinction between the indexer and ordinary browsers and should return same data to both. In the case of that link I gave or any other link to itunes.com, what you see doesn't look the same as what Google sees (see Cached). They have the right to ban the site, I think.
Actually, I think that is the same thing that you would get in the iTunes store if you could see the raw XML. So what distinction does it make, exactly?
The problem is, that page doesn't require Google to use iTunes for viewing its contents. In fact, as you can see the output of wget from my other comment, iTunes tricks both Google and us by sending Content-type: text/html.