News for hackers. Stuff that matters.
Re:ITunes is Anti-Web by mojuba on Thursday July 10, 2008 @10:26AM (#242401)
On top of that, Google has the right to ban them. For example: http://www.google.com/search?q=site:itunes.com+pink+floyd
Re:ITunes is Anti-Web by raganwald on Thursday July 10, 2008 @11:33AM (#242502)
How is that different from Google reaching into PDF documents, Word documents, PowerPoint presentations, or Flash applications?

In all of those cases, end users must use a proprietary application to actually view the content. I clicked on your link and when it opened in iTunes, I saw the same text that Google's spider previewed.

Here is Google's cache:

http://209.85.141.104/search?q=cache:GYFPOvjhhfoJ:itunes.com...

As you can see, the spider saw the full text of the review. The preview in the search page might be some metadata Apple provided as a hint to Google or it may be that Google's spider is really good at extracting interesting sentences :-)

Re:ITunes is Anti-Web by mojuba on Thursday July 10, 2008 @11:49AM (#242529)
It is different in that iTunes.com returns text/html:

  HTTP/1.0 200 OK
  Last-Modified: Thu, 10 Jul 2008 15:46:08 GMT
  Content-Type: text/html; charset=utf-8
  Content-Encoding: gzip
  x-apple-aka-ttl: Generated Thu Jul 10 08:46:08 PDT 2008, 
     Expires Thu Jul 10 08:46:08 PDT 2008, TTL 0s 
  x-apple-application-instance: 6105
  x-apple-asset-version: 46436
  x-apple-max-age: 0
  x-apple-request-store-front: <null>
  x-apple-date-generated: Thu, 10 Jul 2008 15:46:08 GMT
  x-webobjects-loadaverage: 0
  Content-Length: 1940
  Vary: Accept-Encoding
  Expires: Thu, 10 Jul 2008 15:46:08 GMT
  Cache-Control: max-age=0, no-cache, no-store
  Pragma: no-cache
  Date: Thu, 10 Jul 2008 15:46:08 GMT
  Connection: keep-alive
  X-Apple-Partner: origin.0
Now, why should this be handled by a custom application? Isn't it called breaking the Web?
Re:ITunes is Anti-Web by raganwald on Thursday July 10, 2008 @11:56AM (#242546)
Have you used iTMS? It does a bunch of stuff not handled by any browser without proprietary plug-ins. So either (a) they have to cut functionality, (b) they have to give you one or more proprietary plug-ins to give you the same functionality in another browser, or (c) provide an application that is also a site-specific web browser, which is what they have done with iTunes.

If it is a choice between (a), (b), and (c), which choice would you make if you were a product manager for Apple? Which choice would you make if you were an iPod owner?

Re:ITunes is Anti-Web by mojuba on Thursday July 10, 2008 @12:05PM (#242554)
raganwald, the only problem here is, that page does not require Google to use iTunes software for viewing the content. This alone is sufficient for banning the web site and removing it from Google's indexes.

I used wget.

Re:ITunes is Anti-Web by raganwald on Thursday July 10, 2008 @12:41PM (#242601)
"Banning the web site and removing it from Google's indexes."

You and I have a very different understanding of why Google does and does not ban web sites. They ban web sites for SEO black hattery.

In this case, Apple presents information that people searching Google want to find. Why on Earth would it benefit Google to ban this practice?

Are you equating the iTunes application to some sort of drive-by-download malware? If so, please make that point and not frame this as a violation of Google's quasi terms of service.

Do you think that Google is unaware of what is happening, that Apple is fooling them? Send them an email and let them know how Apple is tricking their spiders.

Re:ITunes is Anti-Web by mojuba on Thursday July 10, 2008 @01:11PM (#242644)
Of course I'm not going to fight the windmills, and sure Google is aware what's happening.

What Google's policy is, as I understand it, to index information that's openly available to every host with an IP address. Google wants to see the same content as any other host on the Net can see and that's, you know, pretty much in the spirit of the Internet.

Going against these principles means going against the Web and breaking it. Good or bad, designed perfectly or not, and no matter whether the world will become a better place to live thanks to iTunes and Apple, but this is anti-Web.

Re:ITunes is Anti-Web by Tichy on Thursday July 10, 2008 @11:06AM (#242459)
I think you meant "scan", not "ban"? I was wondering about that, because the marketing of my iPhone app would not be very far reaching if it could not even be found by Google.
Re:ITunes is Anti-Web by mojuba on Thursday July 10, 2008 @11:19AM (#242471)
I mean "ban". Google requires that all pages make no distinction between the indexer and ordinary browsers and should return same data to both. In the case of that link I gave or any other link to itunes.com, what you see doesn't look the same as what Google sees (see Cached). They have the right to ban the site, I think.
Re:ITunes is Anti-Web by silencio on Thursday July 10, 2008 @11:51AM (#242536)
Actually, I think that is the same thing that you would get in the iTunes store if you could see the raw XML. So what distinction does it make, exactly?
Re:ITunes is Anti-Web by mojuba on Thursday July 10, 2008 @11:59AM (#242549)
The problem is, that page doesn't require Google to use iTunes for viewing its contents. In fact, as you can see the output of wget from my other comment, iTunes tricks both Google and us by sending Content-type: text/html.