News for hackers. Stuff that matters.

Has RapGenius been getting away with cloaking?

This is a type of URL that RapGenius ranked very well for before the assassination: http://rapgenius.com/35640/The-notorious-big-suicidal-thoughts/I-like-black-timbs-and-black-hoodies

That page, along with many others for every song, 301s back to the full song for a user with an anchor for the specific line/note.

However, Google keeps the URL that redirects as a top search result and keeps a text only cache that looks like this: http://webcache.googleusercontent.com/search?q=cache:http://rapgenius.com/35640/The-notorious-big-suicidal-thoughts/I-like-black-timbs-and-black-hoodies&strip=1

If this isn't cloaking, why is Google keeping a URL that 301s in their SERPs, giving it top ranking, and keeping a separate cache?

( 12 comments | 22 points | on HN )
Re:Has RapGenius been getting away with cloaking? by nightpool on Thursday December 26, 2013 @02:50PM (#6967689)
I don't believe so, I think instead that the text-only cache you're seeing is what is showed to all clients with javascript disabled. They use javascript to show the annotations on the page and as a fall back they have a stand-alone page that shows the annotation. The same thing happens on a mobile browser, because they don't have the screen real estate to show the annotation in context. So, no, not cloaking, just actual no-javascript accessibility. Something a lot more sites should be doing, in my opinion.

Even if this was a Googlebot-only page, it still wouldn't be cloaking, as they're just showing a more search-engine friendly version of the same content that's in the annotation on the actual page.

Re:Has RapGenius been getting away with cloaking? by Ihaveaproblem on Friday December 27, 2013 @09:37AM (#6970718)
I have never heard of Google keeping cache's of pages in javascript-disabled states, Googlebot has been processing JS for a long time. And how many users (even considering mobile) really don't have JS these days? This may not be by-the-definition cloaking, but it's right up on the line for sure.
Re:Has RapGenius been getting away with cloaking? by DanBC on Friday December 27, 2013 @10:21AM (#6970891)
I'd be interested in numbers, but maybe 3% of users turn off JavaScript.

http://stackoverflow.hewgill.com/questions/121/108.html

Re:Has RapGenius been getting away with cloaking? by Casseres on Friday December 27, 2013 @10:03AM (#6970806)
I and quite a few other HN users use NoScript. It blocks JavaScript per domain name automatically unless you whitelist it. It's great for privacy and security.
Re:Has RapGenius been getting away with cloaking? by bushido on Thursday December 26, 2013 @11:51AM (#6966814)
Would have to investigate this further. But you may be correct.

I haven't seen this one done in years.

It would have been legitimate if the user saw the same content as the crawlers, but displaying different content to SE crawlers would definitely be bad.

If confirmed, this could turn a bad situation in to a ghastly one.

Re:Has RapGenius been getting away with cloaking? by Splendor on Thursday December 26, 2013 @12:11PM (#6966899)
Cloaking, for those who are unfamiliar with the term like me: http://en.wikipedia.org/wiki/Cloaking
Re:Has RapGenius been getting away with cloaking? by mikkelewis on Thursday December 26, 2013 @12:49PM (#6967065)
This was cloaking. Prior to link scheme fiasco they used a javascript redirect on the lyric page, effectively making the the lyric page different from what the user sees and what the bot sees.

I think the 301s are actually their response to Google's ban hammer.

Re:Has RapGenius been getting away with cloaking? by Ihaveaproblem on Thursday December 26, 2013 @12:52PM (#6967079)
Well I can't prove it, but I first noticed this about three weeks ago and I'm 100% positive they haven't changed anything since then. They may have used JS redirects further back in the past, I'm not sure.
Re:Has RapGenius been getting away with cloaking? by mikkelewis on Thursday December 26, 2013 @01:18PM (#6967208)
Override your user agent with Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html). You'll see that both a regular user and a google bot are getting served a 301. So I don't think it's cloaking by definition anymore. However if you go to:

http://webcache.googleusercontent.com/search?q=cache:http://...

You'll see that you get redirected. But disable javascript in your dev tools, and you'll see that you no longer get redirected :)

Re:Has RapGenius been getting away with cloaking? by Ihaveaproblem on Thursday December 26, 2013 @01:23PM (#6967231)
Ya, I did the user agent testing three weeks ago and got the same results I am getting today. I'm certainly not keeping close track of them, so maybe they did change something, but everything I see is 1:1 as it was when I first saw it. Perhaps they are detecting Googlebot via IP instead of user agent. I dunno.
Re:Has RapGenius been getting away with cloaking? by donthe on Thursday December 26, 2013 @01:30PM (#6967271)
I'm seeing a 302 now. http://imgur.com/yOWhanl
Re:Has RapGenius been getting away with cloaking? by badapple on Friday December 27, 2013 @07:39PM (#6973348)
what about others that are buying links???

for example TINT

tintup.com

once you create a free account they have the following:

Dear valued Tint customer, Would you like to write a review about Tint? In exchange for your opinions/blog post, we are giving away 50% off FOREVER promo code for our Plus Plan. To learn more, click the button below:

screen shot: http://imgur.com/wDnAJ4c