News for hackers. Stuff that matters.
Re:Website outages and blackouts the right way by PaulMest on Monday January 16, 2012 @04:04PM (#3472410)
Wouldn't it be possible to detect if it's a web crawler visiting your site (e.g. user agent) and then let them crawl the site as normal? If it's not a web crawler, then display the "blackout" version of the site?
Re:Website outages and blackouts the right way by wmf on Monday January 16, 2012 @04:34PM (#3472532)
That's cloaking and can cause you to be de-indexed.
Re:Website outages and blackouts the right way by PaulMest on Monday January 16, 2012 @05:33PM (#3472905)
Interesting. This site claims that the intent behind the cloaking is taken into account: http://www.smart-it-consulting.com/article.htm?node=148&... ... Do you agree? Or have you had a different experience?
Re:Website outages and blackouts the right way by pdkp on Monday January 16, 2012 @06:57PM (#3473164)
Yes, at least in the past, Google has taken into account intent when it comes to cloaking, letting some big sites get away with it and still rank well in serps.

However, I can see how your initial suggestion of showing full content to Google, could be viewed as solely for preserving rankings in an artificial manner.

As an example of cloaking, some News Sites let Google index all their pages, while requiring actual users to login/register to view it.

Typically, if the user has a Google Referrer, they can view the page one time for free and then need to login/register to view anything else.

Visiting the page directly or with a non-google referrer shows a register/login page.

New York Times was one that does(did?) this. I stopped visiting them when they started. I think Washing Post, or one of the posts, was doing it too, as well as a number of other sites.

Experts Exchange used to basically be the same way, although I think they are doing it differently now, and they were slapped by Google a long time ago for cloaking, so changed to a different method of cloaking...