News for hackers. Stuff that matters.

A comment on How My Popular Site was Banned by Google. Parent comment

Re:How My Popular Site was Banned by Google by thenextcorner on Tuesday October 11, 2011 @06:45PM (#3100729)
Literally Google Webmaster Guidelines say: "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search engines."

So a couple of questions: - What is of value for a user? - Who is determining the value for the users in these cases?

As always, it's not always quite clear on what treatment you should use for search pages!

[ Parent | on HN ]
Re:How My Popular Site was Banned by Google by dlitz on Tuesday October 11, 2011 @10:39PM (#3101267)
I'm pretty sure that having different search engines indexing each other would be bad. Don't cross the streams.
[ Parent | on HN ]
Re:How My Popular Site was Banned by Google by thenextcorner on Wednesday October 12, 2011 @06:37PM (#3104953)
If I recall correctly, crossing the streams did kill marshmallow man!

In any case, I agree with you, search engine indexing search results would be bad, but the line is not that clear all the time!

Some vertical search engine result pages are a great and relevant result from a user perspective on the question they are trying to solve.

[ Parent | on HN ]