News for hackers. Stuff that matters.
Re:How rel=nofollow Works by chalst on Thursday January 12, 2012 @08:06AM (#3456163)
The cited Danny Sullivan article has a little of his interview with Eric Schmidt:

sullivan> I countered that Google seemed to have all the permission it needed, in that they’re not blocked from crawling pages.

schmidt>“That’s your opinion,” Schmidt said, then joked: “If you could arrange a letter from Facebook and Twitter to us, that would be helpful.”

sullivan> I pushed back that both have effectively given those letters since their robots.txt files — a method of blocking search engines — weren’t telling Google to go away.

Well, why stop there? Why shouldn't Google ignore the robots.txt file in its search for shareable nuggets for its search results, which they are equally not "blocked" from using? The answer for both rel=nofollow and robots.txt is that Google has explicitly promised webmasters that it will not do this. Sullivan knows this: this is bad journalism. I'd be curious to see more of the transcript of the interview.

Re:How rel=nofollow Works by abalone on Thursday January 12, 2012 @03:53PM (#3457909)
That doesn't make sense. Twitter's robots.txt allows profile pages and tweets to be indexed, and Google does index them. Google is simply choosing to exclude them from their people & places feature so that they can favor their own Google+ profiles.

It's only the outbound links within tweets that Google is forbidden from indexing. That has nothing to do with why Google excludes content on twitter.com itself.

Re:How rel=nofollow Works by nknight on Thursday January 12, 2012 @08:30PM (#3458756)
> That has nothing to do with why Google excludes content on twitter.com itself.

Um, they don't. That's not what this is about. You need to research the actual feature that's causing controversy, because it's not "Twitter doesn't show up in search", not even close.

Re:How rel=nofollow Works by abalone on Thursday January 12, 2012 @08:43PM (#3458787)
The People & Pages feature. On Google search. Is excluding twitter.com content.

Example: http://parislemon.com/post/15682237911/twitter-keeps-right-o...

Re:How rel=nofollow Works by nknight on Thursday January 12, 2012 @09:51PM (#3458954)
So what? That's not part of the standard search. Since when is Google not allowed to advertise its own services alongside standard search results? What's next, do you whine that Bing has links to MSN and Hotmail?

Twitter actually contaminates ordinary search results on its site with paid tweets, and you're mad Google puts some Google+ content off to one side? WTF?

Re:How rel=nofollow Works by abalone on Friday January 13, 2012 @07:57PM (#3463120)
Oh, it's definitely part of standard search now. Schmidt said Twitter content wasn't included because they didn't have "permission" to index it. That's false. They are purposefully excluding it -- a point of fact which I'm glad to see you acknowledge now.
Re:How rel=nofollow Works by nknight on Saturday January 14, 2012 @12:17AM (#3463538)
No, it's not part of standard search. And since you seem to know everything that goes on between Twitter and Google, perhaps you could share with us the documents showing Google has the permission it needs to include Twitter results in this very not-standard-search feature?

Edit: By the way, are you a lawyer? If not, even given your apparent insider knowledge of Google and Twitter goings-on, how do you know your legal understanding is correct?

Re:How rel=nofollow Works by abalone on Monday January 16, 2012 @05:32PM (#3472902)
Sure, I'd be happy to. Here it is: http://twitter.com/robots.txt
Re:How rel=nofollow Works by masklinn on Thursday January 12, 2012 @09:14AM (#3456346)
> The answer for both rel=nofollow and robots.txt is that Google has explicitly promised webmasters that it will not do this.

No, not for a/@rel="nofollow", the only promise Google does is that it will not transfer pagerank credit to the link's destination[0].

This does not prevent them from using that link as an outbound in order to discover new content, nor does it prevent Google from reversing the link for the "Shared On" feature. Google specifically advocates[0] using a/@rel=nofollow for user-generated links in order to prevent rewarding spammers, Twitter's use of the attribute is not just sensible, it's necessary.

niyazpk's comment[1] looks far more sensible: "Shared On" is a specific feature for content producers and part of a special agreement and API access.

[0] http://googleblog.blogspot.com/2005/01/preventing-comment-sp...

[1] http://news.ycombinator.com/item?id=3456233

Re:How rel=nofollow Works by jasonlotito on Thursday January 12, 2012 @09:27AM (#3456404)
> the only promise Google does is that it will not transfer pagerank credit to the link's destination

Google is fairly clear about this:

http://support.google.com/webmasters/bin/answer.py?hl=en&...

"How does Google handle nofollowed links?

In general, we don't follow them. This means that Google does not transfer PageRank or anchor text across these links. Essentially, using nofollow causes us to drop the target links from our overall graph of the web. However, the target pages may still appear in our index if other sites link to them without using nofollow, or if the URLs are submitted to Google in a Sitemap. Also, it's important to note that other search engines may handle nofollow in slightly different ways."

So, if your tweet includes a link, they drop that link from the post as far as their graph is concerned. And that graph is what makes up their search.

It's not just page rank.

Re:How rel=nofollow Works by masklinn on Thursday January 12, 2012 @10:07AM (#3456531)
Interesting, so their implementation has broadly expanded upon the original meaning of the attribute (and the one standardized as a microformat), and in recommending this be broadly used to fight against spam they've paved the way for an interesting time for everybody else?
Re:How rel=nofollow Works by jasonlotito on Friday January 13, 2012 @08:16AM (#3460401)
Broadly expanded? Standardized? Please.

I think the nofollow attributes intent was fairly straight forward from the beginning: to designate links that should not be accorded attention in search results. In 2005, this was merely via PageRank. Today, it's recommendations in search results.

Why would I want to see search results that shouldn't be given weight in a search result in a search result? Explain that to me.

Re:How rel=nofollow Works by darkane on Thursday January 12, 2012 @09:33AM (#3456427)
Where in Google's blog post do they claim to still follow the link? They provide no technical details for their process, so if we're speculating, it would make far more sense that rel=nofollow adheres to the same guidelines as a meta/robots.txt nofollow.

And this speculation for the "shared on" functionality doesn't exactly add up, either. If they're not following links with rel=nofollow, why should Google be legally or even morally responsible for storing a massive and entirely extraneous relational database, and then checking against it every time they follow a short URL from another site, just to see if it was shared on Twitter at some point?