News for hackers. Stuff that matters.
Re:Google is Breaking the Internet by buro9 on Sunday May 18, 2014 @03:43PM (#7764110)
I have been complaining of this problem for a few years now and it's getting worse. I now receive several new "link removal" requests per day (I receive many more repeats of old removal requests).

Yes, Google isn't sending these so you could argue that Google are not directly to blame.

But, the merchants in question who are sending them are doing so because Google has made them believe they are (or will be) subject to some penalty if these links are not cleaned up.

I run forums and all of our links are natural links. For the first year I would look at every link that was asking to be removed and found that, without a single exception, every link was created by a trusted person in the community helping to answer a question by someone else. Every link I checked was natural and high quality, none... literally none... were suspicious or spam.

It may be that these merchants did involve SEO crews who did create a lot of spam links, but those are not on our site.

What's more worrying is that merchants feel so overwhelmed (numerically there are a lot of links to deal with) and in such a state of panic about this (they fear a bad placement could have significant effect on their business) that now they're hiring people to clean up SEO mess, and those people create 4 problems:

1) They have no clue which links were the result of SEO, so they just request every link they can to be removed.

2) They send these removal requests from emails like [email protected] when the links point at www.watches2u.com so it is impossible to verify as a publisher that the person requesting link removal represents the merchant to be de-linked (and #1 means that they don't enter into dialog as they can't, they're sending tens of thousands of requests).

3) The removal notices seldom have enough info to be acted on. One forum I run has several million posts, and the removal request may simply be "You've linked to blah.com, please remove it". OK, more verbose and almost identical to the OP's example, but not a useful pointer to the page or comment that contained the link.

4) If, as a publisher, one fails to remove a link, then their automated removal request scripts will now spam the publisher routinely with the same request.

If you can finally get a conversation that says "Do you represent the merchant, have you got a full URL to where the link exists on our site, and are you sure this isn't a natural and good link?", then every merchant has so far agreed that the link should stay.

But... for all of that, the effect is very real for the publisher: We get spammed evermore, from unaccountable sources that we don't know has a link to the merchant, asking for links to be removed that are actually very good quality and with good context and content.

It's a mess. A mess for which the merchants may be to blame (for originally doing SEO, and now for handling removals so badly), but this mess is definitely due to Google creating the situation.

My policy now: I put link removal requests in my spam folder.

That is how I regard it, this is just a new type of spam that Google has created. We never engaged in SEO or supported anyone doing so, we took an aggressive stance against spam, but as we have content that links elsewhere we are being spammed constantly with these requests and we carry the burden of the problem.

PS: The example cited above was just the top one in my spam folder, the link can be found on this post in the form of an img tag: http://www.lfgss.com/thread17604-52.html#post2119173 , it's the photo of a watch posted on a horology thread within a cycling forum by a long-standing member of the community with high karma. The request for link removal did come from an @web-marketing-group.org.uk address. The request will be ignored. We don't do "nofollow", but it's irrelevant as that isn't valid on img tags.

Re:Google is Breaking the Internet by Matt_Cutts on Sunday May 18, 2014 @03:55PM (#7764155)
It's not a huge surprise that some sites which went way too far spamming for links will sometimes go overboard when it's necessary to clean the spammy links up.

The main thing I'd recommend for a site owner who gets a fairly large number of link removal requests is to ask "Do these requests indicate a larger issue with my site?" For example, if you run a forum and it's trivially easy for blackhat SEOs to register for your forum and drop a link on the user profile page, then that's a loophole that you probably want to close.

But if the links actually look organic to you or you're confident that your site is high-quality or doesn't have those sorts of loopholes, you can safely ignore these requests unless you're feeling helpful.

Re:Google is Breaking the Internet by buro9 on Sunday May 18, 2014 @04:01PM (#7764183)
> For example, if you run a forum and it's trivially easy for blackhat SEOs to register for your forum and drop a link on the user profile page, then that's a loophole that you probably want to close.

It isn't, and never has been.

We operate a nursery period ( https://www.lfgss.com/thread9350.html ) in which the actions of new members are both severely restricted and observed by existing members, and all initial content must be 100% public and observable by members (no private messages, no user profile customisations).

We very aggressively remove content and ban anyone who creates any content that appears spammy and continue to encourage reports of spam even long after they have been registered.

We report those individuals to http://www.stopforumspam.com/ at the same time preventing signups from anyone that SFS reports as coming from an IP recently used to spam.

Comments are run through the Akismet API and moderated if they are flagged as spam.

When a spammer is caught (a few times a year someone does get past all of this - for a very brief moment), then every piece of content of theirs, and any other content from nearby IPs is reviewed and usually removed.

We made profile pages non-public, we even removed the customised profile fields from display (though we do leave them in the form as it helps us spot any scripts that are trying to spam us and we autoban them).

To put it mildly, we are aggressive.

We've made a really poor experience for new members of our community to help ensure the quality of the content on the site and that we're free from spam. We felt that was worth it from day one, we still feel that it's worth it.

Which is why it's hard to feel good about Google when our reward for fighting spam on our corner of the internet is to be spammed by merchants whipped into a state of panic by Google.

Re:Google is Breaking the Internet by Matt_Cutts on Sunday May 18, 2014 @04:18PM (#7764253)
buro9, this is a great example of a well-maintained forum, and as such I agree that you're doing the right thing by ignoring link removal requests.

Unfortunately, if you look at wider area of forums as a whole, a very large fraction of them are unmaintained, unpatched, or otherwise not kept safe from drive-by spamming. There are entire forum spamming software packages that will register spam users for dozens of different forums in order to drop spammy links. You can even lease the spamming software by the month. :(

Re:Google is Breaking the Internet by buro9 on Sunday May 18, 2014 @06:29PM (#7764664)
Also... what happens when, as a consequence of us ignoring requests, Google decide that a site with a high number of disavowed links is a spammy site?

I figure already that you'll eventually consider that a valid signal, but I'm sure you could already check my site and that there must already be some disavowed links, yet you personally admit this is a well-maintained forum.

There is a fear that we are being caught in the crossfire, and it's hard not to believe that quite a few of us are going to get hit by that crossfire.

Re:Google is Breaking the Internet by buro9 on Sunday May 18, 2014 @05:10PM (#7764442)
Thanks, and I agree. But... it doesn't resolve the problem that publishers who always fought spam, who stayed as far from the SEO battles as possible, and who have largely spam-free sites, are now subject to a constant and increasing flow of link removal requests as a consequence of Google's pushback against the merchants who did use the more spammy SEO tactics.
Re:Google is Breaking the Internet by liveoneggs on Sunday May 18, 2014 @05:20PM (#7764479)
except when he is disavowed as all the removal emails say they will do.

I've added rel="nofollow" as good-enough when trying to help out people throwing too-far of a net.

Re:Google is Breaking the Internet by OPA_Member on Sunday May 18, 2014 @04:11PM (#7764219)
And there it is again, the point that the author of the article is making . .

"Do these requests indicate a larger issue with my site?"

When the truth of the matter is that it's Google's issue. Why does the burden of the cost have to be on the publisher?

This is the 3rd time someone is pointing this out, and it pretty much showcases "Google"'s attitude on this.

At this point the cost of becoming a good source of information has increased greatly to the point that very few publishers with resources can actively manage all the rules and "guidelines" Google decides to throw at the content creators that provide the content that makes the search engine valuable.

I see this everyday in our business as smaller but richer sources of content just disappear on many topics. Blogs, forums and independent enthusiast content sources.

Google has created a one sided ecosystem. That is consolidating traffic to so called "good" sites.

As a long time publisher for a large media company I think it's time to take the internet back from Google. Who care's what Google thinks and does and start building for a better internet.

Re:Google is Breaking the Internet by 205guy on Sunday May 18, 2014 @06:19PM (#7764643)
I've read the thread up to here, and I do think Google is to blame for some heavy-handedness and some rather tone-deaf spokespeople (not that Matt_Cutts ever identified himself as speaking for Google, though he acts like it). But I think the bigger point that everybody is missing is that the Internet has become awash in all sorts of information, "good" and "bad," well-curated and spammy, human- and machine-generated, white-hat and black-hat. Just because YOU post information, and post a blog article about how it is good information, doesn't make it so or not so to an objective observer.

Google is what it is: trying to sift through all that information algorithmically and using some forms of AI so it can extract what humans think is good information. I agree that a good forum that has good policies is getting hurt by some of google's actions, but it's the larger system, not Google. To make an analogy: almost everybody drives and everybody uses roads, but nobody should say: I'm a good driver or I don't drive, I've never been in an accident, therefore I cannot be in an accident. With all the information and intents behind such information on the Internet, and with google as the self-appointed traffic laws, there is still bound to be some collateral damage, sometimes because of how Google tries to deal with the dreck.

As an aside, one can view Google as a just one way of monetizing information on the Internet, competing and co-existing with many other ways. You can also view it as a giant AI algorithm that gets rewarded (advertising revenue) when it finds the good information that humans are looking for and defends itself against others who have other interests.

Re:Google is Breaking the Internet by mike_hearn on Sunday May 18, 2014 @07:04PM (#7764788)
What are people "pointing out" exactly? This is a ridiculous argument.

If you don't care about what Google's view of the internet looks like, then ignore them, and ignore any emails from people asking you to change things on your site for them. Done. Simple.

This issue crops up because people usually do care what Google's view of the internet is and would like to help make it better. Marking links as nofollow and so on is entirely optional: only people who want useful search engines do it, and others don't care.

Re:Google is Breaking the Internet by mcintyre1994 on Sunday May 18, 2014 @04:53PM (#7764381)
Is HN an example of a site that makes it trivially easy to sign up and drop a link on the user profile page?
Re:Google is Breaking the Internet by RealGeek on Sunday May 18, 2014 @06:11PM (#7764625)
Yes, we get tons of spams on our blogs and forums.

Thanks to Google for creating these parasites. The link-spam problem has been caused by Google in the first place.

I don't remember getting such spam in a pre-Google Internet world.

Small website owners like us may not have the technology and human resources to combat such spam. Therefore, we are caught up in the fire and get penalized by Google.

Re:Google is Breaking the Internet by us0r on Sunday May 18, 2014 @08:59PM (#7765147)
" "Do these requests indicate a larger issue with my site?" For example, if you run a forum and it's trivially easy for blackhat SEOs to register for your forum and drop a link on the user profile page, then that's a loophole that you probably want to close."

Wouldn't that be like every vBulletin site?

Re:Google is Breaking the Internet by josephjrobison on Monday May 19, 2014 @01:04AM (#7765789)
Are you not able to simply ignore bad links because when you did that years ago, people would just pump and dump thousands of links to see what stuck?