News for hackers. Stuff that matters.
Re:How we got 100,000 visitors without noticing by mmaunder on Saturday October 02, 2010 @11:49PM (#1752038)
Eek, not to rain the parade, but this is not good historious team. Google's duplicate content filter is generally accurate but in this case it penalized the site that originated the content and ranked yours higher because it assumed you were the owner of the content i.e. the page about sending free SMS's.

Google will quickly correct this error and you'll find the traffic on that page drops to nothing. You may also find other duplicate pages on your site penalized in the same way. You may also find your site penalized by Google for essentially screen scraping sites and copying their content verbatim and in it's entirety.

I'd recommend you set up a robots.txt to block Google from indexing identical pages on your site. Copying content verbatim and republishing in it's entirety is not a good SEO strategy unless you have thousands of throw-away domains and wear a black hat.

Sorry about the negative message, I'm sure this was very exciting for you guys but I don't think it's sustainable.

Perhaps an alternative is to have your users highlight the paragraph or snippet on each page they found interesting and archive that instead. Then publish those snippets on your site. Generally I've found republishing paragraph size chunks of text is OK with Google and will net you decent SEO traffic. It worked for my job search engine when we republished job descriptions and limited them to the first 400 chars. You can also mash the paragraphs up into pages with multiple chunks of text that e.g. show all chunks of text a particular user found interesting or a set of users or by date or location. That will give you the best of both worlds - lots of content to SEO and no dup content penalty.

Best of luck!!

Re:How we got 100,000 visitors without noticing by JeremyChase on Sunday October 03, 2010 @12:11AM (#1752057)
Or set the canonical address of your cached pages to the real page.
Re:How we got 100,000 visitors without noticing by StavrosK on Sunday October 03, 2010 @03:02AM (#1752278)
Do you mean the "base" tag? We already do set that (admittedly, after this happened and because of it).
Re:How we got 100,000 visitors without noticing by fhars on Sunday October 03, 2010 @03:16AM (#1752298)
No, he meant the rel="canonical" links.
Re:How we got 100,000 visitors without noticing by patio11 on Sunday October 03, 2010 @09:48AM (#1752698)
This does not work across domains, guys. (Among other reasons, it would turn injection attacks into $X million bugs for some companies.)
Re:How we got 100,000 visitors without noticing by StavrosK on Sunday October 03, 2010 @12:39PM (#1753014)
The google blog says it does...
Re:How we got 100,000 visitors without noticing by StavrosK on Sunday October 03, 2010 @06:13AM (#1752460)
Oh, that sounds easy enough to do, we'll implement it right away (it should be live in a few minutes. Thanks for the tip, we didn't know about this duplicate content situation...

EDIT: Added!

Re:How we got 100,000 visitors without noticing by davecardwell on Sunday October 03, 2010 @03:40AM (#1752333)
Re:How we got 100,000 visitors without noticing by StavrosK on Sunday October 03, 2010 @06:32AM (#1752475)
Thank you, added!
Re:How we got 100,000 visitors without noticing by JonM on Sunday October 03, 2010 @08:13AM (#1752575)
I honestly didn't know this existed.... thanks!
Re:How we got 100,000 visitors without noticing by StavrosK on Sunday October 03, 2010 @03:01AM (#1752276)
Ouch, we didn't know that :/ Thank you, I'll block robots from the cache right away!
Re:How we got 100,000 visitors without noticing by StavrosK on Monday October 04, 2010 @03:45AM (#1754937)
For the record, something I forgot to mention. All the ads load for the site of the cached page, and all the links lead to it. So, the only way the original site is impacted is that they don't have to bear the load of 50k visits on their site. Otherwise, they get all the ad revenue, signups, links, etc.

Essentially, we're the only ones who don't gain anything from this.