News for hackers. Stuff that matters.
Re:Trouble In the House of Google by cletus on Monday January 03, 2011 @07:40AM (#2062938)
This is really just a rehash of other posts from the last month (linked in article).

This post basically complains about two things: the finer points of SEO and content farms.

Content farms is an easy one. They're the Web equivalent of spam and I'm talking about the likes of Associated Ontent and Demand Media. They re a relatively new (last few years) phenomenon.

My personal view is that no one is better placed to deal with this new threat than Google. Email spam is basically a solved problem on Gmail. Thats not ss there aren't false positives and negatives but it's oohing like it used to be or could be. It'll take time but I believe that content farms are a transitory and doomed business model.

As for product searches, this encompasses many things. Anecdotally I recently searched for "<camera make and model> review" and found what I wanted no problem. Prices I found on pricegrabber (they have an iPad app).

SEO is a trickier beast. For one it's a constantly moving target. A combination of suboptimal source SEO and content farm SEO gaming allows the scrapers to survive. I can't say that keyword position matters all that much. Anecdotally Jeff claims it does but many factors are at pay so it's always best to be careful about making absolute claims.

Jeff claims not to want to be acquired. I'm reminded of a story I heard. Basically: if you wanted money (from angels) ask for advice. If you wanted advice, ask for money (IIRC this story came from either Mark Suster or Jason Calacanis, can't remember).

So, if you want to be acquired, say you don't?

Lastly, I'll reiterate my own opinion that social search isn't the answer in the general case (ie it will have specific use cases).

Content curation is a mixed bag. I believe there will (for at least a very long time) be a place for niche verticals. For example, dpreview is a vertical for cameras. General purpose models like Mahalo I think are doomed for much the same reason that Jeff and Joel have contended that general Q&A sites are doomed.

Re:Trouble In the House of Google by mootothemax on Monday January 03, 2011 @07:44AM (#2062946)
Email spams basically a solved problem on Gmail

I don't want to move the discussion away from web spam, but I disagree with your statement. I have email for several domains hosted on Google (as well as a regular Gmail address) and they all suffer from legitimate emails regularly going into the spam folder. A quick search has shown that I'm not the only one suffering from this problem.

Re:Trouble In the House of Google by Silhouette on Monday January 03, 2011 @10:47AM (#2063438)
I concur. Spam is only a "solved" problem on services like Google Mail if you consider a high rate of false positive detections to be a solution. I suspect this is one reason that many of my friends now use Facebook as their default messaging system -- which is a pain for those of us who don't want to use Facebook, and even more so if we send our e-mails from a personal domain that tends to get arbitrarily flagged as junk by services like Google Mail and Hotmail the first time we write to each friend.
Re:Trouble In the House of Google by eli on Monday January 03, 2011 @11:15AM (#2063558)
If you think gmail has a high rate of false positives, I encourage you to try Hotmail or (especially) Yahoo.
Re:Trouble In the House of Google by ximeng on Monday January 03, 2011 @01:33PM (#2064094)
Yahoo works pretty well for me. Occasional false positives and negatives, but it's generally about right.
Re:Trouble In the House of Google by thesethings on Monday January 03, 2011 @04:05PM (#2064674)
My experience w/Yahoo! mail is surreal. It even marks my own mail as spam. And misses lots of real spam.

(i use gmail primarily, but have an old Yahoo! address for bacn)

Re:Trouble In the House of Google by eli on Monday January 03, 2011 @01:40PM (#2064127)
Really? Hmm, maybe it's because it's a very old address, but my Yahoo spam filtering is terrible.

For example, I get a few very similar Canadian pharmacy junk messages every single day. Even though I usually mark them as spam, they continue to appear in my inbox. It never learns. Additionally about 1 in 4 order confirmation emails ends up in the Junk folder.

Re:Trouble In the House of Google by RyanMcGreal on Monday January 03, 2011 @08:04PM (#2065375)
> Hmm, maybe it's because it's a very old address, but my Yahoo spam filtering is terrible.

My understanding of Bayesian spam filtering is that it's supposed to get better over time, not worse.

Re:Trouble In the House of Google by eftpotrm on Tuesday January 04, 2011 @09:05AM (#2066931)
Supposed to, yes. Until my ISP moved to GMail and hugely upped the spam filtering without telling me (not impressed), I had much the same observation with Thunderbird - its junk filtering was getting less accurate, not more.
Re:Trouble In the House of Google by RyanMcGreal on Tuesday January 04, 2011 @11:11AM (#2067428)
I use Thunderbird and find that spam comes in waves. I'll get a smattering of spam emails in my inbox, flag them as such, and the number tapers toward zero. Then, a month or two later, another smattering will appear, after the spammers realize the filters have adapted and change the composition of their spam emails.
Re:Trouble In the House of Google by fanf2 on Monday January 03, 2011 @11:14PM (#2065826)
If you get any false positives, then it isn't anywhere near right.
Re:Trouble In the House of Google by lwat on Monday January 03, 2011 @06:47PM (#2065173)
I think most people would agree that the Yahoo spam filtering is vastly inferior to Google's
Re:Trouble In the House of Google by silverbax88 on Tuesday January 04, 2011 @08:36AM (#2066840)
I wouldn't agree, but I find Gmail to be awful to use. So I'm already biased.
Re:Trouble In the House of Google by kragen on Monday January 03, 2011 @04:26PM (#2064738)
I have the same problem, and I'm at a loss as to what to do about it.
Re:Trouble In the House of Google by cake on Monday January 03, 2011 @08:10AM (#2063014)
I had the confirmation of an order from an online store directly into the spams recently, I guess too many people dislike their newsletter or something.

It took me a while to figure it out and wasn't sure if my order was properly registered, if I mistyped my email or if it was a problem on their side. Lesson learned, now I systematically scan and empty the spams.

Re:Trouble In the House of Google by true_religion on Monday January 03, 2011 @08:01AM (#2062993)
Whenever I find an efreedom article linking back to Stackoverflow, its on the first page.

The original stackoverflow article on the other hand is on page 3 or 5 or greater.

I'm not sure what Stackoverflow is doing, but in this case efreedom is providing me a service by making this content easily found in a search engine. Sure its one more click, but in my mind the alternative is never being able to find the SO article at all.

My workflow involves looking at multiple articles, so I can't simply put 'site: stackoverflow.com' into my search.

Re:Trouble In the House of Google by cletus on Monday January 03, 2011 @08:08AM (#2063010)
Can you give me some concrete search terms that have this ranking discrepancy? I want to investigate and just haven't seen it yet myself.
Re:Trouble In the House of Google by Hoff on Monday January 03, 2011 @10:28AM (#2063357)
Something has changed here. Aging or a ranking tweak or Phase Of The Moon, something. A search from a couple of days ago is now returning SO above EF, where that wasn't the case a few days ago.

Ah, here's one from the search history with EF above SO:

retaincount instruments

There are several eFreedom hits from the first page, and SO has one hit on page 2 and one hit on page 3.

I have screenshots of the first three pages.

Re:Trouble In the House of Google by Matt_Cutts on Monday January 03, 2011 @12:24PM (#2063850)
Thanks for giving a concrete example. I'd gotten a couple examples when talking to Jeff Atwood earlier and passed that on to the appropriate team, but more concrete examples always help when we're trying to rank the sites in the best order.
Re:Trouble In the House of Google by Matt_Cutts on Monday January 03, 2011 @03:16PM (#2064514)
By the way, if people want to mention more examples of other sites outranking Stack Overflow by replying here, I'll try to circle back to check for them, then use them to poke the relevant team to make sure we're doing everything we can.
Re:Trouble In the House of Google by pbz on Monday January 03, 2011 @04:52PM (#2064835)
I personally would prefer a way to just blacklist sites like this. It's very easy to visually spot a "cloner", and once I found such a site I don't care to see it again. I know this feature has been requested by many others; just adding one more voice to the pile.
Re:Trouble In the House of Google by redthrowaway on Tuesday January 04, 2011 @12:23AM (#2065965)
I'd like them to just have a site to report SEO scammers, where you could just copy and paste the URL. This would make FF/Chrome extensions trivial. Google could rank each submitter in a similar way to how they rank reCaptcha submissions as a way of discouraging abuse.

Still, I think it's becoming apparent that SEO has broken PageRank, and we really are in need of something else. Whether that something else is algorithmic, social, or, most likely, a combination of the two, remains to be seen. Still, I don't think we'll ever have a system that can't be gamed.

Re:Trouble In the House of Google by lazylland on Tuesday January 04, 2011 @11:22AM (#2067469)
Funnily enough, there WAS a way to blacklist/remove sites in the SearchWiki feature ! Could you bring this back, Google ? Pretty pleeeeeeese :))
Re:Trouble In the House of Google by ry0ohki on Monday January 03, 2011 @04:18PM (#2064714)
Here's the only example I have: http://www.google.com/search?q=facebox+value+text&hl=en

Based on my search history I've clicked efreedom links several times, but currently the stackoverflow link is the top one for those searches now. I would be surprised if I had clicked efreedom if stackoverflow was at top.

Re:Trouble In the House of Google by benologist on Monday January 03, 2011 @06:07PM (#2065055)
Heh I get this spammy piece of crap at #1 for that search:

http://pinoytech.org/question/1699545/unable-to-get-value-fr...

Re:Trouble In the House of Google by ry0ohki on Monday January 03, 2011 @04:14PM (#2064700)
I'd love to get more insight into how Google fixes problems like this, like hopefully you don't just hard code a multiplier for stackoverflow.com or a negative one for the spam sites?
Re:Trouble In the House of Google by anonymous246 on Monday January 03, 2011 @02:36PM (#2064374)
HN's "voting based on commenter" at work? This is probably single most useful comment in the entire thread, and Matt's "thank you" note had higher votes than the content? Sorry about the derail.

I've personally not seen efreedom ranking above stackoverflow. I've seen it 2 places down or one down. So this is a very valuable comment. IMHO. YMMV.

Re:Trouble In the House of Google by chrstphrwrght on Tuesday January 04, 2011 @08:18AM (#2066796)
First, searched for: ppa in pbuilder

I received an eFreedom result on the second page of results, and no corresponding Stack Overflow article. The title of the eFreedom page is "DEBIAN - pbuilder storing dependencies - efreedom"

I then searched for: ppa in pbuilder site:stackoverflow.com

This gives one response - not the same article that was scraped by eFreedom. So that seems a little off - as a user I'd expect the Stack Overflow article that was scraped to meet the query if eFreedom does.

Finally, based on the title of the Stack Overflow article that was scraped in my first query, I searched for: pbuilder storing dependencies

Here the original Stack Overflow article is listed first, eFreedom second, so that's good. Unfortunately when you scan a little further down the page, the article "UBUNTU - pbuilder create fail - efreedom" is above "pbuilder create fail - Stack Overflow".

As a side point, my search is not made more valuable by the scraped versions appearing in the results. Making Stack Overflow appear before eFreedom is one thing, but why should I see eFreedom at all? I'd rather the other results be different content, that way I don't have to mentally parse everything to work out if it's the same content with a different skin.

Re:Trouble In the House of Google by jwhite on Monday January 03, 2011 @08:13AM (#2063023)
Apologies for nitpicking, but your post was quite hard to read due to typos. Most of the time my brain auto-translates typos without me realizing it, but the ones in your post really interrupted my reading flow. YMMV.

The content was interesting, thank you. Good point about niche sites like dpreview.

Re:Trouble In the House of Google by bigfudge on Monday January 03, 2011 @04:39PM (#2064785)
I was thinking the same thing, and couldn't even work out how he'd achieved them — commonly mishit combinations seem easy for me to translate on the fly. Dvorak?
Re:Trouble In the House of Google by jwhite on Monday January 03, 2011 @08:45PM (#2065500)
iPad I think... see OP's response to another reply.
Re:Trouble In the House of Google by cdavid on Monday January 03, 2011 @09:27AM (#2063194)
Email spam has no monetary value for google, whereas web spam may (I don't know whether they benefit from ads revenue from link farm and co, but that seems at least plausible).

I wonder if the search quality issue is not also linked to a vast increase of the size of internet - searchs are the same throughout the years (couple of keywords), for more and more websites. Maybe vertical search helps there just because they reduce the amount of data to look for ?

As noticed by some other people, I have used google all my internet life (since 2000 or so), and google has never ever be useful to look for things like hotel, off the shelve software information, etc... I did notice a decrease of quality in technical searchs like API, etc... (where google has always been effective up to now), I wonder if something changed the last few months.

Re:Trouble In the House of Google by dhimes on Monday January 03, 2011 @08:08AM (#2063012)
I find far product searches to be far more efficient on Amazon. In fact, I think Amazon is the real threat to Google. I'll try to browse Google (but it's harder to do now for the reasons mentioned), but when I am ready to buy, I always check Amazon.

OT: did you type your commment on an ipad?

Re:Trouble In the House of Google by zzygan on Monday January 03, 2011 @08:03PM (#2065372)
Google is indeed getting worse for product searches. However Amazon is not the worldwide answer for product comparison since for the vast majority of their non book products they refuse to ship anything outside of the US, thus making pricing pretty much useless. Google definately has more usefulness worldwide.
Re:Trouble In the House of Google by dhimes on Tuesday January 04, 2011 @07:07AM (#2066656)
I'm glad you brought up this interesting point. I didn't know that their global presence was lacking
Re:Trouble In the House of Google by cletus on Monday January 03, 2011 @08:13AM (#2063022)
Amazon is an example of a niche vertical. A pretty big one mind you but nonetheless they specialize in the products they sell.

I can't seem to remember Amazon coming up on product searches I've done. I have to wonder:

- Is this because I went straight to Amazon?

- Am I misremembering?

- Is there something wrong with Amazon's SEO?

- Is Amazon restricting the their content being indexed (I can't imagine this is true)?

- Is Google actually failing here?

OT: Yeah my typo rate on my iPad is pretty terrible. I'm too used to touch typing on a keyboard. Still, I find myself reading the Wen on my iPad instead of a laptop/desktop these days so I need to put up with the typos (and get better with any luck), not comment or move to a computer to comment (not going to happen).

Re:Trouble In the House of Google by jonknee on Monday January 03, 2011 @09:48AM (#2063253)
They show up for in SERPs all the time.

http://www.google.com/search?sourceid=chrome&ie=UTF-8...

http://www.google.com/search?hl=en&safe=off&q=+Asus+...

http://www.google.com/search?sourceid=chrome&ie=UTF-8...

Etc etc. I see them all the time in Google, despite also searching Amazon directly quite a bit.

Re:Trouble In the House of Google by dhimes on Monday January 03, 2011 @10:28AM (#2063358)
I tend to go straight to Amazon when I'm price shopping now. If it's an "Amazon product" (books, electronics, etc) they usually have a pretty good price, and if they don't they give me the comparison. The big difference is: With Google, I'll research and occasionally get my wallet after finding something. With Amazon, get my wallet before I sit down.

As someone who pays for advertising this is a distinction I pay attention to, especially if it generalizes to more people than just me: people going to Amazon are ready to buy and not just doing research.

If Amazon can make that stick I think they pose a significant threat to Google. Why pay as much for research clicks when you can get clicks that are closer to a sale?

Re:Trouble In the House of Google by jonknee on Monday January 03, 2011 @10:55AM (#2063466)
Agreed. Amazon reviews are also my first product research tool.
Re:Trouble In the House of Google by joe_the_user on Monday January 03, 2011 @04:41PM (#2064791)
This is really just a rehash of other posts from the last month (linked in article). This post basically complains about two things: the finer points of SEO and content farms.

Yes indeed, on the money.

Further, the attitude of these various whiners is pathetic.

The war on spam-in-a-general-sense is not never going to be won or lost. The war for attention simply, inherently, has too many dimensions to be winnable or losable. As long as there's benefit for merely getting people attention, there will be people who successfully put out low-content, attention-getting information in one form or another. Human beings themselves are often often satisfied with less-than-stellar-content, how then could a given algorithm be expected to filter for stellar content?

Google and other filtering-entity's ability to provide us with stuff we're satisfied with will fluctuate. They won't ever be done. The "good" providers won't ever be fully satisfied and the "bad" provider won't ever completely go away. Any "new Google" will be "the same as the old Google".

Let's move on.

Re:Trouble In the House of Google by ZachPruckowski on Monday January 03, 2011 @09:49AM (#2063257)
"<camera make and model> review"

Which is fine if you already have narrowed down your camera options to a top-5 or so, but the issue is getting to that stage.