This is a very important principle.

For a time, Target used Amazon's website code. Here is what they found. People leave reviews of books that they like. People only bother reviewing microwave machines when they didn't work.

This meant that Amazon's code didn't work out as well for Target as it did for Amazon...

The whole online review thing is thoroughly broken. I am not even talking about people leaving bad reviews for things like "the seller didn't ship it on time", etc. I am talking about the fact that most people that leave a review have nothing to compare the product to. For example, try shopping for a wireless router. You'll find lots of people either praising or bashing it but having no basis for comparison. Star ratings are completely screwed because lots of people give both one and five star ratings based on completely insane criteria. Long written reviews are naturally most informative, but you have to remember that they are also extremely subjective.

On the other hand, you can't trust expert reviews either. The experts spend too little time with each product to form an actual opinion. They often form their opinions based on some very shallow factors. A canonical example of this is mobile phone reviews by sites like Engadget or Gizmodo. They review so many phones that they can spend only a few days with each.

Edit: one of the worst things for this is reviews of hard drives. While there certainly are hard numbers to go off, there are also lots of opinions: "Well I've owned 6 Seagate drives over the past 4 years, and they all ended up dead. Seagate products are crap!" That does not mean that all Seagate drives are crap, maybe just laptop ones. Or maybe your controller is bad and you keep attaching good drives to it. Or maybe you keep sticking these drives into a case with poor ventilation. On top of that your sample size is too small.

Agreed that online reviews are broken. That's what drove me to co-found a company that takes a different approach to reviews, one which stresses category over individual products.

Start with the premise that the person you most want to give you advice is someone who is passionate about a topic or at least has done a lot research on it. If you ask them for advice, first they'll give you an overview, which hopefully includes what attributes you should be thinking about. Then they'll recommend a small number of models to consider, based on what you seem to be interested in. If you're doing this with other people around, perhaps a few other people will pipe in with suggestions.

So - for example - if you're looking for 3-4 player board games more interesting than Sorry and Monopoly - so interesting that they have a decent chance of getting your friends and/or family members more into gaming - I prepared a guide on our site that does just that:

http://obviously.com/619/gateway-games

(Note: site styling is incomplete - we're not quite ready to launch)

The essential idea is that if you narrow a topic down enough (i.e. 7200 RPM HDDs that are very quiet and can easily survive a several foot drop onto a hard floor), there's probably somebody out there who's taken a great interest in that narrow niche and can write about it very well.

On the other hand, it's hard to write about the too-broad hard drive category as a whole, and it's also difficult to talk about one hard drive with little context.

It's sometimes great pleasure to see how an original topic in HN starts delving into side topics that have nothing to do with the original. I thought it was a one-off thing but seems to happen quite a bit.

Nothing wrong with a good fork!

The original post is sometimes just kindling for other discussions. What can you really say about gmail being down that is that interesting? (That is a question....)

Would be interesting if we could flag a side conversation and they appear collapsed by default, with a button like "Off Topic. Click to Expand."

I think that one's pretty easy: Why is it down? Technical hypotheses, discussion of past history of outages, etc.

This is normal for any conversation.

They aren't great but they aren't useless either. A customer is basically rating, "Was I satisfied?". If everyone who bought this product was 100.0% satisfied, is that not useful information, even if there is another competing product that might be better?

The thing is, there is no way to ascertain the level of competence of the person writing the review. Is this someone really knowledgeable, or is it some dumb newbie? Sometimes it's easy to tell, but other times it's very hard.

Then there's also the issue of reviews with an "agenda" - self-explanatory, no need for me to say more.

One thing that seems to work is to see someone I know using a gadget, to actually see that gadget being used, and talk to that individual briefly about the gadget. The info I get this way seems quite a bit more trustworthy than online reviews. Of course, this could also be an artifact of the way we form opinions.

Another empirical observation: on Amazon, a significant amount of negative reviews is typically an indication of a real issue. But the absence of negative reviews doesn't say much. Positive reviews do not appear to be quite as relevant-information-rich, unless there's an extremely large amount of them.

This is why Google did the right thing tying their reviews to their social network. If I know the person doing the review, I can put it in context. There are people whose opinion on wireless routers I'd listen to, and people I wouldn't.

A lot of people got angry at the time, and publicly declared they wouldn't review any more. But that's not really a big loss. Their reviews might be the most qualified out there, but as long as they were anonymous I would be unable to put them in context, and they would be of very limited value to me.

This just leads to survivorship bias. Its a shitty hack. A good review systems needs to encourage honesty, not discretion (selective revelation). The substitution of selective revelation for "honesty" is not a more honest or more reliable system. This is a common mistake, and one that marketers like to promote. As it works to their advantage: to manipulate the selection and revelation process, it helps to have a system in place such as the one you describe.

I think while that is one benefit of their move, it was not primary purpose. Google is data. They had different users on different services, now they are trying for unified account (Which is probably going to be G+ with everything else attached to it). In essence that is to allow people "convenient" access to all Google services with a single login. However, really it is to tie real people to their services (Easier to sell shit to real people) and to chain them to using Google.

I know there are lots of other reasons, but I believe what I stated above to be main reasons that outline Google's long term strategic plan. Keep driving ad click through rates up by showing more and more custom ads to everyone.

I don't understand a word you are saying.

Aparently.

Are there offline reviews that are better? Do you trust the information of a salesperson better? I think there are definitely problems with a selection bias of who leaves reviews, but sites like Amazon have implemented a number of features to improve upon them like rating distributions (how many 5, 4, 3 ,2, and 1s), the vine program, reviewer rankings, verified purchases, large volumes etc.

Consumer Reports is actually pretty damn good for most things, especially in terms of reliability ratings. Good reviews is one of the things I'm happy to pay for, since the web doesn't seem to have spawned a good free alternative.

PSA: Some libraries have free access to Consumer Reports. Check with your local library to see if they have a membership.

For cooking stuff, Cook's Illustrated is the reference. Are there better recipes and cooking equipment reviews out there? Definitely. But it's generally of very high quality.

Ah, that convinced me to actually buy membership. Thanks for spreading the word.

Yeah, it's really a great publication. I really enjoy that they've taken an ideological stance to never take advertising so that they never have a conflict of interest, in stark contrast to almost every review site on the internet. When testing cars, they have plainclothes buyers to buy the cars, as well (partly so that they're not subject to keeping companies happy to keep the review samples flowing). They seem to also have pretty involved testing methodologies. All of this takes money, and I imagine that the rise of the internet hasn't been kind to their cashflow, so I'm happy to chip in to keep this sort of independent testing organization alive.

I trust some reviews of some products by some people. I differentiate. For example, for car reviews I read Edmunds reviews but don't trust all of their opinions. They don't try to do anything with car reliability which really sucks, but they can't. You don't know what's what until you've owned the car for 10 years, and they are reviewing new cars. I trust my mechanic friend much more here since he can actually look under the hood and tell me if the type of car has a transmission that's prone to go bad, or an engine that's going to burn oil. It's possible that he's pulling his opinions out of his own biases, but what he says usually makes internal sense so I feel better if he tells me something is good or bad.

I don't trust good restaurant reviews, and only trust bad ones if they cite specific issues, like using wrong ingredients, etc. I trust my friends and family when they say a place is good or bad much more because I know their tastes and I can ask followup questions.

For electronics, I pretty much don't trust anyone with anything unless something is backed by the manufacturer. I only trust Apple products because of Apple Care. Everyone has had a bad experience with a laptop, a phone, a tablet, a mouse, a USB cable, etc. It's not worth listening to most times since it's usually a horror story from 10 years ago about a company that who's had their entire manufacturing line re-tooled since.

On Amazon for a generic product, the biggest indication to me is if there are lots of reviews. That means lots of people own the product. If most of the reviews are one star, it's a crappy product. If not, it could be good or bad and I personally find very little correlation between the way I perceive a product and its Amazon reviews.

An impartial third party with a high level of experience and expertise in the area of review. Especially good if they can compare the product side-by-side to its competitors.

This can provide its own bias. For instance, your average film critic has seen so many versions of stupid action movie that they find those movies pretty tiresome.

My favorite reviews are currently from America's Test Kitchen. Their equipment reviews are especially insightful and well written to provide a thorough expert opinion on products. Wonderfully, they've found a business model that allows for their independence.

Agreed about online reviews being imperfect. However one way to counteract extreme subjectivity is context. If you know somebody in person, you'll be able to understand their online review a lot better than you would a complete stranger's. Having access to reviews of friends and family and people you choose to follow is why I built http://www.mybema.com

If you want to fix this with me drop me a note. I have an idea, a design and a viable business model.

"The whole online review thing is thoroughly broken."

"Star ratings are completely screwed"

"Long written reviews "...are also extremely subjective."

Totally agree. Here is my analysis and resolution: http://bit.ly/LRSdkJ or

If you want to fix this with me drop me a note. I have an idea, a design and a viable business model. :)

  > For a time, Target used Amazon's website code.

I remember getting the email saying that we no longer had to special case Target stuff in the retail platform. 'Twas a joyous day all around.

I think half the reason people upvote stories about big important sites like gmail being down is because it makes them feel better about any downtime their own sites have had.

I must be terribly naive or hopelessly optimistic. The immediate reason I thought of that would explain why stories like this are upvoted is for a matter of visibility, and in a distant second place, to head off questions like "Is it broken for you, too?" Judging by a cursory review of the comment here, I suspect that reasoning is fairly common.

It strikes me as awfully pessimistic to immediately assume someone's clicking their mouse thinking "Haha, Gmail is down! Take THAT Google! >upvote<" when the result is very real service disruption for a lot of people.

I'm sure it's a mix of reasons. Like I said, my guess is half. Also, it wan't an immediate assumption, I've thought about it for a long time. Lastly, I've not suggested people are glad gmail is down or that people are harmed. That hadn't occurred to me; but now that you mention it, you're right, it probably does happen occasionally.

Oh. My first thought was "Ouch, we're too dependent on Gmail", but sure :)

I think more that people are more likely to communicate when things change rather than when they don't.

"This is your pilot speaking. The engines are currently not on fire and are still attached to the plane. That is all."

> People are more likely to leave a review/comment/upvote if something is not working for them.

Also known as the 3/11 rule. On average people tell 3 others when they have had positive experience. And on average they tell 11 others when they have a negative experience. [proper citation appreciated]

Add the effect of internet, and multiply by Google effect. More recent posts are given higher relevance, and of course more frequent occurrences are given higher relevance.

News agencies and publishers have known this for ages - after all, there's no news like bad news.

isn't that obvious? it's because being down is the exception, it would be pointless commenting that gmail is up since that's the ordinary state.

I wonder if you could utilize this information. Perhaps negative reviews could get more weight if they're from people who have also left positive reviews.

True, everytime we change something on our product, only the people that have something to complain about, say something.

Every productguy/developer should know this. :)