I don't airBnB, but a friend who does was recently berated by a host after leaving a 4-star review; that host claimed (don't know how true this is) that many hosts will refuse to rent to you again if you leave anything but a 5-star review.

That and other things I've seen and heard about airBnB reviews suggest to me that there a structural problems that are likely to make it problematic for them to become honest and useful in general, even on top of the usual problems in numeric/star rated reviews that come from cultural differences in how people rate when given such a scale even when the actual substantive opinion is the same.

After my stay, I reported it to Airbnb. I didn't want to leave a review because I didn't want to deal with the fallout if Airbnb banned the host and they knew I was responsible. Airbnb ignored the report, the listing is still active, and none of the reviews mention the problem. I wonder how many other guests reported the problem to Airbnb without leaving a review as well.

Until airbnb starts cleaning up their mess I think people will eventually realize that this is just another scam in the long run.

At the same time, the apartment front desk staff and doorman were completely aware of my staying there temporarily. I showed up, asked for the keys, the front desk had an envelope, and there were no questions. They must have someone doing that every few weeks, and the signs are someone else's business.

Most of the inhabitants are people with 9-5 jobs who wish to enjoy a quiet evening after a long day at work. Having tourists next door doesn't help.

People coming home drunk late at night, kids playing soccer in the hallway, Scandinavian women walking around topless and in thong at the pool* - Airbnb guests treat the condo building like a resort, which it isn't.

The walls and doors are thinner, there's no carpet in the hallway and there are less elevators than in a hotel.

Here's a message to everyone considering renting an apartment on Airbnb: GET A GODDAMN HOTEL ROOM!

Thank you SV for "disrupting" my quiet evening [/endrant]

* I know some might actually enjoy this, but Thailand is a conservative Buddhist country. Respect that please.

So you get a window of a few weeks in which you can write a review. If you leave a review, then you get to see the review about yourself straight away. But if you don't review, then you have to wait until your own window has closed. After that, you're able to see the review about yourself regardless of whether you left a review yourself – because you are no longer able to leave a retaliatory review.

Airbnb aren't very clear about this at all, and that certainly pushes people to leave reviews, but it's a side effect that Airbnb play up, not the actual designed purpose of the review system.

It's amazing to go read a novel rated 3 stars (because it operates outside this loop) and find it transcendently good.

The problem with star based rating systems is that they're INCREDIBLY subjective. I oftentimes read yelp reviews that are 3 star reviews where the person raves about the restaurant and then complains about a minor issue. I have a similar issue with app store ratings where I've had a rash of people rate one of my apps 1 star with the review text reading "I just downloaded this, I'll raise my review to 5 stars if love this app".

It's how the "official" reviews from Marriott/Hilton/etc all work

2. Ask each person a different objective question, intentionally filling in the gaps.

3. After a few people, you have an objective, relatively thorough view of the house, but you've not burdened any of your users with more than one question per stay.

It isn't just online, and it isn't just a technology problem. I recently bought a car and the manufacturer sent me a survey about the process. The salesman made sure, numerous times, to let me know that I'd be getting said survey and that anything less than a 10 meant he failed and that there would be negative consequences for him. Since I was mostly happy with the process, I gave him all 10s, even though one or two areas I would have rated a little lower. He got his good review and it didn't really bother me, but the car company lost out on some potentially useful feedback.

Some of the problems - like different people having a tendency to rate on different scales, ratings coming from different factors, etc - seem pretty amenable to a ML approach to make reviews a lot more useful, though. I wonder if anyone outside of a handful of companies (Amazon, Yelp, AirBnB, Google, etc) has enough data to make it useful, though?

I know I personally gave plenty of three star reviews on Lyft before I learned better, meaning them as "no complaints". Who knows what happened to those drivers as a result. Why even invite the misunderstanding?

In France for instance, it's very rare for a student to go above 15/20 or so at the college level (I graduated in the top 10% of my class with a 12/20 average; the top ranked student, whom I was friends with, had a 14/20 average).

But when I graded homework as a graduate student for an american university, the professor told me I was too harsh, and after adjusting my criteria to match what she wanted, I found myself giving many 100/100.

When it comes to ratings used by others to evaluate, no system is more customer and vendor-hostile than one where everything that meets a minimum threshold for adequacy should be rated as perfect.

I guess, at least a small step would be to make it so values less than the highest rating don't count against you so negatively - it should be ok to be a 3/5 or 7/10.

1. normalize reviews to make a 4 star from someone who always gives 5 stars worth less than a 3 star from someone who always gives 2 stars; and

2. make reviews secret?

Perception of books and concepts covered within are entirely subjective.

However, when it comes to service or product quality, you can be almost entirely objective in its value judgment.

Hmm. I'm willing to accept that the average reader is a more sophisticated and thoughtful consumer than for some other product categories. And there may be less overt tribalism and fanboyism in book reviews.

But, in my experience, there's still a lot of people who will almost unthinkingly give 5 stars to a favorite author or genre. (And I know some of those people who are pretty uncritical about SF books or whatever.)

Again, novels and products are entirely different in that you're hopefully shopping for a product by its specs. You can't 'research' novels beforehand (unless you count as reading other novels by the same author as research) and they don't have any specs to go by.

No, all assessments of value are subjective (and for most services involve subjective weighting of multiple different subjective quality factors.) And the mapping from an internal value judgement to a star rating is something which is a separate subjective evaluation, and one on which research has shown substantial cultural variation along ethnic and other lines.

The rating may be inaccurate because you can only measure in full stars, but it'll still be objective and the inaccuracy is the fault of the rating system, not me.

Objective (adjective):

> (of a person or their judgment) not influenced by personal feelings or opinions in considering and representing facts.

On the popular restaurant review site Tabelog, a 3.6 (out of 5) is a very good ranking; there's more 3-star Michelin restaurants in Tokyo than restaurants above 4.5.

https://tabelog.com/en

If you do it secretly like shadowbanning and introduce it softly by slowly blending between conventional score and normalized, it would drive a whole generation of SEO/social/share-economy experts delightfully mad. Think of all the free publicity!

Jokes aside, it might help to regard low ratings as what they are: a way to flag bad behavior, dead serious stuff. We should stop pretending that it is a game, then maybe solutions will be more likely to appear.

There are particular instances (like in this case) where there are a lot of incentives for just about everyone to give 5 stars, 10 stars, or a positive and call it a day.

But, in general, there are issues with self-selection, bias toward the high end of the scale, ratings meaning different things for different types and classes of product, etc.

Airbnb pushes hosts really, really hard to achieve >80% 5-star reviews. If <80% then the listing gets pushed down the rankings and you lose SuperHost status.

Some guests never give 5-star reviews for whatever reason. They really, seriously hurt the host. If your average is >4.0 (and it should be over 4.5) then any review less than 5 stars will lower your rating.

I worked in a restaurant and was trying hard to get our TripAdvisor ranking higher. One quiet night a guy comes in so we chat and give him some great service, special dishes etc. At the end of the meal he said it was excellent and had left us a 4 star review. I said "Dude, our average is 4.8 stars, you just made us look worse."

In the end we got to #2 in our city. The #1 was a cafe that was only open 7 am to 4 pm. All you need for 5 star review in a cafe is a hot coffee, so we settled for #2.

https://qz.com/410264/you-should-never-trust-an-airbnb-revie...

This is an interesting problem, and yes, needs to be accommodated some how. I used to manage a SW product and service offering that had customers around the planet. We sent out quarterly report cards in order to ensure we were meeting customers needs. It was As and Bs exclusively except for one country that was Ds and Fs for the same level of quality and service.

Some type of calibration or break down is needed.

I travel so much that consistency is important and that is what hotels offer. I see AirBnB as an inferior option for hotels, not a way to meet new friends.

I think the idea that AirBnB is going to kill the hotel industry is way overblown - professional travelers (not the startup crowd largely) like consistency above all else.

I'm specifically only hosting in destination places because when people are on vacation they are inherently happy. Not saying you are unrealistic or unhappy, but it's a different mindset and you have different needs.

So yes I agree with you - it takes a lot more effort for you to have a good, consistent experience as a business traveler because you have to spend time wading through listings.

But there's no reason why AirBnB can't solve that by creating a standard for what business travelers want - perhaps a business superhost designation. I don't think it's overblown for them to put a hurting on the hotel industry. All it takes is for them to take a decent chunk out of RevPAR and many will be done. Hotels typically have a 5 year cycle and they've been having record years. So it's definitely overblown as of now. But killing the investments in hotels is more about what happens when you steal share during the bad times. And also - what happens when there's a good AirBnB rewards program to lure business travelers and they can make it easier for them to book and be consistent as you mentioned?

The hotel industry will remain, but there will be a lot less inventory and operators may decide they don't need to fly the big brands or can't afford the fees.

Perfect, and I'll bet you have a great place. See my previous statement.

But there's no reason why AirBnB can't solve that by creating a standard for what business travelers want - perhaps a business superhost designation.

Supply and demand. To curate that hard, you'd have to have a ton of supply of business focused programs. Might as well just open a hotel at that point.

I just looked and they have a section for business travelers and for wading through listings [1]. They already have Business Travel Ready Listings [2] which was what I was suggesting. Have you used that or did you go through the front door for your bookings? I didn't know it existed and

Separately - there are advantages of an AirBnB that are more friendly to biz travelers already, like being able to have more flexibility for check-in, free high speed wi-fi, kitchens in almost all listings, better work spaces, etc. So it's a matter of ironing out a few things to make booking easier, adding a good rewards program, and ramping up marketing to compete and steal market share.

[1] https://www.airbnb.com/business-travel

[2] https://www.airbnb.com/business-travel-ready

But I've gotten guests who nit pick everything and then ask for a refund. I had one guest ask for a refund because there was snow in Tahoe, and they stayed for the full time anyway. Another guest wanted a refund because my shower curtains were expanding tension rods and not permanently mounted.

Generally it still makes sense for me, but running an Airbnb well takes a lot of effort. I have a great support team, and I still struggle at times.

(1) https://www.airbnb.com/rooms/12247150

I had guests complain once that there were a few things in one of the bedroom closets (still tons of storage for their stuff), some things in a small pile in the garage and some food in the cupboards (usually have ketchup, hot chocolate, popcorn, etc).

I had my head of turnover go over and chat with them in person and removed all the stuff just to be nice. She told them that some of the things were destined for Goodwill and then the lady who was complaining started picking through it and her tone completely changed! They also showed up with a small towable U-haul full of stuff and a crazy amount of canned food for a short trip. This was a big trip for them.

So sometimes people are just unrealistic. Complaining about some things in the bedroom closets - you're staying at my house! And the stuff in the pantry is for you to use!

However, some people are spending a lot more of their money proportionately and they have different expectations because of this. They want everything to be perfect. Maybe they're being guilted by their spouse or family for spending that amount. They'll complain about really trivial things.

I don't really like instant book because anyone can book. You have to do legal discrimination - AirBnB doesn't do much to help and hosts are often hesitant to leave bad reviews on renters for some reason.

We had a group of 30+ sorority girls message us wanting to stay. Legally we're approved in our area for 16; if you're doing it right and you pay your TOT taxes and are registered - you get approved based on square footage. So of course we declined their request. If they instant booked, I'm not going to be able to kick them out, I'll get more wear and tear, and I could be fined by the city or get my license revoked. Not worth it.

I see why Airbnb is pushing instant book but not participating helps me minimize risk.

Sure, you can follow up w/ airbnb, but since they outsourced their staff to a Clearwater, FL and Ortigas, Metro Manila call center their SLA dropped from 24 hours to 2 weeks.

I do think people have unrealistic expectations, but my house is pretty damn nice. My cleaners are amazing. I'm pretty happy to stay there myself.

I don't know of a better value for the consumer. One of my family members purchased a place up there about a year ago and she's starting to do better but it's been a grind. She told me that her renters are often unrealistic. I find it odd given the amount of money up there and the low nightly price but it's probably just the supply and the fact that a lower price means people aren't forced to really evaluate the value as they should. That and the fact that primary housing prices are so damn high, that when people get away they don't/can't spend a lot.

My wife is a superhost on AirBnB and it's not challenging - you just have to be responsive. The challenging parts are when you have difficult guests but I also pay my head of turnover to handle those issues a la carte if needed locally, otherwise my wife handles remotely.

It would be very hard to turnover my place without using a company that has staff due to it's size. Laundry takes about 5 hours alone. You have to have multiple sets of clean laundry if you're doing it right. I have a license and pay 10% TOT tax. Other operations aren't registered, don't pay hotel like taxes and don't replace with clean sheets. I did 80k in revenue on one property alone last year and I'm buying another soon. There's a lot of opportunity but you have to buy it right, meet guest expectations, and be compliant.

Update: I'm on phone with AirBNB support about this right now. Frankly their phone support is pretty poor, eventhough I'm a "super host." They can't seem to answer basic questions about how the wifi network and password is provided to guests

The biggest problem to me is poor expectation setting. There are hosts that run BnB quality services, ones that run a mock-hotel like you described, and ones that just rent out a spare room in their apartment.

As a guest, it'd be great to know what I'm walking into. As a host, having guests that understand they are staying in my home and NOT a hotel would be great.

My guess is, dividing up the listings into service tiers would lead to legal issues (like you saw last year in NYC), but it would surely help with the guest-host relationship.

Usually, the description and previous reviews gives you all you need. When it doesn't, I simply ask the host. It's never been a problem.

Very often. I'm counting motels, because I'm comparing the same price points.

My experience with AirBnB has been both positive and negative. For the negatives, I never felt I got less than what I paid, and motels would have been more expensive. For the positive, it has always been cheaper than a motel.

I got to stay in downtown San Diego, renting a whole apartment, for a really good deal. No hotel would have been that cheap.

It was a nice apartment, not a super cheap-o place. And free parking.

I will say I don't demand too much. I'm not the type of person who wants fancy service. Just make it clean. And offer basic stuff (tissues, soap, etc).

Once people make careers out of this, it really ruins the spirit of the service and then it's not much more than someone getting around paying hotel taxes.

This. Something I hate is when people think that 5 stars have to be given "by default". For me, an uneventful stay in a clean place should mean 3 stars. If the place is specially nice or the owner was really kind, 4 stars. And 5 stars should only be for my _favourite_ places.

I stopped doing airbnb when, for several times, I booked 4.5 star places and they turned out to be honestly crappy.

Typically the cheapest places are also the ones where people don't have high expectations so they will rate 5 stars but leave a comment describing the quirks of the place

I've also Airbnb'd over several continents. I've had one sub-par experience out of many stays.

I think with Airbnb it's also "you get what you pay for"...

Uber also expanded everywhere and it only increased its loses. It could be that their business model is not based on subsidies borne by investors as in case of Uber.

As a user, Id rather they err on the side of amateurish. Airbnb can't be the whole market. It works best for recreational travel. Even people who buy or rent to Airbnb generally do it on a small scale.

I think they don't have that requirement anymore because most properties I see locally have smartphone camera quality pictures with improper lighting.

I get that it's cheaper, but after a certain age, I just want to be comfortable on a vacation, not have to think "will it be terrible?"

The only exception is if the property offers something hotels don't (like a lodge in the forest or a rooftop apartment overlooking the Dome of the Rock in Jerusalem)

An interesting question is: is it better to optimize for more inventory or better quality?