News for hackers. Stuff that matters.
Re:The Problem with Feedback by TeMPOraL on Wednesday December 05, 2018 @03:09AM (#18606483)
Cybernetics didn't fail expectations. It works astonishingly well. But it seems people had some pretty weird and dumb expectations when lifting it from the domain of hardware and into more "fuzzy" real-world systems. Feedback loops are an incredibly powerful tool, but they get misused (often purposefully), and then the blame for that misuse somehow falls on the feedback system.

Consider:

1) As a user, won't be satisfied with a feedback system if the company on the other end is evil. Take Uber as an example. The tension between drivers and passengers isn't caused by the 5-star rating itself. It has another critical component - that Uber set the system to squeeze out as much value out of drivers as possible. Since they essentially optimize for bad driver treatment, drivers get defensive, and this screws up user experience. Hitting with 1-stars until morale improves isn't the solution; not exploiting people is. Similar thing happens in some customer service interactions, where rating is - again - not a signal on quality, but on whether or not I hate the other person enough to want them to lose their job.

2) A rating system won't work for you if you connect it to a money-printing machine. This one should have been obvious immediately, and definitely should be by now. This is why Amazon, eBay, Yelp, et al. have problems with reviews. As long as their presence or absence impacts immediate sales, they'll be gamed to the point of uselessness.

3) Some companies like to deploy ratings to skip doing the work, and then they cry foul when the results don't materialize. Again, Uber won't solve driver problems with star ratings; that would require actually meeting and evaluating people. I can hear the screams - "but that doesn't scale!". So what. I want to go to the Moon by lifting myself by my own bootstraps, but I can't, because that's not how physics works. I'm not going to cry that I can't get what I want without doing the work that's necessary to get it.

4) Feedback loops must be actually closed for them to work. I.e. not just collecting the data, but acting on it.

5) If you can't capture the entirety of your vision in directly measurable metrics (and you probably can't), then don't follow them blindly. In particular, be mindful of what you actually measure, 'lest you end up pissing off a lot of users and defending yourself with "data told me so". "You make what you measure" works both ways.

--

TL;DR: Blame the people/companies deploying feedback systems for being clueless or having malicious intent, not the feedback systems themselves for simply working the way they work.

There are no comments on this comment yet.