News for hackers. Stuff that matters.
Re:Metrics-driven product development is hard by thewarrior on Monday December 27, 2021 @05:23PM (#29707544)
I recently had a revelation that merely using a product can often reveal the organizational structure and the cultural incentives that went into making it.

A company that is famous for being metrics driven and having teams that optimise for their own metrics is Facebook. You can tell that the product has "seams". Each page and widget feels like it was individually optimized and then stitched together. Overall it somehow feels off.

Apple puts design on top and has a kind of dictatorial veto over all aspects of the product. Hence products made by Apple feel seamless and cohesive. They are also more polished because when you solely rely on metrics you make it as good as it needs to be and then stop.

Re:Metrics-driven product development is hard by karaterobot on Monday December 27, 2021 @05:58PM (#29707959)
Sounds like you've independently discovered Conway's Law, one of the most terrible, true things you can say about software organizations:

https://en.wikipedia.org/wiki/Conway%27s_law

Re:Metrics-driven product development is hard by andrewjf on Monday December 27, 2021 @10:32PM (#29710025)
Apple relies on intuition and experience and uses data to support decisions, or not (and data be regularly overruled). Rarely does pure data drive decisions. People coming from other companies usually fail if they can’t adapt to a “data second” mentality.

I feel like Apple’s method is much better than blindly trying to interpret data and let that drive you over the cliff. Data isn’t a replacement for understanding of your product and users.

Re:Metrics-driven product development is hard by kqr on Tuesday December 28, 2021 @01:54AM (#29711197)
Are you aware of the work by Meehl on actuarial vs. clinical judgment? One of the conclusions of that line of work is that fairly dumb, mechanical data-driven rules ("actuarial judgment") consistently outperform expert opinion ("clinical judgment") even on what we think of as fairly complex topics.

The more surprising conclusion is that the dumb, data-driven rules outperform experts even when the experts get access to the same data and the decision made by the data-driven rule. Why? Humans have a tendency to think they're better than dumb rules and override these dumb rules in ways that make their decision ultimately worse.

Re:Metrics-driven product development is hard by andrewjf on Tuesday December 28, 2021 @11:54AM (#29715459)
I haven’t, but I don’t think it’s a stretch to assume that conclusion doesn’t apply in creative fields.
Re:Metrics-driven product development is hard by kqr on Wednesday December 29, 2021 @10:02AM (#29725372)
The funny thing is that people say that about every field, yet one by one, Meehl and the people doing work along the same lines consistently disprove them.

The result has remained remarkably consistent across many fields where people were sure it wouldn't apply. At this point it's stable enough that I think the conclusion is a good base assumption, and it would take serious evidence to show that it does not apply in any given field.

Re:Metrics-driven product development is hard by thewarrior on Friday December 31, 2021 @01:47PM (#29751127)
What is a creative field ? Can you write a novel this way ?
Re:Metrics-driven product development is hard by thewarrior on Monday December 27, 2021 @10:54PM (#29710160)
That's interesting. I was actually just guessing that Apple is not data driven.

I would be grateful if you can share Apples general approach to product development.

Re:Metrics-driven product development is hard by andrewjf on Tuesday December 28, 2021 @12:33PM (#29715886)
The closest publicly available info is an HBR article about Apple’s organizational structure:

https://hbr.org/2020/11/how-apple-is-organized-for-innovatio...