News for hackers. Stuff that matters.

A comment on Feature Flags: Theory vs. Reality. Parent comment

Re:Feature Flags: Theory vs. Reality by adamesque on Monday July 10, 2023 @02:58PM (#36671290)
It’s actually usually the paying customers asking via support to be added to the holdback, improved experience or no.

This is more true for larger flags that substantially change the experience and may not implement niche or edge-case functionality. Obviously you want to avoid these kinds of tests if possible but it’s not always possible.

[ Parent | on HN ]
Re:Feature Flags: Theory vs. Reality by esafak on Monday July 10, 2023 @03:13PM (#36671474)
Users should not be allowed to select their treatments; it defeats randomization, which is what allows causal inference.
[ Parent | on HN ]
Re:Feature Flags: Theory vs. Reality by mandelbrotwurst on Monday July 10, 2023 @06:50PM (#36674081)
Sure, they'll be more predictive that way, and simultaneously it's valuable to not piss off your customers.
[ Parent | on HN ]
Re:Feature Flags: Theory vs. Reality by esafak on Monday July 10, 2023 @07:33PM (#36674508)
In that case I would take them out of the experiment and impute the censored data.

https://en.wikipedia.org/wiki/Censoring_(statistics)

[ Parent | on HN ]
Re:Feature Flags: Theory vs. Reality by SketchySeaBeast on Monday July 10, 2023 @03:21PM (#36671582)
Ah, that makes way more sense.
[ Parent | on HN ]