Re:How Netflix uses Druid for realtime insights
by Twirrim on Monday March 16, 2020 @11:46AM
(#22593654)
> How do you account for the possibility that the update only performs badly because it’s different than what users are used to, but would actually be an improvement in the long run?
That's assuming that the update contains a UI/UX change. A lot of updates that will roll out won't include that, they'll be fixing or optimising things.
Re:How Netflix uses Druid for realtime insights
by catnzhat on Monday March 16, 2020 @02:00AM
(#22590408)
You acknowledge that fact and if you're making a severe change you either run a longer test or you break up the large change into manageable chunks (Google Maps is a great example of this).
For something that is effectively binary and can't be changed incrementally the only way is to extend the evaluation window or make a call with limited information.
Re:How Netflix uses Druid for realtime insights
by 2rsf on Monday March 16, 2020 @08:46AM
(#22592194)
regressions due to "users are used to" usually goes away after a while, you'll see a temporary dip in the results.
If you are convinced it's better for the long tun then what's the point in measuring ?
How do you account for the possibility that the update only performs badly because it’s different than what users are used to, but would actually be an improvement in the long run?