ayy88

I think the strongest use case for interfacing R with JS (be it through plot.ly, Shiny, rCharts etc) is when there is more serious statistical modelling going on (i.e. plotting things like credible intervals or model coefficients). But often you find that you add a 'statistical layer' after you already have an existing plot (i.e. adding a confidence band to a line plot). Maybe it makes sense to have one tool that handles all these cases.

It's not that they don't put in 'language like Bayesian', it's a different method. Yes, it is an improvement on the t-test straw-man they mention, but it's less flexible and powerful than Bayesian methods. Once you have a posterior, you can ask different questions that their p-values/confidence intervals don't address. For example, probability of an x% increase in conversion rate, or the risk associated with choosing an alternative. Not too mention multi-armed bandits, which not only are expected to arrive at an answer faster, but also maximize conversions along the way.

Another neat approach to dealing with multiple comparisons is Bayesian hierarchical models. Instead of correcting significance thresholds after the fact, all comparisons are represented as parameters in the model from the start. You would still need to 'preregister' segments as you suggest. See Gelman's paper, http://www.stat.columbia.edu/~gelman/research/unpublished/mu...