I know how wonderful it can be to feel that you have realized The Truth, but the frequentist school of statistics does not remain in existence just because statisticians are too stupid to recognize the obvious superiority of Bayesian methods.
I mean, many frequentist methods directly contradict Cox theorems! That should convince anyone they belong to History, not Science, doesn't it?
So. Who in her right mind would reject the three assumptions behind Cox's theorem, and for what reason? Assuming we don't reject those axioms, why should we not reject Frequentism at once?
I think a lot of people improperly understand the Frequentist/Bayesian divide; both look at likelihood functions but the Frequentists assert, as a matter of experimental interpretation, that the likelihood function should depend only on the observed data while the Bayesians are willing to admit "out-of-experiment" modifications to the likelihoods implied by the data (i.e. the "prior").
Frequentist arguments correctly point out that the prior is, from the isolated standpoint of the experimental observations, purely subjective and non-falsifiable. Bayesian arguments correctly point out that experiments are designed by humans in context, and that the context may imply a prior distribution that is not evident within the observations themselves. Neither is incorrect, but they answer different questions. Which question is the right one to answer is context-dependent.
As a concrete example, consider an A/B test where A and B perform very similarly. Merely seeing A's performance against B does not carry the contextual information that A has consecutively beaten one hundred previous non-B competitors that sort-of-look-like B. The Frequentist will correctly argue that, if you don't explicitly model the A/B test generation process (as opposed to just the A/B test itself), then all the data can tell you based purely on the laws of probability is that A and B are likely to be pretty similar. The trouble faced by the frequentist is that it's pretty easy to model an individual A/B test for, say, binary outcomes, but it's very hard to explicitly model the generation of A/B test competitors. So the Frequentist says "all I can tell you and still be strictly objective is that A and B look similar".
The Bayesian, rather than explicitly modeling the generation of tests as the Frequentist would like, wraps up that contextual knowledge into a subjective prior distribution that heavily weights A over B simply because A has won so many times before. The Bayesian's advantage is that the previously-blocking part of the probability modeling problem is punted into a subjective, experimenter-designed, input. This is a tradeoff, not an increase of correctness! It will "do the right thing" assuming the new B looks a lot like the old Bs, but will do the wrong thing if the new B is dramatically different and maybe looks a lot more like A with small tweaks.
Bayesianism is not a universal solution to probability modeling problems, and Frequentism is not an obsolete unhip method.
But if I finish, I'll probably have learned a lot about when to use each. Hopefully because I figured it out correctly, and not because I said something egregiously wrong and got corrected by someone who knew better!
> We will follow Evan's lead and use a frequentist approach.
I though: "how about using the correct approach instead?", followed by a helpful comment about how you are a Bad Person¹. Hem.
Then I noticed that the way you talk about p-value doesn't sound very frequenty. And then your comment. I eagerly await for the rest of the sequence –err, series².
[1]: https://xkcd.com/386/
[2]: Come to think of it, this does looks like LessWrong material to me.