News for hackers. Stuff that matters.
Re:How to teach sensible elementary statistics to lower-divi... by swiley on Monday November 04, 2019 @06:35AM (#21440690)
Having an argument for why rather than just being handed a formula helps me. I don’t see why people think this would make it harder.
Re:How to teach sensible elementary statistics to lower-divi... by chongli on Monday November 04, 2019 @09:02AM (#21441661)
Because the “why” of statistics has many turtles before you get to the bottom. Measure theory, topology, real analysis, abstract algebra. You need to learn a lot of math before you get a complete picture of the theoretical underpinnings of modern probability theory, which forms the foundation of all of statistics.

Most people just want to calculate a P value or a 95% confidence interval for the mean of whatever they’re researching. They’re not interested in how it all works.

Re:How to teach sensible elementary statistics to lower-divi... by adrianN on Monday November 04, 2019 @09:24AM (#21441862)
You need a bit of analysis, but I'm pretty sure that you can understand p-values for normal distributions of a single variable without abstract algebra or topology. Understanding simple cases from first-ish principles makes it much easier to swallow handwavy explanations for complicated stuff.
Re:How to teach sensible elementary statistics to lower-divi... by soVeryTired on Monday November 04, 2019 @10:47AM (#21442681)
You can't understand the why of the normal distribution without Fourier analysis though - which is pretty heavy going for anyone who's not hardcore science or engineering.
Re:How to teach sensible elementary statistics to lower-divi... by adrianN on Monday November 04, 2019 @12:41PM (#21444088)
Can't you introduce the normal distribution as the limit of the binomial distribution? I think you can prove the central limit theorem without using terribly advanced math.
Re:How to teach sensible elementary statistics to lower-divi... by chongli on Monday November 04, 2019 @05:34PM (#21447358)
That's only the most limited version of the theorem which has since been renamed the de Moivre-Laplace theorem [1]. The rabbit hole goes much deeper when you talk about the most general form which works for any set of independent and identically distributed random variables, not just binomial random variables.

[1] https://en.wikipedia.org/wiki/De_Moivre–Laplace_theorem

Re:How to teach sensible elementary statistics to lower-divi... by adrianN on Monday November 04, 2019 @11:52PM (#21449718)
Sure, but do you need the most general form to build some intuition about p-values? I don't think so.
Re:How to teach sensible elementary statistics to lower-divi... by chongli on Tuesday November 05, 2019 @08:43AM (#21452257)
That’s moving the goalposts. The original claim was about getting a complete picture. A full understanding of the “why” of statistics going all the way to the bottom.
Re:How to teach sensible elementary statistics to lower-divi... by swiley on Tuesday November 05, 2019 @05:50PM (#21457641)
Fourier analysis is not that bad. Most kids in the calculus class at the university I went to had the basics of that explained to them.
Re:How to teach sensible elementary statistics to lower-divi... by madhadron on Monday November 04, 2019 @03:31PM (#21446156)
I'll bite. Why do you need Fourier analysis to understand the normal distribution?
Re:How to teach sensible elementary statistics to lower-divi... by soVeryTired on Tuesday November 05, 2019 @08:44AM (#21452263)
The only proof of the central limit theorem I know of relies on it. Iterated convolutions tend toward a Gaussian. Essentially you take the Fourier transform of any nice probability distribution, do a Taylor expansion, and Bob's your uncle.

Another child poster pointed out that in special cases (like the limiting case of the binomial distribution) you can get away with other arguments. And you can certainly 'prove' it via simulation. So maybe you don't need the full generality of Fourier analysis in the end.

Re:How to teach sensible elementary statistics to lower-divi... by madhadron on Tuesday November 05, 2019 @10:58AM (#21453492)
Iterated convolution was the proof I learned, too. Probably a semantic difference that I don't equate proof with understanding.
Re:How to teach sensible elementary statistics to lower-divi... by chongli on Tuesday November 05, 2019 @11:38AM (#21453955)
Probably a semantic difference that I don't equate proof with understanding.

I would call that a colloquial definition of understanding. Very different from mathematical understanding.

If you ask a mathematician what you’d need to know to understand Fermat’s last theorem, he won’t say “high school pre-algebra.” That’s only enough for you to understand the basic statement of the theorem. It doesn’t get you to the why. To understand the problem involves a deep dive into both algebraic number theory and analytic number theory.

Re:How to teach sensible elementary statistics to lower-divi... by madhadron on Tuesday November 05, 2019 @06:14PM (#21457861)
On the contrary, I use "understand" quite precisely. It's interesting to study what practitioners of various disciplines mean by "I understand."

For example, most mathematicians would say they understand large numbers of theorems that they cannot prove off the top of their heads. On the other hand they probably have what Borovik in "Mathematics under the microscope" called a "recovery procedure." That is, a set of constraints or path that reproduces the result. They know they can reconstruct the proof if they need to from the various gambits and skills they keep polished.

Also, the proof via the normal distribution being an attractive fixpoint of convolution is fine, but it only works on a particular subset of functions. We know the theorem applies beyond that subset, and there's a cottage industry of extending it in bits and pieces and calculating better convergence bounds. There is no proof available today that says central limit theorem applies iff conditions x, y, and z. So in this case the proof really can't be said to be understanding.

Now, that's well and good for probability alone, which is a field of mathematics. Statistics isn't a subfield of math, or is a subfield of math the way physics is. For a statistician, understanding the central limit theorem is much more about knowing what kind of observations it is reasonable to expect it to approximately apply to, what kind of tests rely on it and which don't, how to check if it applies in a rigorous way, what kind of visualizations and exploratory data analysis is enabled if it does, how the normal distribution and convergence to it fits into a whole family of distributions and features thereof...

Re:How to teach sensible elementary statistics to lower-divi... by mikorym on Monday November 04, 2019 @07:35AM (#21441024)
This dichotomy seems almost insurmountable today.

1) Mathematics majors should be taught how to prove. 2) Engineering/Applied Math majors should be taught to use.

Fortunately or unfortunately they take the same classes. Personally, I think the obvious solution would have been to just take whichever classes you want to and to get "a degree" once you fill some sort of criterion. But the CAs and actuarial scientists, uhm basically everyone, don't want that human resources headache to actually read through someone's CV patiently and ask a few patient questions.

Re:How to teach sensible elementary statistics to lower-divi... by commandlinefan on Monday November 04, 2019 @10:28AM (#21442487)
Well, why not both, really? We’re talking about four years of education - is it really that hard to learn proofs and applications with four years to spend looking at it?
Re:How to teach sensible elementary statistics to lower-divi... by mikorym on Tuesday November 05, 2019 @12:40AM (#21449949)
I don't think the current system is abysmal or anything like that—I don't think I am that vociferous or opinionated—but what I do think is that in your first and second year it is likely that you don't know yet whether you want to go the "applied" route or the "abstract" route.

So you might drop out of either of the two by year two and so you don't have the full four years. [1]

[1] In South Africe, a bachelor's degree is three years. I don't know whether this is a good or bad idea, but that is how it works. Your fourth year is called "honours" and is a separate degree.