I took a fantastic one-off class as an undergraduate called, IIRC, "Statistics for Physicists". The class taught us Bayes' Law, what tests were, what significance was, and how one might go about deriving and using simple tests. Everyone came out of the class with a very clear understanding of what statistics meant, and no one who took that class would take a statement like "this drug had a significant effect on such-and-such" without an appropriate grain of salt (specifically, how big was the effect). The class didn't include a giant grab-bag of tests.
One of my friends took a regular introductory stats class the following quarter. At one point, he asked me if I could help him do a such-and-such test (Student's t, maybe, but I don't remember which one). I asked him what he was trying to do with the test, and he didn't know.
I think it would be enormously valuable to teach most students what statistics means, what it can do, and what it can't do. I'm not sure how valuable it would be to teach everyone how to evaluate a big pile of magic statistical functions.
They should teach it from the perspective of what problems you are trying to solve, what your options are, and how those options work in practice.
The "in practice" part should be lots of simulations with something like a ChromeBook, so people could really see the ideas working. Create a (simulated) bucket with 70% red balls and 30% blue, then pull samples of various sizes and see how well the sample means match what you already know is the "truth" (70/30). You'll soon have everybody seeing that bigger samples aren't always better, but they tend to be, and they'll see how much better they get as a function of size. Make charts of the improvement. Talk about how good the estimate really NEEDS to be.
Then try some tests where the students DON'T know the "truth" about the population and have to use samples to estimate it. How far off do they think they are. How confident about various intervals, etc.
Then start explaining some of the math that lead to formulas that let you calculate these estimations, sample sizes, confidence intervals, etc.
Have estimation games: you get points for getting an estimate within some delta of the population mean, but you have to pay points to buy your samples. So, how big a sample does your team want to buy?
Then continue with other basic prob/stat ideas such as bayesian cancer tests (does this mean you have cancer or not?) and courtroom dramas (does this mean he's probably guilty or not?), commonly misunderstood situations (ex: Simpson's Paradox), how Gaussian estimators begin to fail when things aren't independent, and all sorts of other introductions to quantitative thinking about real life scenarios.
I was a very math oriented student at a high school with a heavy bias towards liberal arts ... So when it came to picking senior year electives, I was delighted to enroll in AP statistics.
The classes turned out to be tutorials on the built-in TI-83 stats functions, with step-by-step instructions on how to answer the various types of questions we would encounter on the test. I forgot everything and gained virtually no statistical intuition.
Thankfully I had a similar experience to you ("Probability and Statistics for Engineers") with a fantastic professor who researched computer networks. His lectures were sprinkled with tangents tying the current topic back to his own field / research, which to me made the biggest impact. Contextualizing a taught subject markedly improves the course experience.
Fortunately, I had a great professor in Calc III who more than made up for this.
And you need stats all the time. Every time someone says crime has never been worse, you need to know how to decide if you believe it. When someone says flying is safer than driving, you need to know how to compare. A lot of stats thinking is also not formulas, but qualitative things like "Am I comparing apples to oranges".
Calculus by contrast is pretty common sensical. Acceleration is the derivative of velocity, you find the local extremes by finding the zeros of the derivative, the integral of a net shaped field is the rim... that all makes sense with some pictures.
I also haven't come across it much in finance, apart from derivatives work, where funnily enough you can't do with high school calculus, but you need stochastic calculus. Which contains some quite statistics related concepts.
There are some basic calculus concepts that are useful. Area under a curve. Basic derivatives (for finding maxima/minima, etc.). But the truth is that I've rarely used calculus outside an academic setting and I use at least basic statistics/probability thinking all the time.
Though stats/prob are perhaps more 'useful' in non-academic life. Imparting the notion of what probability actually is and how Bayes' rule fits into it requires a very gifted teacher with a fundamental grasp of the real mathematics and on top of that the ability to communicate it and link it to previously learned material aka algebra/ pre-calc. You can make it visual, but it would seem to require defining the concepts of sets, sampling, distributions etc.
The issue imo is that teachers tend not to teach the intuitions of math, rather they focus on the mechanics of it. As such, most students turn into machines calculating things as opposed to viewing math as a way to reason.
That's why calc is central. I wouldn't be surprised if it's the same in the US.
This means most people have no idea what the math they learn is used for, because they never reach a level where they have to use it.
So they leave school with the idea that math is annoying bullshit that they don't understand and can't use.
I think it would be better to stream math by ability much earlier. Make everyday math compulsory for everyone - interest rates and loans, basic stats, currency rates, areas/volumes, simple trig, maybe some slightly more advanced finance, perhaps touch on some calc ideas.
Only teach calc and scientific math to those with the talent and interest for it. Give everyone else a taste so they understand a little, but don't try and get them ready for a career they'll never go into.
Why most people don't understand the math they learned and what it is used for, I would argue has much more to do with how they are not taught to interpret what they see around them mathematically- granted that takes years of practice, but we do not care to teach this to children.
Now streamlining math is all well and good, but we may loose some of the greatest mathematical minds by doing so. Many famous mathematicians (Euler, Lagrange for example) were slated to be someone else- professionally that is. And it took other great mathematicians to notice them.
Yes. I agree generally--often even at relatively advanced levels.
The stats course I took in engineering grad school was so heavy on the math that I'm not sure I appreciated what the math was in service of. By contrast when I took a less math-heavy stats course later I much better appreciated the underlying principles.
But yeah, it's very easy to turn any of these math-related topics into: memorize the formula, plug the numbers, come up with a result.
Right before any of the doors is opened we have no idea what to expect, so there are 3 equaly likely possibilities:
1. prize is behind A 2. prize is behind B 3. prize is behind C.
In case (1) you win by staying with your choice of A. In case (2) Monty Hall must open C (by the rules), so you'll win if you switch to B and in (3) the host will open B(by the rules) and you should switch to C.
In 2 cases out of 3 (2/3) you win by switching and in only one case out of 3 (1/3) you win by staying with your first choice. Since 2/3 > 1/3, you should switch.
This problem is easy if you know the defintions of sample space, event and the Equally Likely Probability Formula.
You are shown 100 doors 1 ... 100. Imagine you pick 1 door.
Of the other 99 doors Monty tells you which 98 of them don't have the prize.
He now asks if you want to switch to the 99th door that he didn't cancel out.
Intuitively that seems to make much more sense to me.
I think that commitment bias or what it's called is also an explanation for the misunderstanding in the first place. The argument against the increased chance, as I would use, works both ways and that is much more apparent the smaller the initial chance is. But I only intuitively used it to defend my position.
A sample space is the set of all possible outcomes of a random experiment. An event is a subset of a sample space.
Let S be the finite sample space with equally likely outcomes in it. Let E be an event. Then the probability of E is the number of outcomes in E over the total number of outcomes in S.
Now consider 52 cards in a deck. The sample space of outcomes here are the 52 cards. What is the event that the chosen card is a black face card? E = {J-club,Q-club,K-club,J-spade,Q-spade,K-spade}. What is the probability that the chosen card is a black face card? 6/52 by the formula above.
Monty Hall problem can be solved by using the above as a guide.
Sample space here : {switch, switch, stay-with-your-original-choice}. By the formula above, the probability of switching is 2/3 and the probability of staying put is 1/3.
If you're doubting the formula, I can provide you with a proof.
Two thirds of the time, you pick the wrong door on your first choice. So far, so obvious. Now, you picked a wrong door, Monty opens the other wrong door, and there are only three doors. Obviously the remaining door has the prize.
Every time you pick the wrong door, and then switch, you win. That means two thirds of time, if you switch, you win. That makes switching the right strategy.
These days you can solve many practical statistical problems with Monte Carlo, but understanding why those curves look the way they do is still very useful.
Perhaps linking integrals/derivatives into stats could be done from a more discrete approach as opposed to a continuous one first.
For example: Starting with a representation of an integral as sum of areas of rectangles (something most students can do) and then saying what happens if the width of all the rectangles gets smaller and smaller. A la Riemann
This could then tie into the pdf quite reasonably?
When we got to continuous distributions, we kind of hit a wall. Here's the puzzle. Suppose the data are recorded to two decimal places, and are reasonably bounded, for instance they are between 0 and 100. That's a discrete distribution. All of the data in the examples and problem sets were fixed-point.
I literally couldn't justify to someone who had never taken calculus, why a wholly different kind of distributions with its own collection of formulas, was necessary or useful. The teacher had never explained it either.
Instead, to deal with this 'a lot of really small things' we need a slightly different approach. Then draw a pdf, and talk about area under the curve. Perhaps relate this to the discrete case where the curve is just really blocky.
Vs
Trigonometric substitute to solve analytically.
For example.
Until calc is needed in the statistics. On which level do you even compare this, space or time complexity? I mean, are you saying, the complexity of the sum of the stochastic in use exceeds that of calc?
I think you gave an unfounded statement, without proof. i'm guessing you are biased to think statistics is harder, because you were less exposed to those classes and now you have remorse. If it was the other way around, you'd want more calc.
The way curricula are laid out, I'm getting the impression it is not so much about what is taught, school is rather a benchmark of abilities and a day care.
If we can't have better math instruction, let's at least have more relevant math instruction. Calculus is very useful in some disciplines, like physics, but startlingly useless elsewhere. Cool, yes. But useless.
More here: http://short-sharp.blogspot.ca/2015/11/grade-12-math-somethi...
Most who teach stats at the undergrad or hs level have little choice but to defer a lot of things to a later course.
same way you can do linear geometry without calculus.
the class really needs to be taught well for people to understand it. also. i think it should include probability too.
I think a good stats education requires heavy emphasis on doing real problems with what you learn. It's so easy to do and apply to the world around you. Unfortunately, most books that even touch on this have absurd text and problem sets that revolve around pulling jelly beans out of a bag or rolling a dice (I mean yeah, but super literal). Mostly the professors and teachers I knew that taught statistics at the lower levels always seemed bored and wanted to be elsewhere. A shame, really.
At my high school (Texas public school), Calculus and Statistics were both optional AP courses.
You can usually choose whether or not to take whichever combination you want. Some people took neither. I took both.
If high schoolers had to take one advanced class, I think it should be discrete math. Keeps their algebra skills sharp, introduces them to proofs, and still introduces them to basic probability. Obviously the availability of this offerring makes this impractical at the moment.
The problem with statistics education is that it is generally heavily watered down, particularly the classes at the university level for business, psychology, and other majors that need the knowledge but aren't math or engineering majors. People heavily misunderstand the usage, difficulty, and nuances of stats education if they are only exposed on this level. It's better than nothing, but leads to serious issues that remind me a bit of the novice arrogant programmer vs. the grizzled veteran smart senior programmer. Generally, most people I see with limited stats education come out of it thinking mean, median, normal distributions, and at best linear regression are the whole of stats.
Another problem that also related to the parent with regard to stats vs. calculus is that calculus is actually a pre-requisite for most useful statistics work. It is true there are plenty of topics in stats you can cover without knowing calculus, but you are at a severe disadvantage and won't really understand the "why" and just be regurgitating the memorized method. I feel a lot of statistics is a combination of the "why" and "how" because so much of it is applying math at a more discretionary and opinionated level, like is done in science. Really learning stats requires a lot of math, from advanced algebra to diff eq possibly to matrices to calculus. To do good work in stats, you need a lot of tools to draw from, otherwise you're stuck with tools that often don't work (common example is people who only are familiar with normal distributions and nothing else).
Yet another problem with stats is that you can really make up whatever you want when you conduct statistical applications such as an analysis. The rule is that as long as you provide justification and proof for what you are doing - i.e. how and why, it is acceptable along with some text to explain. This is why good statisticians do things like describe and reveal their sampling methods, explain what did and did not work, and why they selected various tools. Obviously most dubious work can be quickly disputed and identified by someone who knows what they are doing, but the reality is that so many people never follow-up, fact check, or even read anything but the conclusion.
Stats is in so many ways about showing the process and justifying your answers than the quantitative answers. This is similar to financial forecasts as well for example which also use a lot of applied math, and people just believe what they read even though the evidence to disprove the report is right there. The science comparison applies here too. Just like with research studies, a lot of people don't check, verify, and re-run the results so people just believe what they read or rubber stamp it. I recall I had a very mean professor that once gave a final exam question that actually could not be solved at all using the methods we learned because the types of analysis and models we were asked to try to apply were invalid for the data. The simplest version of this familiar to many people is trying to create a best fit linear equation to non-linear data - simply draw it on paper against a data plot and you'll see it makes no sense.
If there's one important thing I learned in stats, it's to be skeptical and prove things with evidence. Nearly every time I watch the news, read an article, etc., I go nuts because they are making huge conclusions without giving you a way to understand who, what, where, why, and how about the study. The presidential elections in the US are an obvious example I see a lot with things like "Survey Monkey" passed off as scientific data in some contexts. Even represented as a casual, unscientific process, such things can be dangerous because people still accept statistics at face-value usually.
In summary, yes, people should learn statistics but it requires a lot of hard work to develop the tools and analytical skills to do it right. Like calculus, it can require many courses which can equate to years of study. Like anything, the skills involved also tend to get better with time and increased knowledge. At least an extra course or two beyond what is taught now would go a long way, but people really need to emphasize more than stats as repeating some processes of functions, formulas, proofs, and such, and more as a way of thinking and set of tools. Stats should teach you to be very paranoid and skeptical, and verify what you see in the world around you as best you can before you believe it. Once you do that, like the parent says, you will see so much clearer what is true, false, dubious, could be better, etc. in the world around you.
I had the opposite problem, which was that I learned stats as a bunch of proofs with practically no applications. I only use fairly basic stats today, but I often test my conclusions by running some sort of monte carlo analysis. I graph everything.
It sounds like a lot of what you're recommending could be described as teaching general quantitative scientific methodology.
This is the general problem with math education in general. Just equations. Solving equations is relatively easy. Coming up with them is where the real challenge and innovation is.
I still feel like calculus is one of the more valuable types of math. It really does teach you a lot, but like anything, it requires a good teacher. I find it all ties together for me in the end, like when doing graphics programming and games, I use a lot of trig, geometry, algebra, and calculus even, and stats some (and even more for business-type stuff).
From a computer science, more specifically programming, background I find linear algebra, logic, calculus and the combination thereof to be more insightful when handling abstract ideas.
I fear that if too much focus is put on statistics then we many end up with another generation of heavy statistical machine learning (and programs that consume high amounts of power to function).
Or do you mean statistics as a bunch of applied methods, where you follow some recipes without understanding the math behind them? It's taught that way even at university level, but I can't see how this could be helpful in the long run.
Statistics is relevant to more fields than calc. It matters a lot more in business and in all the jobs where you use calc you will also have to use statistics. It's also useful for citizens to understand statistics better - and educating citizens is truly one of the purposes of public education.