News for hackers. Stuff that matters.
Re:Programmers Need To Learn Statistics Or I Will Kill Them All by AlphaMonkey on Saturday January 09, 2010 @03:42PM (#1041957)
Look, I know it's none of my business, but if you're a newbie and you want to learn some Statistics, wouldn't you be better off checking the MIT OCW pages, or the countless lecture notes in PDF format that can be found on the web? This is an honest question.

Contrary to what many short-sighted "pure" mathematicians think, Statistics is hard. You can't explain statistics in a bunch of blog posts, not even if you are a true expert. You can't condense all that knowledge in a blog post, one must read the books, though painful that may seem.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by loumf on Saturday January 09, 2010 @04:57PM (#1042038)
To learn statistics, yes, of course. To get psyched up about learning statistics? That's what this article did for him.

Sometimes it's just fun to read someone go nuts about something like statistics, and it may give you motivation to go learn more about it so that you can understand the whole thing.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by tobtoh on Saturday January 09, 2010 @05:43PM (#1042106)
I agree. The last time I dabbled with statistics was at university and whilst I could see it was useful, it was boring as anything. I couldn't see any really application in my day to day job - after all, besides understanding the difference between mean and medium, what else would I need?

But Zed's article, juvenile as it was in tone, had undeniable passion about statistics. And it was enough to keep me reading through the whole article and pay attention to the examples. As it is, I will be doing further reading on statistics and so Zed's article has been incredibly useful to me.

"Maybe it's just me, but Statistics is a beautiful field, and such intellectual beauty should be all the "psyching up" one needed."

It's just you. Nine out of ten people think statistics are nothing more than boring numbers made up on the spot.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by AlphaMonkey on Saturday January 09, 2010 @05:51PM (#1042124)
Nine out of ten people learned Statistics from teachers who knew jack of Statistics. The difference between Statistics and Math is that Statistics is data-oriented and somewhat "experimental". Yeah, I hated Statistics in high-school, but when I learned Information Theory and Statistical Signal Processing, then I understood what one could do with knowledge of Statistics, and what the field is all about.

If you can't see the beauty in it, you've probably been taught pseudo-Statistics. Don't feel bad. All the Math that engineering students learn is kind of pseudo-Math. All one learns in high-school is BS. If you want to learn something, here's my advice:

i) Don't use modern, over-designed textbooks.. you know... the thick expensive ones with many colors and boxes highlighting the formulas one must memorize.

ii) Instead read the classic books from the 1960s, many of which are published by Dover. They look boring at 1st sight, but their content is rich. Another option is to get the old Soviet books from the 1960s, which tend to be forgotten gems. The Russians are the best at marrying theory with application.

[deleted]
Re:Programmers Need To Learn Statistics Or I Will Kill Them All by skorgu on Saturday January 09, 2010 @06:04PM (#1042144)
So how is one to ever know that stats is beautiful if all they see is impenetrable jargon? Where is a good, free, up-to-date resource for doing what Zed covers?

We're all busy, accessibility and immediate relevance are common and useful heuristics for deciding what to investigate.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by AlphaMonkey on Saturday January 09, 2010 @06:10PM (#1042148)
I find MIT OCW courses on Statistics a good source of references. Reading books takes time, but you can find some lecture slides and lecture notes there.

I, too, found Statistics impenetrable. One solution I found was to learn Statistics from the Electrical Engineering and Computer Science perspective. Instead of weird abstractions, in EE and CS one thinks of more concrete things, like random signals, or data sets. The best way to learn Statistics is to apply it.

One problem, though, is that knowing only a little Statistics might be worse than knowing nothing.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by skorgu on Saturday January 09, 2010 @06:53PM (#1042215)
Considering the article is a concrete introduction to statistics with applications and is passionate about your last point you seem to agree with Zed.
Re:Programmers Need To Learn Statistics Or I Will Kill Them All by ErrantX on Saturday January 09, 2010 @04:56PM (#1042035)
> You can't condense all that knowledge in a blog post, one must read the books, though painful that may seem.

No, that's a silly approach. If your, say, launching a web app advice like this [sic] is useful. You dont need or want a deep understanding, you just need the pointers to analyse stuff right.

(as it stands I never mind Zed's tone and when I first read this article it provided some useful info to me in a simple format :)

Also r.e. your first post - from what I recall Zed can be considered some sort of an expert on this stuff.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by AlphaMonkey on Saturday January 09, 2010 @05:16PM (#1042064)
Ruby ninja, guitar-player extraordinaire, self-taught Statistics expert... wow!!! Zed's propaganda machine rivals that of Kim Yong-il.

Get this: you need deep understanding of the fundamentals, otherwise you're not doing Statistics, you're doing Voodoo Magic. If you learned some Statistics from Zed's post, then you must know even less than he does. But then, one of the many symptoms of ignorance is the false belief that one knows. Do you even realize how many years one needs to become an expert on any field, especially a tough field like Statistics?

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by ErrantX on Saturday January 09, 2010 @05:52PM (#1042126)
Look, I dont want to get into a protracted argument... but.

It seems your definition of "expert" is a lot higher than mine; which is fine. Lets use a different word: experienced.

However with that said what Zed wrote in that post stood up to extra research by myself and I actually understood what he had to say.

If you wish to claim it is wrong and he has no clue, fine; but can you please - for our benefit - explain why and specifically where he is wrong?

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by AlphaMonkey on Saturday January 09, 2010 @06:01PM (#1042140)
I am not saying Zed is wrong, I am merely saying his post is not really that insightful... at least when compared to other essays on Statistics one could read instead. For example, what Cosma Shalizi writes on Statistics is much deeper than what Zed did. Another example: Andrew Gelman has a great blog on Statistics, and he's a true expert.

For me an "expert" is someone who has at least 10 years of experience in one field. An expert on Statistics must have carried out extensive data-analysis work, or must have published original papers. Reading books gives one some understanding, but only when one must solve a problem, does one realize that one's knowledge is utterly superficial.

Zed is no superman. His area of expertise is not Statistics, it's something else. I am not saying he's stupid. All I am saying is that he has not invested the many years of effort into studying Statistics to earn the title of "expert". By calling him an expert, one insults all the true experts out there, the one who do not write blog posts using juvenile language because, you know, they have better things to do... like spending time with their friends and family...

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by ErrantX on Saturday January 09, 2010 @06:18PM (#1042163)
Cheers, Ill look those guys up.

I do think 2 things are happening here though. I think firstly Zed's style doesn't really appeal to you - which is fine, I can see how he wouldn't gel with lots of people. As a result the second thing happens, his content seems irrelevant.

As someone who quite enjoys his approach :) I found quite a bit of useful content. I dont think the point is really to infuse a deep understanding of statistics - it's to correct some common mistakes hackers like us make when playing with stats :)

For example I scanned the blogs of those 2 names you mention and their stuff certainly seems interesting; but frankly I didnt notice anything massively relevant to things I might need to use day to day in my "startup". Zed's stuff I've already taken to heart and corrected some of my "work practices" when dealing with stats.

As to the rest I think it is just a case of differing definitions of expert :) as I said "experienced" is a better word to use - assuming our definitions match :P.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by AlphaMonkey on Saturday January 09, 2010 @06:25PM (#1042171)
The world changed faster than Statistics. These days, anyone with a computer can analyze data sets, which is awesome. The problem is that the Statistics books are not designed for the masses, and the experts are concerned with technicalities that programmers, for instance, couldn't care less. If Zed helps brings Statistics to the programmers community, I guess that can only be a good thing...

The good news is that there seems to be an opportunity here: write Statistics for the non-statistician. Not everyone who needs Statistics can take 2 years off to learn it. Back in the 1970s, Digital Signal Processing was an advanced grad course... now it's a basic undergrad course. Maybe Statistics will go the same way, becoming more and more prevalent, and less and less ivory tower.