News for hackers. Stuff that matters.
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by dm8 on Wednesday August 01, 2012 @03:09AM (#4321566)
Berkeley, sadly, is perhaps too large and diverse for an overall characterization.

So true. As a Cal alum, I think he is spot on here. :-)

I pulled about 400 followers from each school, and added a couple filters, to try to ensure that followers were actual attendees of the schools rather than general people simply interested in them

How did he ensure followers were actual attendees of the schools programmatically? It would be really hard to find out this type of information. And can be considered as borderline creepiness in some cases.

EDIT: He also works as a data scientist at Twitter. So I'm sure he has access to lot more internal data rather some sort of mashup between Twitter, FB, LinkedIn APIs.

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by aggie on Wednesday August 01, 2012 @03:55AM (#4321668)
How did he ensure followers were actual attendees of the schools programmatically?

He mentions in the comments "I basically just checked that they didn't follow any other schools (from a small list). It's certainly not the greatest filter, but it did seem to work for a small number of people I hand-checked."

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by oraj on Wednesday August 01, 2012 @03:11AM (#4321573)
400 people might not be a lot to go through if you have lots of spare time on your hands.
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by ScottBurson on Wednesday August 01, 2012 @02:50AM (#4321516)
Interesting. Would also like to see p(topic|school=X).
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by ballooney on Wednesday August 01, 2012 @03:15AM (#4321581)
Well, god gave us Bayes' rule for a reason.

The article does not point out that sampling quora users might not be (I would say is probably not) an unbiased estimator of the the students of these places as a whole. Quora attracts a certain kind of person, I've never seen it mentioned anywhere outside of techy startup / valley circles. Maybe that's implied by virtue of it being in the HN ecosystem, but still it should be explicitly stated as a flaw in the method. Lies, damned lies and statistics etc.

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by azylman on Wednesday August 01, 2012 @03:29AM (#4321608)
The article does not point out that sampling quora users might not be (I would say is probably not) an unbiased estimator of the the students of these places as a whole.

From the article:

Also, a word of warning: my dataset was fairly small and users on Quora are almost certainly not representative of their schools as a whole (though I tried to be rigorous with what I had).

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by ballooney on Wednesday August 01, 2012 @03:38AM (#4321624)
I completely missed that, mea culpa, my apologies to the author. I'll leave the message above unedited as a little warning sign to other people who can't read: you too can look silly like me.
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by ScottBurson on Wednesday August 01, 2012 @12:50PM (#4324031)
> Well, god gave us Bayes' rule for a reason.

She did. But She did not give us P(topic) and P(school) for this dataset.

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by rjtavares on Wednesday August 01, 2012 @05:00AM (#4321847)
MIT and Stanford like Hip-Hop Music. I'm pretty sure MIT prefers Biggie and Stanford prefers Tupac (note: this has nothing to do with east/west coast, but rather pure skill vs. charisma)
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by carlob on Wednesday August 01, 2012 @09:39AM (#4322751)
Interesting, but wrong use of conditional probabilities. All OP is saying is that the frequency of people from school x following topic y is p.

The dataset is just not the right one to say that P(x|y) = p, because of, say, all the people who follow food in NYC and go to NYU which were not taken into account here.

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by sesqu on Wednesday August 01, 2012 @01:55PM (#4324508)
Surprisingly diverse interests, considering the huge bias in the dataset (public quora profiles). Well worth reading, though I was left wondering about the inverse probabilities.
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by sadga on Wednesday August 01, 2012 @01:19PM (#4324243)
Article title should be "Reverse index of where people went to school who talk about various topics".

Obviously, the author chose a more wieldly title, but a far less correct wrong.

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by romain_g on Wednesday August 01, 2012 @03:59AM (#4321682)
Taylor Swift ?
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by probably on Wednesday August 01, 2012 @09:52AM (#4322811)
Dude Taylor Swift is awesome. I can relate.

But good eye on that. ;)

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by bitwize on Wednesday August 01, 2012 @11:26AM (#4323393)
Yeah, how did she score higher than Daft Punk?
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by wilfra on Wednesday August 01, 2012 @03:42AM (#4321636)
"Berkeley, sadly, is perhaps too large and diverse for an overall characterization."

That isn't sad at all. It's great. As a UC grad, the diversity of the student body was one of the absolute best things about my college experience. Do I wish I went to Harvard or Stanford? Sure, but not for their homogeneous student bodies.

Otherwise, pretty interesting read.

Re:Stuff Harvard, MIT, Stanford & Caltech People Like by echen on Wednesday August 01, 2012 @03:55AM (#4321669)
It was a tongue-in-cheek comment :) -- sad from the perspective of me trying to find a cute pattern for Berkeley.
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by baritalia on Wednesday August 01, 2012 @03:24AM (#4321598)
Interesting thing to read while sipping your morning coffee but that's it, really.
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by nuttendorfer on Wednesday August 01, 2012 @04:20AM (#4321750)
Which is just what I did. Nothing wrong with that.
Re:Stuff Harvard, MIT, Stanford & Caltech People Like by yen223 on Wednesday August 01, 2012 @04:06AM (#4321704)
Not every article has to be a profound, life-changing one :)