News for hackers. Stuff that matters.
Re:Ask HN: flirt140, how are they determining the gender? by patio11 on Wednesday May 13, 2009 @09:41PM (#608073)
I don't know about Twitter, but determining gender (on average) from English language text is pretty much a solved problem in natural language processing.

http://www.hackerfactor.com/GenderGuesser.html

See this paper:

http://u.cs.biu.ac.il/~koppel/papers/male-female-text-final....

Sidenote: this was an epic win for me in Gender Studies class, when I said that not only was the professor's claim that the genders communicate "essentially identically but with some individual variation" wrong, but I would constructively demonstrate it by blind-assigning a stack of any papers she cared to give me. (I also gave her a copy of the above paper, which she read with interest.) The class then retreated from the scary notion of binary decisions on the basis of the scientific method to the more comfortable one of endless arguing whether the differences were socially constructed or biological.

Re:Ask HN: flirt140, how are they determining the gender? by aneesh on Wednesday May 13, 2009 @10:47PM (#608135)
There's a similar service at genderanalyzer.com, based on UClassify's API - http://blog.uclassify.com/gender-text-analysis/.

It thinks Sergey Brin writes like a woman: http://www.genderanalyzer.com/?url=too.blogspot.com

Re:Ask HN: flirt140, how are they determining the gender? by jdale27 on Thursday May 14, 2009 @12:53AM (#608265)
Maybe that ad for 23andme was actually written by Sergey's wife.
Re:Ask HN: flirt140, how are they determining the gender? by blurry on Thursday May 14, 2009 @02:29PM (#609320)
What a load of crap. According to this tool, Linus Torvalds writes like a woman, and so do Zed Shaw and Steve Yegge.
Re:Ask HN: flirt140, how are they determining the gender? by kwamenum86 on Wednesday May 13, 2009 @10:59PM (#608149)
But does this work this same way with a 140 character limit since people are often forced to express themselves in a different way?
Re:Ask HN: flirt140, how are they determining the gender? by jey on Thursday May 14, 2009 @12:48AM (#608261)
As long as you have an appropriate training set, I don't see why it would be significantly different. You might run into problems if you train the classifier on some radically different dataset but then try to run it against twitter accounts.
Re:Ask HN: flirt140, how are they determining the gender? by daveying99 on Thursday May 14, 2009 @04:57AM (#608447)
The twitter API gives access to a large number of a user's latest tweets. So it's a series of 140 characters which makes the gender classifier more accurate.
Re:Ask HN: flirt140, how are they determining the gender? by kngspook on Thursday May 14, 2009 @07:56AM (#608574)
Yeah, but the API doesn't give you access to those users' gender (since Twitter doesn't ask/store that info), which means you have no way to tell the classifier "Here are 500 males' tweets, here are 500 females' tweets."

I suppose you could manually find males and females and train based on their body of works, but it won't be great; you'll likely run into a selection bias.

But if you seeded with that approach, and then used a SpamAssassin-style auto-learner...maybe you'd have a chance?

I suppose this is a case where you don't want "Perfect" to get in the way of "Good enough", especially since it will never be perfect...

Re:Ask HN: flirt140, how are they determining the gender? by johnnybgoode on Wednesday May 13, 2009 @11:49PM (#608209)
That was great. Gender Studies class, huh?

By the way, Gender Guesser gave me a good laugh when I ran it on a comment from HN. It said, "Verdict: Weak MALE Weak emphasis could indicate European."

A scientific basis for the effeminate European stereotype??

Edit: I was joking about the stereotype.

Re:Ask HN: flirt140, how are they determining the gender? by patio11 on Thursday May 14, 2009 @12:10AM (#608232)
Gender Studies class, huh?

It was a requirement for graduation with a Arts & Sciences degree (I dual-degreed with CS from Engineering) that I demonstrated an interest in subjects covering things other than dead white males. For reasons which were never really clear to me, "I'm an East Asian Studies major!" was not sufficient, so my choices were either Gender Studies or The History of Jazz.

I will say this for required common curricula: if you only take courses which you're interested in or whose built-in intellectual biases are flattering to yours, then you're missing out on a good deal of the educational experience.

I will also say this against required common curricula: there were no gender studies majors in AI class.

Re:Ask HN: flirt140, how are they determining the gender? by johnnybgoode on Thursday May 14, 2009 @12:56AM (#608270)
Oh, I wasn't criticizing your choice. But surely your experience led you to question certain things.
Re:Ask HN: flirt140, how are they determining the gender? by dangoldin on Wednesday May 13, 2009 @11:27PM (#608184)
Thanks for the links!

I find it pretty cool that the first site only does the guessing based on a pretty small list of words - you can see it by viewing the source.

Some words are more commonly male and others are more commonly female - looks as if that's all it is doing.

Re:Ask HN: flirt140, how are they determining the gender? by MichaelApproved on Wednesday May 13, 2009 @10:04PM (#608098)
I submitted this HN post and it says you're Male for both formal and informal.

So, whats the verdict Patio11, are you Male or Female?

Great link nonetheless.

Re:Ask HN: flirt140, how are they determining the gender? by d0mine on Wednesday May 13, 2009 @10:28PM (#608119)
Gender Guesser is less accurate (60%-70%) for HN then a Perl one-liner (>95%):

    say 'Male'
Re:Ask HN: flirt140, how are they determining the gender? by wlievens on Thursday May 14, 2009 @05:16AM (#608452)
Maybe they integrated that bias into their system?
Re:Ask HN: flirt140, how are they determining the gender? by vijayr on Wednesday May 13, 2009 @10:08PM (#608101)
patio11 is male. I submitted a few text too, its quite accurate.
Re:Ask HN: flirt140, how are they determining the gender? by patio11 on Wednesday May 13, 2009 @10:41PM (#608131)
I have always found it kind of... (weird? An amusing quirk of the culture?) that my HN account is tied very closely with my real identity and yet I'm always addressed by login name rather than real name here. I guess you can file that away as yet another example of the power of defaults.

On other business-related forums, like the Business of Software board, everyone either calls me Patrick or "that bingo guy". That might be because of the display name alone.

Re:Ask HN: flirt140, how are they determining the gender? by Poiesis on Thursday May 14, 2009 @12:13AM (#608239)
Oddly enough, given the post subject, I knew who you were immediately after reading a comment of yours about a week ago (and it had nothing to do about bingo, though it may have been entrepreneurial). I read it and immediately though it sounded like your writing style but didn't recognize the username. Clicked through, and lo and behold, there you were. Just had to register and mention that after your comment.
Re:Ask HN: flirt140, how are they determining the gender? by JacobAldridge on Thursday May 14, 2009 @03:02AM (#608371)
I figured you had the choice of username, so even though you're Patrick in your profile and if I ever met / emailed you I would use that, you had chosen to be patio11 in this forum.

I also find it odd at times - my username is my full name, yet people responding to me will call me JacobAldridge not Jacob.

Re:Ask HN: flirt140, how are they determining the gender? by jhamburger on Wednesday May 13, 2009 @10:51PM (#608139)
Just looked at a few dozen tweets, the use of exclamation points seems to be a dead giveaway.
Re:Ask HN: flirt140, how are they determining the gender? by badger7 on Thursday May 14, 2009 @06:07AM (#608491)
I hate that!!!!!! It's one of my myriad pet peeves!!!!!!! Along wif spellen like ur a morun, wevver it's on perpose or not!!!!! It makes you look like a drama queen!!!!!!! Which, to me at least, is a huge insult!!!!!!!!!!!!!! And it's some inane piece of misspelled, new-age bullshit far too often!!!!!!! Shock!!! Horror!!!!!!! Shock!!!!! Horror!!!!!!!! <slap>

Sorry. Has anyone built the total perspective vortex yet?

Re:Ask HN: flirt140, how are they determining the gender? by vijayr on Wednesday May 13, 2009 @09:43PM (#608076)
that is a neat link. thank you for posting it.