News for hackers. Stuff that matters.
Re:How to parse a sentence and decide if to answer "that's w... by r00fus on Wednesday April 27, 2011 @07:47PM (#2491569)
Answer ignores a very public and growing corpus of data: IRC channels.
Re:How to parse a sentence and decide if to answer "that's w... by enjo on Wednesday April 27, 2011 @07:59PM (#2491597)
The author makes a great point, however, about the ability to link those two together. I'm sure you'd end up with a lot of false positives from IRC as even if it's attributed (ircuser: TWSS!) the flow of IRC conversations is such that it's not always the last thing ircuser said that is being replied to (because of network lag and just general reply lag in IRC).

Although now that I think about it... twitter probably often suffers from the exact same issue, although somewhat mitigated as people often include the original tweet in their reply.

Re:How to parse a sentence and decide if to answer "that's w... by DasIch on Wednesday April 27, 2011 @08:36PM (#2491668)
Unless I'm missing something, this only occurs if someone sends two messages to someone and the receiving user responds to the first message but gets the second message before sending his.

Using average read/write times for a message - taking its length into account - it should be fairly easy to check if someone could have read and written a response to a message. Assuming it is a response if he could have written one in time should be fairly accurate.

You could probably even cheat and use a constant time while still getting good results.

Re:How to parse a sentence and decide if to answer "that's w... by follower on Wednesday April 27, 2011 @07:59PM (#2491598)
I initially thought that myself but this is covered in end note [4]:

"[4] I think I once heard of an MSN chat transcript dataset that was really awesome, but I can't seem to find mention of it anymore. Let me know if you know where I can find that or any other instant message datasets. I know that some IRC rooms get publicly logged — is there a single place where one could grab all of them at once?"

Re:How to parse a sentence and decide if to answer "that's w... by r00fus on Wednesday April 27, 2011 @08:11PM (#2491627)
I wouldn't assume just grab an existing log, wouldn't it be just like twitter? (ie, just "follow" a bunch of conversations/channels and build your own corpus by "tapping into the firehose").

You could use pre or post filtering to weed out connect/disconnect and other noise.

I only thought of this because there was like two "TWSS" replies in the past couple of days on #rubyonrails

Re:How to parse a sentence and decide if to answer "that's w... by DasIch on Wednesday April 27, 2011 @08:43PM (#2491680)
You could write a simple client that just ignores connect/disconnect messages from the server.

Reading this post I remembered a small project called sociograph a friend of mine created. It's an IRC bot that logs messages and draws a graph of the people communicating with each other in realtime, see for an example http://www.youtube.com/watch?v=A_ah-SE-cNY.