News for hackers. Stuff that matters.
Re:Happy Little Words – Analyzing the Bob Ross Twitch Chat by degenerate on Friday November 20, 2015 @02:59PM (#10603447)
Semi-related curiosity question: When viewing twitch chat as a logged-out user, if someone enters a banned word, it shows me "<message deleted>." However when there are dozens of messages per second like the bob ross chat, the message deleting seems delayed, because you can sometimes catch a glimpse of the bad words before they are "removed". I assume the removal is done with javascript? Are you seeing the non-censored version of the chat when scraping?
Re:Happy Little Words – Analyzing the Bob Ross Twitch Chat by renekooi on Friday November 20, 2015 @04:50PM (#10604105)
Messages are sent through plainly first over Twitch's IRC-ish protocol. You can connect to Twitch chat with a simple IRC client, or through the browser. Twitch's own browser chat connects to a WebSocket server that passes IRC commands to the browser, where they are parsed & executed.

Twitch has a few additional IRC-ish commands like CLEARCHAT, which deletes messages by a given user. Most IRC clients don't support this, but Twitch's browser client of course does :) In larger streams spammy messages are usually removed by bots like http://www.nightbot.tv/ or http://twitch.moobot.tv/. That's where the delay comes from: messages have to arrive at the moderator first.

Interestingly, CLEARCHAT can only delete _all_ messages by a given user (as far as I know), so non-offending messages are also removed. This is done by the client, the chat servers only pass "CLEARCHAT #channel_name user_name". (e; it's all messages by a given user, or all messages in the channel if no username is given)

In the Twitch browser client, you can double-click the <message deleted> text to show the original text if you really want to! :)

Reference: https://github.com/justintv/Twitch-API/blob/master/IRC.md

Re:Happy Little Words – Analyzing the Bob Ross Twitch Chat by stevelosh on Friday November 20, 2015 @03:02PM (#10603465)
I'm not 100% sure how they're doing it. It seems to be both bad words and spammy stuff (e.g. a line of just "KappaRoss KappaRoss KappaRoss..." a hundred times over will get removed too).

The IRC gateway passes through everything so I'm seeing it all unfiltered (as far as I can tell).

Re:Happy Little Words – Analyzing the Bob Ross Twitch Chat by eertami on Friday November 20, 2015 @06:59PM (#10604703)
Yeah the removal is clientside.

Most "twitch chat users" use BetterTTV[0] which adds some pretty useful features, and also stops messages from being deleted.

[0]: http://nightdev.com/betterttv/

Re:Happy Little Words – Analyzing the Bob Ross Twitch Chat by kencausey on Friday November 20, 2015 @03:01PM (#10603462)
I believe the message removal is done by a moderator. This may be a human being or a bot. In either case the message has to appear in the chat before either party can decide whether to leave or remove it.
Re:Happy Little Words – Analyzing the Bob Ross Twitch Chat by philipov on Friday November 20, 2015 @03:14PM (#10603547)
It's a bot. It's called XanBot [0]

Twitch's chat protocol is compatible with IRC, but IRC doesn't support all new features, so things like replacing banned words with <Message Deleted> won't show up.

[0] https://www.reddit.com/r/Twitch/comments/2dkro7/a_detailed_g...

Re:Happy Little Words – Analyzing the Bob Ross Twitch Chat by fmueller on Friday November 20, 2015 @03:22PM (#10603592)
In the desktop version of twitch chat the deleted lines are crossed but visible. So I guess it's just some kind of markup which marks a line as deleted.