News source: https://www.bbc.com/news/uk-63330885
Twitter thread: https://twitter.com/petesherlock79/status/159025317484515328...
And a response thread explaining the flaws in the methodology:
As news source: https://novaramedia.com/2022/11/09/the-bbcs-investigation-in...
And on twitter: https://twitter.com/AyoCaesar/status/1590403667974582272
Basically the model was developed at google to find “toxic speech”, but the researchers had a very biased definition on what constituted as “toxic”. Racist speech didn’t count, while angry shouts did.
> I entered a selection of racist tweets I’d received in the past year into Perspective’s API, of varying lengths and sentence complexity. Just to make it easy for the machine, I deliberately chose one which included a common racial slur against South Asians.
> None of these were registered as potentially toxic at all by the AI – but, “You’re a fucking G”, a compliment, popped up with a 90.29% likelihood of being toxic.
Your hypotheses that white MPs experience more racist speech than non-white MPs, is testable. I have a pretty strong feeling that you are wrong, but I haven’t seen any data to support either (only anecdotes; and this deeply flawed analysis). What you can do to test your hypotheses is find all British MPs on twitter, scrape all tweets directed at them, and measure the proportion of racist speech against them. However, I advice you to do a more traditional cluster analysis, rather than a training model, the latter is very likely to yield a flawed model just like the journalist’s “8 month labor of love”.
This kind of reminds me of when some journalist on twitter used machine learning to find hate-speech against British politician. Of course the journalist found exactly what they were looking for. There was more hate speech against conservatives... Of course, the journalist was only an amateur in statistics, and used a flawed model, that could have yielded anything they wanted
disclaimer: I didn’t read the article because paywall, and waiting for an archive to be posted.