Re:XCheck at Meta: Why it exists and how it works
by 1letterunixname on Sunday October 16, 2022 @05:29PM
(#33227349)
So I work at Meta and just got MetaMorphized s/@fb/@meta/g.
When I started, my personal account was banned twice on the condition of adjusting 2FA. It was sorted out. I had a past account that I could no longer access as email domains changed, so it looked like I was trying to maintain duplicate accounts. Why it triggered on adjusting 2FA didn't make sense. Then, it wanted me to verify my identity in a way that created a Catch-22 of wanting a login from itself or to a device that didn't have a valid login session.
I recently spoke with someone in Readiness who works with hiring human verifier contractors around the world. A primary issue is scale. You would need to hire the entire world's population to moderate the content being produced. Still there will be bias and misinterpretation of sarcasm, and varying standards of acceptability and decorum. AI is a force-multiplier to an extent, but it takes human judgement to rectify mistakes and data to identify brigades, scammers, terrorists, and political manipulators seek to exploit an imperfect system. Sadly, the humans with the best judgement typically have better career options that they couldn't be paid to do social media moderation. It can be made better, but the ultimate realization is with even great care, good intentions, and attempts at making things sensible and fair, there are always going to be mistakes. It's trying to minimize mistakes and not enable genocides, election sentiment manipulation, or product scams. It's doable to minimize mistakes, but it take persistent vigilance, wisdom about human nature, and creative solutions to deter and prevent harm while avoiding harming innocent persons. Mistakes are bad and disappointing and it feels bad when they happen.
In case anyone were wondering, the security is the inverse of Twitter's. Everything is logged and access requires a business purpose for a limited time and approval to get that access. Almost no one has access to production data. PII is taken very seriously. There are no laptops with copies of user data. All laptops are encrypted. I can see my work/personal FBID user object in the graph, but as soon as I try to prod any links to other users, big warnings appear.
When I started, my personal account was banned twice on the condition of adjusting 2FA. It was sorted out. I had a past account that I could no longer access as email domains changed, so it looked like I was trying to maintain duplicate accounts. Why it triggered on adjusting 2FA didn't make sense. Then, it wanted me to verify my identity in a way that created a Catch-22 of wanting a login from itself or to a device that didn't have a valid login session.
I recently spoke with someone in Readiness who works with hiring human verifier contractors around the world. A primary issue is scale. You would need to hire the entire world's population to moderate the content being produced. Still there will be bias and misinterpretation of sarcasm, and varying standards of acceptability and decorum. AI is a force-multiplier to an extent, but it takes human judgement to rectify mistakes and data to identify brigades, scammers, terrorists, and political manipulators seek to exploit an imperfect system. Sadly, the humans with the best judgement typically have better career options that they couldn't be paid to do social media moderation. It can be made better, but the ultimate realization is with even great care, good intentions, and attempts at making things sensible and fair, there are always going to be mistakes. It's trying to minimize mistakes and not enable genocides, election sentiment manipulation, or product scams. It's doable to minimize mistakes, but it take persistent vigilance, wisdom about human nature, and creative solutions to deter and prevent harm while avoiding harming innocent persons. Mistakes are bad and disappointing and it feels bad when they happen.
In case anyone were wondering, the security is the inverse of Twitter's. Everything is logged and access requires a business purpose for a limited time and approval to get that access. Almost no one has access to production data. PII is taken very seriously. There are no laptops with copies of user data. All laptops are encrypted. I can see my work/personal FBID user object in the graph, but as soon as I try to prod any links to other users, big warnings appear.