I've worked at a few large social media platforms. The number one thing you realize is content moderation is hard. It doesn't scale well, human moderators aren't necessarily better than AI, and you're going to make people mad at you one way or another. Mental health issues of users are also complex. Different people might find the same content empowering or triggering.
To Dorsey's point, I understand how people miss smaller, federated communities of the early internet, but those don't scale. If you want a platform with the potential to reach everyone with an internet connection, it's going to get dumbed-down, and so will the conversations. Or you can build federated echo chambers.
90% of the criticisms you hear are armchair quarterbacking that's easy to ignore.
You might be missing the problem people have with AI moderation. People generally aren't upset that AI moderation gets it wrong. The problem is when AI moderation gets it wrong, there are no humans to review the process. A good example of this is every "Google closed me account and I don't know why" post on the front page of Hacker News every other week.
It made me realize that some people truly do not think critically about the impact of their work. I think Facebook inadvertently selects for those kinds of people (or at least filters out people on the other end of the spectrum).
There are internal discussions into ethical/moral consequences of decisions. Large organizations dedicated towards privacy, security, and integrity are given power to ensure the company operates ethically. It’s not perfect but it’s a hell of a lot more than most do.
I hate this moralization of companies btw. Companies are not good or bad, but they are capable of causing harm or good. Instead I see it often used because person X decides that they hate company Y, so in their mind they label the company as “evil” or “bad”. But of course they ignore harmful behaviors that their own company are making.
I’ve heard it all from folks in industry. Customer data leaking into Slack channels, data security being non-existent, no audit trails available in investigations, implementing only SMS MFA because it’s quicker and let’s them focus on more “impactful” projects. Ignoring verified accounts being sold on marketplaces. Making decisions which ultimately make the company a juicy target for attackers, without a discussion of trade-offs. Promoting culture which prioritize the company’s goals over the safety of users. Taking the “well that will never happen here” stance because their employees have “ethics” compared to those dirty FAANG employees. Promoting messaging platforms to children without proper safety protocols in place. Treating decentralization as the thing to fix all things, when historically the internet has been responsible for amazingly disgusting things long before the days of social media. The list goes on and on and on and on.
I try to write about this on HN to combat the popular narrative but I’m slowly thinking that this isn’t the place to do it. But alas here is another one.
Also, what’s wrong with moralization of a company? All companies cause some good and some harm, it’s worth determining what the net is and judging the company appropriately. I don’t think it’s too far off to call a company (subjectively) “bad” if their net impact is (subjectively) “bad”.
In my 40s now, I've had recent reflections about the small, golden friendship group of my university years. What they were then, vs the kind of life they pursued, and where they ended up. This little group of harmlessly rebellious nerds, playing computer games, smoking the odd joint and listening to heavy metal full of lyrics against The Establishment[1], etc. Playing our guitars and inwardly sneering (or more likely, laughing - we weren't really the sneering types) at the corporate world. [1]term used tongue-in-cheek, but I'm sure you know what I mean!
20 years later, and of the half dozen, I'm probably the only one left outside The Establishment.
Among the others, we notably (and disappointingly) have the aloof senior professional fully integrated into the Old Boy's Club of his industry, think Mason-y power conglomerates which run their local region for the profit of a few; and the commerce professional who regularly and gleefully spams LinkedIn with info about his latest Salesforce certifications, alongside Likes for Boris Johnson content.
Do they think they're "the bad guys"? (and in fact, if we're going to be really honest - ARE THEY the bad guys?). The answer to both may well be Probably Not. They have their own justifications and reasoning just as we all do.
I can't help being somewhat disappointed, but as the odd-one-out, who am I to say what's normal and good?
Golden handcuffs will blind you to the atrocities you are building. Or maybe the company has fully siloed off the teams. Maybe the worker drone doesn't know his/her project is actually being used to create these monsters. Either way, I have been resigned to not work for F(M)ANG companies. The engineering is beautiful, but the use of their work by the business is not.