I think what might be needed here is that anyone with enough resources can run their own relay, and PDSes can subscribe to multiple relays and deduplicate certain things.
That is how it's designed, yes!
Bluesky itself is US-based.
Coming from Silicon Valley companies, all the federation stuff aren't sincerely-intended ideas to promote free society values. They're an escape hatch. It's a free-speech zone to divert unwanted activists onto—away from the company and its brand, and away from its users—where they can blow off steam quietly. And it's something their PR can point to to deflect accountability for the awful things they will definitely end up doing, in their uncompromising pursuit of market share. ("Oh, we're not really censoring that; it's only censored on the main corporate instance, which is the only one people use").
Federated social media could be used for very bad purposes and it would be impossible to stop. Consider this situation
https://www.amnesty.org/en/latest/news/2022/09/myanmar-faceb...
in the centralized case we can point to Facebook as a responsible entity. The thing is Facebook later got kicked out of Myanmar not as accountability for those crimes but because they were insufficiently pliant to the government. In the future a country like that might just run its own Mastodon network where it is all genocide all the time.
--
https://www.europarl.europa.eu/RegData/etudes/ATAG/2020/6581...
As you can use their guidelines:
European Union (EU), there are several types of social media content that are considered illegal.
---
* Incitement to Terrorism: Any content that encourages or promotes terrorist acts, violence, or extremism is prohibited.
* Illegal Hate Speech: Social media posts that spread hate based on race, ethnicity, religion, gender, sexual orientation, or other protected characteristics are not allowed.
* Child Sexual Abuse Material: Sharing, distributing, or creating content related to child sexual abuse is strictly illegal.
* Infringements of Intellectual Property Rights: Posting copyrighted material without proper authorization violates intellectual property rights.
* Consumer Protection Violations: Misleading advertisements, scams, or fraudulent content that harms consumers are prohibited.
--
These rules are further strengthened by stricter regulations for four specific types of content, which have been harmonized at the EU level:
* Counter-Terrorism Directive: Addresses terrorist content.
* Child Sexual Abuse and Exploitation Directive: Focuses on combating child sexual abuse material.
* Counter-Racism Framework: Aims to prevent and combat racism and xenophobia.
* Copyright in Digital Single Market Directive: Deals with copyright infringement online1.
---
You could ideally put the EU illegal categories in a drop down like old Slashdot Style - but each mod who selects the same selection from the drop down adds a point to the tag - the post is removed upon some thresholds of the points.
This could be different for each reagion - and a post could be flagged with points for each region... so have a region selection, then the illegal tag list sets to that region. A post maybe could be tagged by multiple regions infractions based on where the mod sits?
Also - you can keep metrics for all posts flagged for what infraction types for - plus you can move them to the "if law enforcement needs this post S3 bucket" -- based on whatever time period the laws require.
Add a moderation channel per country and let clients apply them depending on location/settings. It's naturally not perfect, but as one can just travel to other countries and get their (potentially less restricted) view or even simpler use a VPN, it's as good as basically any other such censorship measurement.
Also, filters which are optional on the user’s part can’t really be counted as moderation.
> There are also "infrastructure takedowns" for illegal content and network abuse, which we execute at the services layer (ie the relay).
My understanding, here, is that any app has the ability to shut down entire accounts from being able to provide content for that app. And my expectation is that states will have laws that say "operators of an app must ensure that they don't provide illegal material" - at least to the extent of CSAM. So you have state motivation for app-runners to moderate illegal content on their app, and you have app-level mechanisms for shutting down content. And while it can still be hosted on whatever relay was hosting it to start with (if that isn't the one that shut down the content), I would be surprised if sharing that content to another relay didn't give away a ton of information that a person doing illegal activities wouldn't necessarily want published. Put more simply: it's unlikely that if I have to shut down some CSAM coming from your relay, that I can't also turn that relay data over to the authorities. Meaning you have a pretty strong incentive to not actually share your CSAM content to any law-abiding apps.
This Japanese flood casually nears or exceeds 50% of content by volume and is a specifically Japanese phenomenon; it does not generalize into Asian cultures or Sinosphere languages[1] - all the others are easily 1% or less or proportional relative to English. It also isn't happening with Facebook but it is with Mastodon.
To be even more frank, everyone should just set up a Japanese containment server with an isolate IdP, and get Yahoo! Japan or NTT Corp fund it, have it monetized via phone contract or something, and that could solve a huge bulk of problems with microblogging moderation. Then everyone could go back to weeding out those few of actual pedophiles, smugglers and casino spams, occasionally reinstating not-too-radical political activists.
Should "outside" users be eligible for signup with such isolate system is a separate problem, but that will be foreign crime anyway and should not bother the main branch operators that cater to the rest of the world that CAN unite.
1: https://bsky.app/profile/jaz.bsky.social/post/3klwzzdbvi22t
2: worse version of [1]: https://images.ctfassets.net/vfkpgemp7ek3/5kYcWXcFUYSLBAEkrS...
So you can have anyone fulfilling these roles. At present there are somewhere around 60 data servers with one large one we run; one aggregator infra; and probably around 10 actively developed clients. We hope to see all of these roles expand over time, but a likely stable future will see about as many aggregator infrastructure as the Web has search engines.
When we say an infrastructure takedown, we mean off the aggregator and the data server we run. This is high impact but not total. The user could migrate to another data server and then use another infra to persist. If we ever fail (on policy, as a business, etc) there is essentially a pathway for people to displace us.
Sorry about going off-topic, I realise it's only tangentially about labelling.
But if it's already working (because you're running an aggregator), there doesn't seem much reason for anyone else to run one? In other words, isn't there a significant risk that there will be fewer aggregators than there are search engines, i.e. just a single one?
It might not be necessary or useful to have multiple aggregators right now. However, I do feel better knowing that if Bluesky the company goes under or changes to a degree where I'm not happy with their decisions, it's possible for someone to stand up a second aggregator.
For that matter, if someone's a free speech absolutist and if they care enough about it to spend the money, they could stand up an aggregator right now with more permissive standards.
Running an "AppView" (an atproto application aggregator/indexer/API server) is generally an order of magnitude more expensive and complicated. But still not beyond the reach of a user coop, non-profit, or small startup.
So these services should all be well within the capabilities of at least multiple companies operating in the atproto ecosystem as it scales.
And in many cases it should make good sense for these companies to do this since it will improve their performance by colocating their services and enable them to do things like schedule their own maintenance windows, etc.
Rather, moderation is a cohesive whole that defines the direction of the community, the same rules and actions apply to everybody.
The model of labelers as we're releasing in this first iteration is, as you say, an additive filtration system. They are "jurisdictionless." We chose this model because Bluesky isn't (presently) segmented into communities like Reddit is, and so we felt this was the right way to introduce things.
That said, along the way we settled on a notion of the "user's personal jurisdiction," meaning essentially that you have certain rights to universally control your own interactions. Blocking is essentially under this umbrella, as are thread gates (who can reply). What's then interesting is that you can enlist others to help run your personal jurisdiction. Blocklists are an example of that which we have now: you can subscribe to blocks created by other people.
This is why I'm interested in integrating labels into threadgates, and also exploring account-wide gates that can get driven by labels. Because then it does enable these labelers to apply uniform rules and actions to those who request it. In a way, it's a kind of dynamic subreddit that fits the social model.
As as user, "personal jurisdiction" is a critical feature to me. If I start a thread, I want to maintain some minimal level of agreeable behavior in the responses associated with my original post.
It's sort of like online newspaper comments sections. Many unmoderated comment sections were once full of 20 disagreeable trolls who drove everyone else away. The bad drives out the good, and trolls accumulate over time. This doesn't even need to be ideological—I knew a semi-famous tech personality that had a handful of personal stalkers who infested every open comment section. Many newspapers fixed this by disabling comments or actually hiring moderators.
I won't post to a service if the average reader of my posts will see a pile of nasty, unpleasant comments immediately following each of my posts.
This is why I mostly prefer blogs, and moderated community forums. Smaller, well-moderated subreddits are great, as are private Discords.
For Bluesky, by default does power lie with the highest authority by or the lowest authority (i.e. the user)?
The US model was originally designed bottom up, with power having to be granted to the higher authority. Admittedly we've effectively abandoned this today.
As a consequence, the personal jurisdiction model as I described is a construct of the application. It can ultimately choose the logic of what gets shown. Therefore it’s top down in this case. What an application can’t do is remove a user’s identity or posts from the internet. And consequently other applications can make other decisions.
All decentralized networks are an expression of a political philosophy.
Also: frankly, if there were someone willing to put a lot of effort into moderating stuff Hacker News doesn't--stuff like people asking questions you can answer in the article or via Google--I would opt into that as I find it wastes my time to see that stuff.
And with delegation of moderation, I think it will start to feel like people voting for rulesets more than a bunch of eclectic chaos; if a lot of people agree about some rule that should exist, you will have to decide when you make a post how many people you are willing to lose in your potential audience.
For example, some rouge user starts posting offensive content about other users, on the brink of breaking the law. Let’s say these other users will mention it to labellers, who this time will refuse to take this content down.
Can you tell me what will happen in such scenario?
They also don't want to see offensive content?
> What if others won't subscribe to my labeler?
Then maybe your labeler is not providing any value? Or you need to market it more?
I see what you did there.
Under US law, the user posting the content is the only one legally responsible for it. Someone hosting the content could be required to take it down by court order or other legal process (like under the DMCA SH provisions) if subject to US jurisdiction. Bluesky is, so they'd have a process same as anyone else in the US, and of course could make their own moderation decisions regardless on top. But the protocol allows 3rd parties to take on any role in the system technically (though certain infra roles sound like they'd be quite expensive to run as a practical matter), so they could be subject to different law. Foreign judgements are not enforceable in the US if they don't meet the same bar of the 1st Amendment a domestic one would have to.
Labellers from the description would never have any legal responsibility in the US, and they do not "take content down", they're only adding speech (meta information, their opinion on what applies to a given post) on top, best-effort. Clients and servers then can use the labels to decide what to show, or not.
At any rate "on the brink of breaking the law" would mean nothing, legally. And "offensive" is not a legal category either. Bluesky or anyone else would be free to take it down anyway, there is zero restriction on them doing whatever they want and on the contrary that itself is protected speech. But they would be equally free to not do so, and if someone believed it actually broke one of the very limited categories of restrictions on free speech and was worth the trouble they'd have to go to court over it.
ETA: Rereading this, that is probably not a very helpful HNy comment, so let me elaborate.
Maybe I am old-fashioned, but one of the things that the internet is most useful for is exploring places and ideas you would otherwise never encounter or consider. And just like taking a wooden ship to reach the North Pole, browsing around the internet comes with significant risk. But given the opportunity for personal growth and development, for change, and so on, those risks might well be worth it.
That model of the internet, as I said, is somewhat old-fashioned. Now, the internet is mostly about entertainment. Bluesky exists to keep eyeballs on phones, just like Tiktok or Instagram or whatever. Sure, Bluesky is slightly more cerebral -- but only slightly.
People are generally not entertained by things that frustrate them (generally -- notable exceptions exist), so I can understand an entertainment company like Bluesky focusing on eliminating frustrations via obsessive focus on content moderation to ensure only entertaining content reaches the user. In that sense, this labeling thing seems really useful, just like movie ratings give consumers a general idea of whether the movie is something appropriate for them.
So in that sense, wonderful for Bluesky! But I think I'll politely decline joining and stick with other platforms with different aims.
https://theintercept.com/2023/06/01/bluesky-owner-twitter-el...
"Liu, who answered some of my questions, did not respond when I asked for the exact language the Bluesky PBLLC used to describe its public benefit mission when incorporating the company. She also didn’t say whether the company would publish its annual benefits reports — reports that PBLLCs are required to create each year, but PBLLCs incorporated in Delaware, where Bluesky was incorporated, are not required to make them public."
"Our mission is to develop and drive large-scale adoption of technologies for open and decentralized public conversation."
One of the things that frustrated me about browsing twitter now is the constant bad faith discussions about everything, one-off potshots that waste pixels and lead nowhere. A moderation tool that sifts that and just gets me to the people that actually know wtf they’re talking about and are engaging honestly would benefit me greatly!
To analogize to real life, I have friends with whom I agree 100% on politics, but I never talk to them about it, because they're annoying when they do it. But I also have friends who disagree with me on political and other issues, but we have wonderful conversations because of the manner in which we disagree.
I don't what Bluesky is doing will actually help with this problem. For one thing, I think it's design as a "feed" basically precludes any solid sort of discussion (compared to an Internet forum). The medium kind of encourages the "one-off potshots" you mentioned, and moderation won't do much to cure it.
I could be wrong though!
As McLuhan didn't say, the content is the message. That is, all those things arrive at you via "content", which is where the moderation lies.
I tend to agree about the utility of Bluesky as a medium for discussion, but that’s not what I want to use it for, so that’s fine by me.
That being said, if this affects one political group more heavy-handedly than another because their political strategy is more inflamed, I’m willing to hear less from them or only hear from the members who can communicate in a sensible manner.
In particular, applying Steam Curator functionality to content moderation is just perfect and I can’t believe I didn’t think of it before.
Do you/anyone reading know of any prior work? The closest I know of is this site, in fact, which is opt-out but not pluggable. Or maybe email spam filters, from the POV of the server admin at least
My understanding of Bluesky is that the equivalent would be a ‘feed’ tracking #trains filtered by some union or intersection of moderation teams. Is that correct?
Now, with this model, I don't see such a scalable structure. You're not really offloading any work to moderation, and also, all mods will be working on all of the content. No subreddit-like boundaries to reduce the overlaps. I know, mods can only work on certain feeds, but, feeds overlap too.
It's also impossible to scale up mod power with this model when it's needded: For example, Reddit mods can temporarily lock posts for comments, lock subreddit, quarantine subreddit to deal with varying degrees of moderation demand. It's impossible to have that here because there can't be a single authority to control the content flow.
How do you plan to address these scalability and efficiency issues?
Not necessarily, that's up to the moderator.
Today, I subscribe to the #LawSky and AppellateSky feeds because I am interested in legal issues. Sometimes these feeds have irrelevant material: either posts who happened to use the "" emoji for some non-legal reason or just people chatting about their political opinions on some legal case.
Someone could offer to label JUST the posts in these feeds with a "NotLegalTopic" tag and I would find that filter quite useful.
All that happens is mods just lock any post with any hint of a problem. It's become or rather started out as being ridiculous. They just lock instead of moderate.
By design Reddit's moderation model is a tool incentivizing unhealthy, one sided conversations and echo chamber-y communities. Reddit moderators have to go the extra mile to avoid those.
I think everyone at some stage has been burnt by top-down moderation (e.g., overzealous mods, brigading, account suspensions, subreddit shutdowns, etc.) and generally everyone finds it lacking because what's sensitive to one person, might be interesting to another. Community driven moderation liberalizes this model and allows people to live in whatever bubble they want to (or none at all). This kind of nit-picky moderation can be offloaded in this way, but it doesn't obviate top-down moderation completely (e.g., illegal content, incitement to violence, disinformation, etc.) Though a scoring system could be used for useful labellers, and global labels could be automated according to consensus (e.g., many high-rated labellers signalling disinformation on particular posts)
Not in a 'I can ban these moderators from moderating my instance' way. I need a metamoderatorion mechanism. I need to see how good moderators are to establish trust and when a moderator is taken over by a hostile actor I need to see its score tank.
Do you have something like this on the roadmap?
If I’ve got that right, then a client could be created that, e.g., displays labels from different moderators rather than filter the content. In fact, I’d guess most clients will have that mode.
Then I need users who (hopefully) vote on/rate/report labels, which is its own problem.
But "lots of reviewers" could be tough. Look at the "Spider Shield" example: if Spider Shield is going to block 95% of spider images, they're going to have to look at 95% of the content that I see, before I see it. This is a big ask if the people doing the labeling hate spiders! (Someone who values a clean feed might want to have a time-delayed feed)
It seems also that the labels themselves would become a thing for people to argue about, particularly if they get attached at the 50% point of the visibility of a post as opposed the first or last 2%.
Something based on machine learning is a more realistic strategy in 2024. Today anti-spiders could make a pretty good anti-spider model with 5000 or spider images. The tools would look a bit like what Bluesky is offering but instead of attaching public tags to images, you would publish a model. You could use standardized embeddings for images and text and let people publish classical ML models out of a library, I am looking at one of my old recommender models right now, it is 1kb serialized, a better model might be 5kb. Maybe every two years they update the embeddings and you retrain.
We had all this on slashdot before quite a few folks here were born.
And what if I don't like your moderation? Can it be overruled, or is this just a system for people who want stricter moderation, not lighter?
A partial answer* is that the Bluesky moderation enforces their community standards, so if you don't like that, then the platform may not be for you.
* - because, yes, this does still have a single entity at fundamental control. But I presume their focus is on the basic threshold (ie legality under US law) of content.
This article is about how different hosting instances can customize the moderation which is offered, and how the user can choose among the offered moderation settings. The whole point is to allow different moderation implementations, because different communities may have different needs.
The application client sets its labelers and filters entirely client side. Our client hardcodes our labeler then lets users choose more. Other clients could choose otherwise.
The infra layer handles illegal content takedowns. We do that, and we try to keep that minimally scoped. Other people can run their own infra which is outside of our control and with its own infra moderation
> another client can choose a different primary Labeler
So you can overrule the moderation, but not if you use the official client.
From the article:
> No single company can get online safety right for every country, culture, and community in the world.
From this post:
> There are also "infrastructure takedowns" for illegal content and network abuse, which we execute at the services layer (ie the relay).
If there's really no point in running relays other than to keep the network online, and running relays is expensive and hard work that can't really be funded by individuals, then it seems like most likely there will be one relay forever. If that turns out to be true, then it seems like we really are stuck with one set of views on morality and legality. This is hardly a theoretical concern when it comes to the Japanese users flooding Bluesky largely out of dissatisfaction with Twitter's moderation of 'obscene' artworks.
I certainly am not saying that server operators should feel obliged to content they do not like, especially if they believe it is illegal or immoral. After all, a huge draw of the Fediverse is the fact that you get to choose, right? Sure, personally I think all obscenity law is weapons-grade bullshit regardless of how despicable the subject matter may be, but also, server operators shouldn't feel pressure to compromise their ideals, attract a crowd of people they simply don't like, or (of course) run the risk of breaking the law in their jurisdiction, so what happens on the Fediverse seems like it is the right way for things to go, even if it is harmful to the federation in the short term.
But that's kind of the double-edged sword. You either have centralization where someone decrees "the ultimate line" or you don't. With Bluesky, there's a possibility that it will wind up being decentralized properly, but it could wind up being defacto centralized even if they uphold their promises and I think that strongly devalues the benefits of decentralization where they count most. Today, there is in fact one company that holds the line, and it's unclear if that's going to meaningfully change.
There are some aspects of AT proto and Bluesky that I think are extremely cool: an example is identity, identity in AT proto is MUCH better than it is in ActivityPub right now. However I'm not surprised they are not going to acknowledge this problem. Just know that they are 100% aware of it, and my opinion is that they really do want to find an answer that won't piss everyone off, but they also probably want to avoid the perception that Bluesky is a haven for degenerates, especially early on when there are less network effects and they desperately need to appear "better" than what is already out there. Unfortunately, I can only conclude that their strategy is most likely the best one for the future of their network, but it still rubs me the wrong way.
IMO, the point is, all of this architecture and the headache of decentralizing it is a waste of time if the most critical part for actually ensuring user sovereignty on a network winds up being centralized anyways. And based on the replies so far, what I'm getting is that nobody at Bluesky has a clear idea why anyone else would ever run a relay other than for the good of society. That's nice, but of the people running relays, none of the other entities that might run relays have the same level of stakes in it as they do. Bluesky is a company that presumably needs to eventually make money to stay afloat, but they basically built out the entire protocol, will run the app, and all of that jazz. They are the face of this. The users that are on Bluesky right now are by and large going through their infrastructure; presumably millions of users. Why in the world would someone else run a relay in this condition? You'd essentially be doing it to mostly support a network that is vastly controlled by one single entity, and there is no obvious reason why it will stop being vastly controlled by one single entity, too. It is a problem that the vast majority of users are on a small number of Mastodon servers, but with Bluesky there's little reason to believe right now that a vast majority of users ever won't be using Bluesky's infrastructure. If we're going to compare it to e-mail, then this is like if the only e-mail provider you could use for years was just GMail, and then slowly other providers were allowed to get whitelisted in. It is nice that the design of the network is so amenable to the potential of having it not be controlled by one entity, but there's the rub, I don't think they actually have any incentive to actually make serious efforts to fixing that problem. I don't know if they would even with substantial pressure from users to do so. Right now their messaging is basically "There might be a few entities that would run relays" suggesting that it is basically out of reach for people not at the scale of Google.
One could easily misread this thread to be about fringe content or more charitably, censorship, but I think the most accurate way to sum up my concerns is this: I don't think Bluesky's own incentives, nor the incentives that they are laying out with this design, are particularly fantastic for the health of the network they are building. It's not impossible that things could wind up working out anyways, but I worry that this is not actually going to be a sustainable system and that if there isn't substantially more thought put into this entire operation, it may basically wind up being just a very elaborate and overly complicated way to re-implement the exact same problems that centralization has.
That's not to say I believe they are working in bad faith or that they don't care or anything like that. Like I said, I assume they are actively trying to figure out how to make everything work without pissing everybody off. And I'd love to be highly optimistic and say that it will all work out, but I have my doubts. The silence is deafening.
I fully agree with this but I'm giving Bluesky a lot of benefit of the doubt because I think Mastodon has even worse incentives and I can't come up with anything better. (I can come up with different tradeoffs but they always leave somebody pissed off.)
Getting back to the Japan problem, I think "content we care about is blocked on the main relay" should be enough incentive to run an alternate relay and maybe a patched client. Obviously there's a bootstrapping issue there but you could say the same about Bluesky vs. Mastodon and yet here we are millions of users later.
Let’s say we ask a question to a politician and they ignore it.
Can we label the question as unanswered, so clients will remind users the question is unanswered?
It's broadly a system for publishing metadata on posts called "Labels". Application clients specify which labeling services they want to use in request headers. Those labels get attached to the responses, where they can then be interpreted by client.
This is an open system. Clients can choose which labelers they use, and while the Bluesky client hardcodes the Bluesky moderation another client can choose a different primary Labeler. Users can then add their community labelers, which I describe below. We aim to do the majority of our moderation at that layer. There are also "infrastructure takedowns" for illegal content and network abuse, which we execute at the services layer (ie the relay).
Within the app this looks like special accounts you can subscribe to in order to get additional filters. The labels can be neutral or negative, which means they can also essentially function as user badges. Over time we'll continue to extend the system to support richer metadata and more behaviors, which can be used to do things like community notes or label-driven reply gates.