Pretty sure they could get rid of all the bots if

a) it was a non profit and

b) people didn’t cry “censorship!” every time they do anything to curb abuse and

c) it would be ok to deactivate a % of real users by accident (these researchers can’t tell who’s a bot vs who’s an actual user _for sure_)

It’s not an easy problem.

The people complaining about censorship aren’t concerned about the removal of bots. They’re concerned about a pattern of behavior from twitter whereby they shadowban, remove, and editorialize viewpoints that don’t agree with the Valley’s ideology.

Alternatively, I wouldn't be surprised if a lot of the censorship complaints came from bots or people deploying bots that aren't happy they're being caught.

Good point and I agree, but perhaps the accusation that it’s a corporate mandate is unfair, when it’s just a lot of rogue actors.

None of these claims is real. “shadowbanning” doesnt exist, and that has been proven many times over. They also don’t remove any content that doesn’t infringe the TOS. This “valley ideology” nonsense is pure propaganda.

> None of these claims is real.

Dude, you can’t even form a grammatically correct sentence.

You should go back to Blind or 4chan. Better places for that kind of discourse than hackernews.

Your original comment added nothing of value. Everyone on hn is now dumber for having read it. I award you no points.

"The Valley's ideology" is to drag your feet, do as little as humanly possible, and to make vapid appeals to freedom of speech rather than do anything about rampant, massive-scale abuse of their services causing real and actual harm to people and society in general.

I know this isn't what you intended to say but it what your statement implies is that everyone who disagrees is uncaring, idiotic, or evil. It might do you well to spend some time finding people to respect on the other side and listen to why they advocate for things you find so distasteful.

I am trying to follow this supposed implication but I don't see the jump from "the valley" to "everyone who disagrees" and also from "dragging feet" and "making vapid appeals" to "idiotic and evil".

I will agree it's a bit of a stretch and I pattern matched to other arguments I have seen but "massive-scale abuse" and "real and actual harm" is language that presupposes a negative moral judgement about what is happening.

You think it is worth debating whether massive troll farms spreading misinformation and stoking hatred and division on a nation-state scale is good or bad?

I wonder if there's a disincentive for social networks to purge accounts like that due to MAU being a core metric in their financial reporting.

How many real people is it OK to squash when squashing bots? How confident can you be when banning accounts?

It’s a lot harder problem then it might sound on the surface. And realize if you find bots 100%, they will change for round 2 and the game starts over.

Counter point (that I'm not sure I believe in): if your behavior is so similar to that of a bot, maybe Twitter would prefer you not on their platform.

(Assuming Twitter was actually incentivized to be anti-bot/bot-like behavior.)

Maybe they don't need to ban accounts, but instead put up a "bot check", sort of like their new editorial process? Something along the lines of "This account has been identified as likely being a bot. Read more." To remedy a false positive, the human could provide Twitter with a an official ID to remove the warning.

Then you get hate from the hn crowd about how they're violating everyone's privacy by deanonymizing them

Worse, Twitter can't be trusted to keep the IDs safe. They've suffered many data breaches in the past.

And it wouldn't even stop the bots, because Twitter isn't limited to Americans, so bot makers could easily send fake official IDs from someplace Twitter can't check. Or send stolen offical IDs from other data breaches.

I bet there are some ThisIDDoesNotExist websites that bots use to autogenerate vast majority of fake ids, probably by using ThisPersonDoesNotExist website.

You can generate realistc photos of IDs from countries that Twitter cannot positively verify those bc in USA there are ways to check authentity of your document. But IDs from Russia or Ukraine - no way.

> To remedy a false positive, the human could provide Twitter with a an official ID to remove the warning.

That would work only against casual bots. The big nation-sponsored botnets won't have issues with providing genuine government issued identities for bots. Which kind of defeats the whole purpose and wastes a lot of resources twitter would like to use elsewhere.

They already do that. Fun fact, I was never able to make and use a twitter account even after verifying it with my phone and email. Apparently I was a bot from the start, before even posting anything.

It's twitter, not people's healthcare.

Just put in a super basic unlock process that is costly to game for "non-real" people

> Just put in a super basic arbitration process that is costly to game for "non-real" people

I think your "just" is hand-waving away a lot of complexity. A person deployed these bots, and if a bot account gets put into an arbitration process, that person can go through the process themselves to pass the arbitration, then set the bot running again.

> that person can go through the process themselves to pass the arbitration, then set the bot running again.

That is not scalable, which is exactly the point.

It's far more scalable than the arbitration process itself. Arbitration will never win such an arms race.

All this bot stuff/millions of fake accounts amazes me.

Whenever I sign up at Twitter (entirely because some companies do support there) the account is flagged within a few minutes for "suspicious activity", such as making a coffee and coming back to the PC or searching for the company I want to contact.

Possibly because of your IP? Any ideas why that is happening?

I suspect it's because I don't give a phone number. I have no idea why they allow signups without a number though, if that's the case.

Allowing signups without phone numbers serves as a honeypot.

Well, you can put catchas in case you doubt there is a bot activity.

They also have so many phone number and email they can use that to do additional checks.

They you can use the network effect: bots usually have something in commons: topics, likes, ip, followers...

If you have millions at your disposal, something can be done.

They are not, so I'm more encline to assume there is benefit for them to avoid doing so.

Would it take millions?

At twitter scale, doing anything is costly.

I actually am trending towards the view that once a social network reaches a certain size, all users must register their real identities with the network owner to maintain their accounts.

This information would not need to be stored, it would just be needed to verify an account as belonging to a real person.

Once you add things like clickstream, email, ip address and user agent to the data used, it should be very easy to pick set with negligible false positives.

And you assume a company as big as Twitter doesn’t do all that because...?

There are a lot of statistics that can be analysed to indicate a bot. If you start with extreme cases and catch 'real' people it might be for the best.

Think about an account that is active 24/7 every day all year, or one that fires off multiple actions per second for long periods of time.

Beyond that, having a limited vocabulary, repetitive phrases, consistently accessing the same thing every few minutes, etc. are all low hanging fruit.

What are the technical hurdles to these bots? Is API access required?

No bots could use webpages or apps just like people

Many, many “bots” are just people paid to tweet

That's not a problem. If a person is indistinguishable from a bad/rogue/misbehaving bot, it's much better for everyone to ban them.

You’d ban half of Twitter. It’s pretty much impossible to differentiate an actual “person typing like a bot” from a legitimate user that, say, likes to tweet memes a lot. People that think this is an easy problem just don’t have the slightest idea of what it entails.

My point is it's an easy problem once you decide that you don't care about false positives, ie temporarily banning "legitimate" users whose behaviour is indistinguishable from that of bots.

Arguably, those "legitimate" users don't bring anything of value to the platform or society as a whole and should rightfully be shunned.

Most online forums or chats have automated rules whereby posting too fast or too often will get you slowed down or banned. Sometimes legitimate users get caught by the filters, no big deal. Twitter could make their existing rules much more stringent.

Sacrificing a few genuine accounts to save the sanity of millions and protect democracy is not much of an issue. The question is how does one revive such accounts without too much hassle while not reviving bots.

Easy: solve 5 captchas (all known captchas of course, not 2 knowns and 3 unknowns) a day for 2 days and you get authorized again. Next violation, solve 5 captchas per day for a week. Next one, 5 per day for a month.

That could work indeed. Annoying but necessary. Tho what does one do about real bots - people in spam farms that would manually login, post and logout.

Lots of those studies pick a few criteria to label something "probably bot", with little verification that the mechanism is actually accurate. Which still makes for big headlines, but doesn't tell you if it would be effective and fair treatment if actually used to ban users.

(I'll add the link if I can find it, but a recent example pretty much boiled down to "actually, these accounts giving many repeated identical replies have totally normal timelines otherwise. And it turns out, many identical replies in short time shows someone knows about copy-and-paste, not that they're a bot")

I’ve worked at a global-scale platform for user-generated content doing anti-abuse and anti-spam tooling. In my experience those researchers do exist in-house and there was an imperative to remove abusive and spammy accounts.

The difference in my perception is that the presence of the bots and bad actors on Twitter that come to the attention of reporting like this increases engagement and views, and thus top line revenue.

This isn’t saying “bots count as views, so we get more ad dollars”. It’s that bots and bad actors promote topics and conversations that bring more real users to the platform, and increase the session duration for new and existing users.

I would imagine that Twitter sees spam bots and purveyors of illegal content as unwelcome and probably has an engineering team that dispatches those accounts quickly. But whether deliberately or unconsciously, they probably don’t apply the same rigor to accounts that break the TOS but manage to drive the top line up.

I’d love to hear from an engineer from Twitter who works in this space.

Its pretty crazy that you assumed the research was factual just because it was in NPR / marketwatch.

As long as they are a public company held to unattainable growth metrics, they will allow Bots and Charlatans. Its good for business as they say...

I see these "x number of bots found on Twitter" reports every so often. What are the most common methods used to determine if an account is a bot or not?

It's pretty crazy that Twitter obviously knows this, and hasn't said anything.

Twitter's primary metric is "Monthly Active Users" which is driven by a) tweeting and b) engagement (aka activity) on those tweets.

It's not in their best interest to wipe out the bots at any large scale. Sure, the occasional cleanup or purge as a token effort or to get the most egregious stuff but remove them all? Nope.