Many "bots" though are actually giant networks of humans being orchestrated and they share things that are compelling to a subset of real users (e.g. Hillary did this bad thing... lock her up), so now real users are retweeting and muddying any signal that was there to detect a bot network. It's an incredibly challenging problem at scale and you basically only see the < 1% that Twitter missed.
They've got a pretty sizable team that works on it so hopefully in a few years they'll have solved it much like Gmail has mostly solved spam.
Gmail has not solved spam. Gmail and Microsoft dump so much legitimate e-mail that it's basically made e-mail totally unusable:
http://penguindreams.org/blog/how-google-and-microsoft-made-...
I run a small SaaS and it sends out quite a lot of notification emails to customers (they are paying me for these emails, they aren’t spam or newsletters). I was originally using SendGrid but their deliverability rates were pretty bad and I was about to hit a ceiling for my plan, which would meant the monthly cost would more than double, which was unaffordable for me.
I decided to try setting up my own mail server. Everyone said don’t do it, you emails won’t be delivered, it’s too much work, etc. I setup a VPS with OVH as I’ve heard their policies regarding email spam are pretty strict (I contacted them before to clarify - if they find out you are spamming they cut your off ASAP to protect their other customers). The IP I got was actually on one blacklist, but I got that resolved in a few days.
The deliverability rate was better than SendGrid and it was costing me 1/10th of the price. One thing that I would assume helped is the domain had already been in use for a few years, and I setup proper DKIM and SPF records.
The server has been running for a year and a half now, and the only bounces I get are due to invalid or suspended email accounts. Most of my customers are small to medium sized businesses, a lot of which run their own Exchange servers - with SendGrid those were usually the ones who were bouncing my mail due to being on a blacklist. Now admittedly just being accepted doesn’t mean it is ending up in a users inbox, but I haven’t had any complaints from paying customers.
Any non-Google, non-Microsoft mailers will be blocked, lest users be asked to renew their VIAGRA prescription from a Canadian pharmacy, or a non-compliant political message be received. It's for your own good, citizen!
Thank you, wise and great Google! Thank you, wise and great Microsoft! I wouldn't know how to use this free, non-controlled computer network without your benevolent work!
unsnark: Seriously, the way that Google et al have captured "anti-spam" mechanisms and converted them into "pay a big company for your mail" mechanisms is extremely sad, and speaks to the true forces at work behind all the warm and fuzzies about an "open web".
If it's down to a few big senders and the rest is rejected, they might as well get rid of SMTP, close it off, and just talk a proprietary protocol among each other.
Of course we have configured SPF, DKIM and a proper reverse dns lookup on both ipv4 and IPv6, but that’s just being a good email-citizen anyways.
https://gimletmedia.com/episode/112-the-prophet/
After Andrea is attacked by a stranger in Mexico City, she just wants to figure out who the guy was. Investigating this question drops her right into the middle of one of Mexico’s biggest conspiracies.
These things are much easier to orchestrate than bots, especially if your specialization is not CS but scamming / manipulating opinions, and in many cases, especially with scale, human labor can actually be more expensive.
Of course, subsidizing the labor to impoverished countries definitely helps; these networks tend to lack the sophistication used in more sensitive political or industrial topics.
Sometimes for national or global campaigns you'll find a mix of both cheap labor, a small team of experienced astroturfers with a good grip on their persona acquisition and management, and bot networks, each targeting a different set of demographics. If your target demo is sufficiently ignorant or radicalized in their beliefs, it doesn't take much to convince them and bots will do just fine.
They're not doing it because they like the big follower counts themselves. They're in on the con.
Also, bot makers don't care about fooling the New York Times, they care about fooling Twitter. If Twitter applied this analysis, the bot makers would adapt to it. It only looks like it works until you start to use it.
The problem with the arms-race argument here is that it implies that just because an arms race is inevitable that the only acceptable course of action is total capitulation. Twitter is bigger than any of the botters, and if it wanted it could raise the cost of entry of botting high enough that nobody would be able to afford the bots in any quantity large enough to make a difference. They just simply don't want to.
(As for false positives, I've seen plenty of people whose accounts were suspended for reasons unclear to them; Twitter seems to care very little whether its captive userbase is pissed off or not.)
How would they do this without triggering enough false positives to destroy their product?
I mean seriously, if you are capable of doing this that well, you could probably walk into a job paying >>$750k/yr at any of the large tech companies, because they'd be falling all over themselves trying to get at your magic.
This implies that Twitter has demonstrated that they care about practicing discretion when suspending accounts, and that they care what 99% of their (human) userbase thinks, and that Twitter's product is Twitter accounts. But they haven't; they don't; and it's not. The state of being pissed off at Twitter is the perpetual existence of all Twitter power users. Unless you're a celebrity, Twitter doesn't care what you say about Twitter. And the only thing important to Twitter's business model is keeping celebrities on Twitter. The reason that they don't give a damn about cracking down on bots isn't because they don't have enough $750k engineers, it's because dealing a blow to the egos of their Influencers by banning their botted followers would threaten their bottom line, both in terms of outrage from the 1% of Twitter accounts that matter and in terms of depressing their wildly inflated metrics.
It’s easy to see how much harder Facebook and Google accounts are to create bots for than Twitter. Twitter doesn’t care.
- captcha that isn't hard for humans is often easy to automate
- captcha that is hard to automate frustrates a lot of humans too
- and to add insult to injury you can "automate" the hard ones too by paying like $1 per 1000 captchas or something (think Amazon Turk like stuff)
This really does seem to be the case. And I find it incredibly distasteful that they were used by DJT in his campaign and continues to be used. It was rarely a place where politicians posted anything, and now it seems to be the first place where any of them do.
Twitter was founded in 2006 and Barack Obama joined in 2007. Today he has 15.4k tweets. Seems like a lot to me.
It might be relatively easy for humans to catch clusters of fake followers, but that doesn’t scale. If you try to create heuristics to catch the fake folllowers, the adeversaries will try to adapt. If you try to learn rules, you can update your heuristics faster, but you need trading data, your adversaries could learn too, and adversaries could try to do data poisoning attacks.
It seems like a really tricky situation for Facebook and Twitter.
O.P. asked for alternative explanations and people are trying to give these companies the benefit of the doubt. There are actually some much less generous explanations, such as that they are knowingly in the pay of forces adversarial to US interests.
Spammers, ultimately, act like spammers, and bots like bots. Unlike legitimate accounts, which typically either a) follow a small number of related accounts or b) are fairly-well-known high-profile accounts (and yes, pseudonymous accounts may be high-profile), bots either need to act within specific networks or be very indiscriminate in whom they follow / link with. And they're tremendously active.
Whilst Google+ is not Facebook, Stone Temple Consulting ran an analysis of 500,000 profiles in 2015 to see what posting activity looked like.[1] I'm fairly familiar with the methods used as I'd pioneered most of them on a smaller, 50k sample.
Looking at public posts only (not private posts, and not comments), there are roughly 53,000 G+ users posting 100+ times monthly, and 106,000 posting 50+ times monthly.
From an independent analysis, I've estimated that Facebook traffic is, very roughly, 10x greater than G+ (as of August, 2015). So you might bump these values up by an order of magnitude to get Facebook-level scales of operation. Twitter is actually far closer to G+ in scale.[2]
That is, the search space for bots is roughly 500k - 1m accounts of major active players. And it should be possible, even with only computer-assisted human intervention, to make a pretty good cut through that space.
So the problem is not all that intractable.
________________________________
Notes:
1. https://www.stonetemple.com/real-numbers-for-the-activity-on... STC's Eric Enge credits me on that page.
2. https://www.reddit.com/r/dredmorbius/comments/3hp41w/trackin...
I jumped down the rabbit hole and set up my own mail server about a year ago. With postfix configured to look at the above I get zero spam. Looking at logs, I do get lots of connection attempts from random IPs in China and such, but my postfix unilaterally rejects anyone that doesn't have good DNS settings.
>people are trying to give these companies the benefit of the doubt.
I think that's a good thing; we're all too susceptible to cynicism. Being charitable is perfectly fair (and ideal) when dealing with people who haven't demonstrated a propensity to deceive (or operating with malicious intent).
I usually hate those scroll-responsive animated web pages, but the scrolling illustrations and data visualizations in this article were particularly well done and pretty amazing. Not just pretty (the initial face sequence) and clever (the scrolling iPhone) but also actually useful and relevant (like the subscriber graph data visualizations expanding over time). I would love to know more about the tools they used to make those too.
A plea to NYT from a subscriber: Please share some of that great software you've developed, and publish it on your github account!!!
We'd like to eventually open source some of the stuff we built to do the analysis as well, though it depends on time and priorities.
https://twitter.com/Rich_Harris/status/956961714578317312
https://github.com/sveltejs/svelte/pull/1137
It seems like Svelte is worth taking a good look at!
I've been shopping around for a "magical disappearing UI framework" to re-implement my old jQuery pie menu component. I was considering Polymer because it was minimal and followed the latest web standards for components, shadow dom, events, etc. How does Svelte compare and interoperate with Polymer?
http://www.donhopkins.com/mediawiki/index.php/JQuery_Pie_Men...
Thus NYT can find some bots that Twitter missed, especially by using labor intensive methods that don't scale to all of Twitter, but there are plenty that NYT missed too. Their methods aren't necessarily better than Twitter's, just different (and likely much more expensive). Note also that the NYT people don't have to care about their false positive rate and don't have to adapt their methods to adversaries since this is a one time analysis.
The moment you start to treat cancer is the moment you realize that identifying and treating the problem are leagues apart. Cancer mutates, cancer squirms, cancer goes incognito until the storm has blown over. By attempting to treat cancer, we’ve made it our adversary and it now will fight against our treatment. UV treatments cause mutations that the cancer might use to avoid further treatment, some drugs target specific causes of cancer which the cancer will then change to, and some treatments just kill the patient (Just as Twitter can’t just ban everyone). There is a balance that these large tech companies need to find just as a physician needs to determine the correct path.
I know this is a very optimistic view of the problem, but it aligns with my experience working, generally, in this space at similar scale.
Ultimately: you monitor direct and indirect effects, offer an appeals and communications process, and try to sort out who is and is not participating in good faith.
You might target suspect accounts for increased validation -- Captcha or other elements, or simply logging them out more frequently. Changing programmatic elements such that they cannot be scripted as easily (this is one of the primary credible arguments I've seen against APIs for large public services).
(Craigslist, and several other services, have applied what's effectively a service-degredation level to accounts suspected of malicious activity. It's not a hard fail, and it's possible for a real human to get past the hassle, but it slows down attackers considerably.)
Because it takes a lot of manpower to do so on a wide scale and almost always raises the question of fairness - the NYT (and others) focus on the egregious examples, but if Twitter would do the same (and kill the followers) their customers (the big celebs) would complain, and when they say "okay, bye Twitter", the normal users who want to follow them would disappear... thus reducing the popularity of Twitter for advertisers. The bots don't count as advertisement recipients, so Twitter doesn't have advertisers complaining about fraud from these fake followers.
In addition, the existing (moderation) manpower is focused on keeping the system being overrun by actual, human-powered Nazi and other troll accounts - fake followers are waaaay on the bottom of the priority list. They don't generate bad press headlines about shitstorms or harassment.
Leadership funds teams that generate positive ROI. A successful bot-hunting team will likely decrease revenue... although kicking out bots makes the platform more sustainable.
It's not just social media. The same question kicks around retail sites like Amazon - why are external researchers better at identifying fraudulent reviews? Because review factories propping up shoddy Chinese imports are very profitable for Amazon. These days customer obsession is just a "nice to have".
The methods described in this article would be corporate espionage. I don't know what the legal implications would be, but ethically, I'm personally a bit frightened by the idea of a company as large and in possession of so much personal information as Twitter getting into the espionage business.
A Twitter/Facebook/whatever client plugin that flags or hides bots seems feasible.
It's even in a certain business interest to track bot traffic for more accurate click through tracking so perhaps there's even a market need for this.
Or....this has nothing to do with talent and more to do with market dynamics - they lack any monetary incentive to change.
What is all the hubbub about machine learning, and things like neural networks, if they aren't being actively employed by the tech giants?
There's only a couple possible scenarios I can come up with for why this continues to occur:
1) The best and the brightest actually don't work for any of these companies; they're just constantly trying to catch up to teams of developers more highly skilled than them. They are bright, and decent, but ultimately mostly average.
2) The developers in these companies are on par with those who can graph and identify these networks of malicious and fake content makers, but they just don't care because # == $.
3) The companies actually have their head so far in the sand that they don't have the technology, or the resources, to combat this.
I find it hard to believe that people outside of the ecosystem of these companies have more skill, knowledge, or capabilities (1). It also pains me to think that hundreds of millions or billions of dollars simply cannot run a company that can combat this (3).
So that seems to leave complacency (2), which is disturbing and definitely should send up more red flags about whether we should be giving these networks any attention at all.
I'd love to hear other possibilities or maybe more information to back up any of the other scenarios.