Great question. I've been dealing with APTs from Russia (and to a lesser extent China) for two years now, and there is nobody out there to help independent sites.

GDPR recently drove us to build our own application level firewall from scratch, which turned up behavior that is never reported by CloudFront or Google Analytics - those services are only hiding the severity of the problem.

Maybe a silly question, but what is an APT?

Advanced Persistent Threat - a hacking group that finds an open door (or creates on by phishing), gains access, spreads little backdoors and malicious code into a bunch of machines, any of which, if removed, can be re-created by the others. Not easy to detect, very hard to get rid of once infected. Many groups have differences in the tools they use, which enables those in the know to say "Oh, that's APT X" when they look at an infection.

You may have heard of "Fancy Bear". That's one APT out of Russia.

For more information, see https://en.wikipedia.org/wiki/Advanced_persistent_threat

Advanced Persistent Threat

https://attack.mitre.org/groups/ https://en.wikipedia.org/wiki/Advanced_persistent_threat

If you keep getting pwned by APTs and you need to build a WAF then the security of your codebase is, er, poor, to say the least.

I never said they were on my network.

When I see patterns in traffic coming from 45,000 one-off hosts for a month straight it is clear that there is a distributed botnet behind the requests.

When I see a vulnerability scan from an Azure cloud instance seconds after I ban a block of Russian addresses, I can be sure there is coordination.

And don't get me started on the Moldavian Registration Bots. Those are a combination of automated and human-assisted CAPTCHA solvers, and it took me almost a full week of careful observation to weed them out.

These are some of the things my application firewall can detect automatically. Every now and then I see a new pattern, that is all I was trying to say.

This seems like the worst possible reason not to have a WAF.

Right, you got it. Sometimes I feel like we're being attacked by bespoke systems, but it really must be off-the-shelf stuff since our full content is easily licensable. We shouldn't be worth the trouble.

We just weren't getting enough information from Bing or Google Analytics or CloudFlare, and when I developed a realtime activity dashboard, patterns started emerging: distributed web scrapers, registration bots, vulnerability scans, and some of these in tandem (i.e., scans commencing immediately after blocking a block of addresses). And many of these are coming from cloud hosts, Azure being the worst, with Google a close second. This is the type of traffic they don't want you to see, so those respective analytics services just supress it because it would be a negative advertisement if we could actually see what is happening realtime. I compared the numbers - Google was consistently underreporting our traffic by at least 40%, and a lot (not the majority, but enough to be noticible) of that traffic was coming their own hosted servers (not the indexing bots, but the user cloud instances).

CloudFlare implements temporary bans but I needed something permanent for those threats that were recognizable based on their request patterns.

The ARIN squatting is the latest thing I'm seeing - a lot of requests coming from netblocks that are former DoD and RedHat addresses. The publicly available ARIN databases aren't entirely up-to-date and the bad guys know it, some of the checks we depend on have to be taken with a grain of salt.

So far, I've been able to develop business rules to separate out the human activity from the carefully constructed scraper/probe attempts, but I fear that if they get just a bit more sophisticated I may lose that ability.

Wouldn't distributed scrapers be blocking Google Analytics scripts by default? Or are you sending data there server-side?

That is a fair point. It used to be that way, but around 2016 we started noticing bots and scrapers that use that use full webkit implementations to run JavaScript. Those clients should be triggering Google Analytics just like a desktop browser, but it was difficult to make the correlation due to information hiding in the GA dashboard (tuple of IP address, timestamp, resource would be needed, but they do not provide that, so it was impossible to test).

By your own logic China is equally dependent on the West.

No they are not. What do you think why the Belt Road Initiative exists? In my opinion it's mostly to get Africa up to speed, similar to the post WW2 marshall plan, so that African demand can fill the gaps that Europe/Africa leave.

Also, China has massive sums of dollars and euros, they can simply play the "long game", i.e. weather out the storm until Europe/US caves.

Yeah that's not how it works at all thanks to globalisation. About 10 other countries will step in to provide all the cheap widgets in Chinas place about 3 seconds after they 'cut us off'

Which countries have anything even remotely matching the supply chain of Shenzen built up and trained - and with the spare capacity available?

Many companies are moving out of China at the moment, either due to the threat of a Trump-caused trade war or because other countries are even cheaper than China, and it takes them years.

China can weather one year of export blockade to EU/US. No problem. But EU/US can not, not in any way.

War?

China is a nuclear power plus any open US-China/Russia war would immediately extend to South Korea and Taiwan (in case of China) or Eastern Europe (in case of Russia). I highly doubt that even the hardest falcons in the DoD would advocate for war...

Europe is closer to Moscow than ever in history, since Russia breaks in pieces that want more democracy.

All of them have a pretty harsh stand against Russia.

Crimea ( Ukraine) almost made Russia's entire naval fleet useless.

Russia is still getting punished for it economically and Putin has never seen these low approval rates. There are also a lot of protests there.

China thinks they can possess the world, but they are even having severe problems in their own country. We just need to wait, China's plan was smart, but they forgot the will for freedom in every execution/plan they had and are doomed to fail. Sooner or later.

Depends on how you look at things :)

Hawks* not falcons

Correct. Policy Hawks are people who want F-16 Falcons deployed into action.

That would be absurd. Facebook and Twitter are not worth dying for.