A lot of smart people have spent a lot of time thinking about this problem.
If they only scrub half of that traffic, you're still getting a massive DDoS attack against your origin servers anyway. And the frustrating thing about buying DDoS mitigation is that you have absolutely no way of knowing how much traffic any DDoS mitigation service can actually scrub of a given attack before it hits you...
If the DDoS mitigation provider is doing their job, they will ingest all the traffic, scrub out the bad and return the good. While obvious, there are things a customer should know when entering into that arrangement:
- Does the provider have the ingress capacity to absorb all the attack traffic?
- Are their scrubbing centres peered with Tier 1 transit providers to reduce carrier congestion?
- Do they have a policy on dropping traffic at certain volumes?
- Do they charge you based on attack volume or clean traffic?
- Do they have rate-limiting in place towards the customer to protect them from high-volume attacks while mitigations are optimized to catch all the attack traffic?
Ensuring your provider has these technical and contractual terms in place will make sure they can actually offer value when under attack.I am curious if the traffic analysis at the scrubbing center is anything more than coarse grain I am assuming it is.
I work for F5 Silverline.
It would be naive to brush off that size of attack as it would stress any DDoS mitigation provider.
We have the capacity to take it (and are adding a lot more in the coming months precisely because of attacks like this and the previous big one that hit Brian Kerbs) but you would have to talk to Sales about what sort of guarantees we would be willing to provide.
That said, at least that would be our problem, since we would have to pay for the bandwidth consumed - not the customer! ;)
> Does scrubbing work with a bunch of randomized source ports and source address? How do you find the "signature" in that to scrub? It was my understanding that the traffic in the Dyn attacks was indistinguishable from legitimate traffic. Can you explain how traffic scrubbing works in such scenarios? Or do you just drop stuff on the floor?
(Answer from one of my more knowledgeable colleagues since I work on supporting the infrastructure rather than DDoS mitigation itself):
Most attack traffic has a signature of its behaviour beyond source IP and port.
For simple flood attacks, connection rate-limiting per source IP is very effective. Typically there are some simple firewall rules that will work too, especially with attacks generated against targets that don’t support the attack vector (e.g. block all GRE traffic coming to your website which is only TCP 80/443).
If the attack is operating at L7, there are additional fingerprinting methods we can use around HTTP headers, intelligent rate-limiters, JavaScript injection to prove an actual human, etc.
">For simple flood attacks, connection rate-limiting per source IP is very effective."
But if a each of the 8 million bots in a hypothetical botnet sends exactly one syn packet then you can't really rate limit per source IP. Or the case of DNS a single UDP datagram which might be indistinguishable from a "legitimate" DNS query datagram.
Also rate limiting is different that what I would think of as "scrubbing" unless I am thinking about scrubbing wrong. Back to my example of the botnet where the command and control tells each node to send a single packet or a small handful of packets towards a destination, theres very limit match a rate limit filter on in that case no?
The single SYN will fail the 3WHS test, so we would drop it. For DNS we can usually fingerprint the attacker and block traffic. Because we are a full proxy we can do packet inspection as well. ;)
There's a fair amount of intelligence already built into our products which we of course make use of (BIG-IP LTM/AFM/ASM, "IP Intelligence").
Eventually it may well come down to the SOC monitoring and adjusting the filters in real time as required - that's life.
This isn't global this is per region/per AS. 1.2Tbs would be 12 100 Gig ethernet ports on a router or 120 10 Gig ethernet ports. There are not many people that have this capacity in a POP or even even in multiple POPs in a region such as North America or Europe. Just the money involved in being able to connect that much transit to your edge is insane. Because not only do you need enough transit provider diversity but you also need multiple chassises populated with enough switching fabric. I doubt there is a CDN or hosting provider that has 1.2T of capacity at any kind of a region level. A Tier 1 provider yes but not a CDN or hosting provider.
There are estimates that the Dyn attack was 1.2Tbps. What provider do you work for that can absorb that?
>"scrub the traffic using using some expensive fancy appliances(*)"
Does scrubbing work with a bunch of randomized source ports and source address? How do you find the "signature" in that to scrub? It was my understanding that the traffic in the Dyn attacks was indistinguishable from legitimate traffic. Can you explain how traffic scrubbing works in such scenarios? Or do you just drop stuff on the floor?