Bhavin's Blog

20 Oct, 2008

Denying a Denial of Service attack

Posted by Bhavin Turakhia

At our size, not a day goes by without being under some DDOS attack on atleast one of our services. They are now an assumed constant. Most of them do not impact us in any manner. Some of them manage to make a dent. Others can result in unscheduled downtimes. Here are a set of tips in no particular order that we have learnt the hard way to improve your infrastructure resilience -

• Monitor your traffic at the network adaptors and switch ports using MRTG and Nagios, and ensure any spikes result in immediate alerts and escalations
• Ensure that your switches, routers and other devices can provide you with packet capture and other pertinent monitoring information
• Since DDOS attacks originate from multiple source IP addresses, typical DDOS mitigators use heuristics to determine bad traffic patterns and block those source IP addresses. This can result in blocking genuine traffic patterns / sources. If instead you can identify the traffic pattern of the DDOS you can block that pattern atyour firewall, as opposed to blocking source IPs. For eg, we have had UDP attacks on our DNS Servers where the UDP packets were of a fixed length. We blocked packets matching that configuration and thwarted the attack
• Last week we got hit by the mother of all DDOSes – a 4.8 GBps sustained attack on one of our deployments. An attack of this size is cannot be thwarted by regular DDOS mitigators or source blocking. The only choice we had was to null-route our destination IP Address. Try and ensure that your service is bound to multiple IP addresses, with a low TTL in your DNS server to allow you to modify your DNS entries rapidly. Many a times DDOSes target a specific set of IPs in which case you can simply give up that IP address and substitute it with another one. Sometimes DDOSes may target a domain name in which case the attack may be visible on all the IP addresses that the domain name resolves to. In this case, if your app-design can allow for a modification of the service URL – it would make it easy to block the DDOS. Otherwise your best bet would be to modify the ip addresses in your DNS for that domain, and hope that the DDOS clients are caching DNS resolution and the attack will not migrate to your new IP addresses. It is also highly recommended that your application have multiple different service access URLs and that you provide different ones to different users. That way a DDOS may not affect all your users.

This is a hurriedly drafted article and there is significantly more knowledge that we have amassed over the years on this subject. If I find some additional time I will likely pen it down in a more structured format on some other day.

Comments

Leave a comment

Name (required)

E-mail (required)

URI

Your Comments

Spam protection by WP Captcha-Free