DDoS Attack

17 Sep

As many of you may have noticed, we’ve recently been experiencing issues which have resulted in some downtime for some of our customers.  In the spirit of clarity and disclosure, this post is a brief analysis of what happened, what we did to try to fix the problem, and what you can do to help protect your stores from this kind of attack in future.  We’ll try not to get too technical, but that’s kind of inevitable with this sort of thing.

What Happened ?

Firstly, here’s what happened.  Some individual or entity decided to launch what’s called a Distributed Denial of Service (DDoS) attack against one of our IP addresses.  If you’re not sure what a DDoS attack is, the basic idea is to send a massive amount of traffic to an individual target, to put as much strain as possible on their equipment until it is unable to serve any legitimate requests to their site or service.  These are unfortunate incidents which are largely out of the target’s control.  All you can do is react to them.  We’ve experienced these before, but have usually been able to identify the intended target, drop the traffic aimed at that target temporarily, and you guys will have barely noticed that there was an issue, if at all.

This one was a little different though.  This particular type of attack is called a DNS Amplification Attack, where the attacker basically launches a large number of small requests at thousands of badly configured DNS servers, spoofing an IP address (in this case, ours) as the client that made the request.  The result is that the DNS servers send back a huge response to the supposed client, hence the “amplification” part.  The other thing about the attack was that it was aimed at an IP rather than a domain or host name, so we couldn’t filter out the traffic in any way.

As this attack start on the 11th September 2013, we believe it may have been part of a larger attack on financial institutions on the 9/11 anniversary.  We have no real way of knowing who it was targeted at though, it could have been Freewebstore or one of our customers.

How We Responded

To mitigate the attack, we first had to drop any traffic to the IP address that was getting attacked.  There was no way around this, we just couldn’t absorb it and keep things running.  We then started to move the services hosted on that IP to other addresses, and then point our DNS records over to that new address.  This managed to mitigate the attack for the majority of our customers, but there were some that had pointed their DNS records directly at the IP address that was getting attacked – we’ll talk about that shortly.

We were hoping that the attack would subside within a few hours, as they usually do, so when things died down we switched the traffic back on to the attacked IP.  This meant that all of the domains that were directly pointed at that IP started working again, and everybody is happy again right ?  If only things were that simple.  Unfortunately these attackers were very persistent, and more attacks followed in the next few days, so we had to drop the traffic to that IP again.

Unfortunately it’s impossible for us to determine which customers have their DNS configured to point straight at our IP, as a lot of it gets configured by the customers without the intervention of our support team.  As DNS can be quite confusing for non-techies, and the customers with their DNS configured this way should have been in the minority, we decided not to put out an alert to all customers as it would have just created confusion and possibly caused some customers to break working DNS records.  For the most part, we think this was the right call.

How Can I Protect Myself ?

The key to this is DNS.  There are several ways to point your domain name to your store.  The recommended way is to have your URL as http://www.mystorename.com which you configure by creating a DNS CNAME record for “www” and point that at “shop.freewebstore.org”.  The great thing about this is that we have control of the “shop.freewebstore.org” destination, so when the IP it was pointed at came under attack, we could change where that pointed to and all was well again.  The same applies for if you want http://store.mystorename.com or http://shop.mystorename.com, you just need to make sure that “store” or “shop” DNS record is a CNAME record pointed to “shop.freewebstore.org” and this will allow us to ensure that your store experiences the minimum amount of downtime during attacks such as these.

Now, it is desirable for some stores to use http://mystorename.com as their URL.  This is what the @ A record is for in our DNS guides, and that instruction was added due to popular demand.  The reason we didn’t initially include it in the instructions, effectively discouraging it, is because it locks those DNS records to a specific IP.  This @ record is called the Zone Apex or Naked Domain and the DNS “rules” specify that it has to be an A record, and an A record has to be an IP address.

While we understand it is desirable for this to work, we would not recommend that it is your default URL.  You should always ensure that your default URL is one with www’s or some other host name, and that it’s configured with a CNAME.  If your current URL for your store doesn’t use www’s, but you have a www host pointed with DNS, you can fix this by visiting the “Marketing > Domain Name” section of your Freewebstore Control Panel, and detaching, then re-attaching the domain.  This means that a minimal number of requests will fail if the IP changes, only for customers who have manually typed the URL in that way.

Another way to configure the Zone Apex is to use redirection or Web Forwarding in your DNS or Domain Provider’s Control Panel.  Now, we usually discourage using any kind of Web Forwarding, as sometimes it isn’t obvious whether the provider will use Stealth or Masking, which will break the basket system on your store.  If you use web forwarding with your domain, you’ll know if it uses stealth because whenever you change pages on your store, the URL in the address bar will not change at all.  If you’re sure that you can switch stealth off, then what you could do is set a web forwarding rule to forward any requests at http://mystorename.com to http://www.mystorename.com automatically.  This is what we essentially do when you point it at our IP, but at least if you configure it with your domain provider, you’re taking that IP out of the equation.  Just make sure that the URL changes to http://www.mystorename.com in the address bar when you visit the forwarded URL and you should be good to go.

If you would like assistance with configuring your DNS in this way, or would like for us check your settings, please get in touch with our Support Team via the Support section of your FreeWebstore Control Panel, or take a look at our Free DNS Configuration Service in the “Marketing > Domain Name” section.  Either way, our team will be more than happy to help.

What Happens Now ?

Right now, the attacks have died down in strength and longevity, but they are still ongoing.  We’re experimenting with switching the traffic back on, but may need to shut it off again at short notice.  If you’re still experiencing issues with your store, we’d encourage you to get in touch and let us help you to reconfigure things.  As always, we’ll take the opportunity to review our procedures and see if there’s anywhere we can improve.  Sometimes there are things that are just out of our control and can’t be prepared for, even Google has downtime.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: