Evading a DDoS Attack Using mod_evasive

mod_evasive is an Apache module that provides evasive action in the event of an HTTP DDoS attack or brute force attack. It is also a detection and network management tool. Let’s take a quick look at how it can help organisations evade a DDoS attack.

A Denial-of-Service (DoS) attack is one that is initiated by malicious actors to render a service connected to the Internet unavailable for legitimate users. This is typically achieved by flooding the target system with requests that consume all the bandwidth and overload the computing resources, thereby making the serving of requests slow or even entirely impossible.

A DDoS is a distributed type of the Denial-of-Service attack that uses multiple sources to mount an attack. This makes it difficult to stop such attacks by blocking the source IP address, a move that would otherwise work well for a Denial-of-Service attack. At times, the attack may not even target a service directly but, instead, target the auxiliary services such as DNS services, in order to prevent access.

One thing that holds true with any form of DDoS attack is that it not only compromises the user experience of your website, but can also cause financial losses in terms of compute wastage, additional bandwidth costs and administrative overhead to keep the services up and running.

A DDoS requires a network of devices that can be used to launch attacks. A common way of acquiring such networks is by hijacking smart connected devices, such as routers and IoT devices exposed to the Internet and using them to carry out the attacks. These networks are called botnets. It should not come as a surprise that attackers gain access to most of these devices using default passwords. Another reason is the poor security measures implemented on smart devices.

Types of DDoS

The DDoS attacks can be broadly classified into two major categories – application layer attacks and protocol attacks.

Application layer attacks are launched using the application protocols such as HTTP and FTP, and since the OSI network mode puts these services on the seventh layer, they are also referred to as Layer-7 attacks. A DDoS on this layer can cause a two-fold impact on the target services by first flooding the available bandwidth, and second, by causing the server to consume computing resources serving the requests. This could be a simple URL request that is being sent to a Web server from a large number of sources. Legitimate users may then get a slow or no response at all. We will look at using the mod_evasive module for Apache Web Server to mitigate this kind of attack, later in this article.

The protocol layer attacks target the session (Layer 5) and transport (Layer 4) layer defined in the OSI network model. The attacks are typically manifested in ways that exploit the weaknesses in the protocol layer to exhaust the available connection limits. An example of such an attack is when TCP packets are sent to the target with a spoofed source address during the TCP handshake process. A spoofed source address causes the target system to send a response and wait for the next step to take place, which never happens. This causes the TCP layer to exhaust the number of connections available and prevents it from serving requests queued behind the spurious requests.

These attacks could be routed through the botnets. Since each bot in the botnet is a legitimate device, discerning a pattern in the incoming traffic could be next to impossible.

An overview of the mod_evasive module

Apache mod_evasive is a module developed by Jonathan Zdziarski under the GPL v2.0 license,  which permits commercial use as well as distribution, on the condition that you attribute the licence and share the source code of any derivative work. The source code for mod_evasive is available on GitHub at https://github.com/jzdziarski/mod_evasive.

mod_evasive is a security module for Apache and provides you with a way to take evasive actions in case your Web server running  Apache comes under a DDoS or a brute force attack. A brute force attack differs from a DDoS attack in the sense that it is commonly used to attempt logging in to a service using many combinations of user names and passwords. The combinations can run into the thousands.

The common thread between a DDoS attack and brute force attack is the volume of connections hitting your Web servers, and that’s where the mod_evasive module comes into the picture.

It gets instantiated for each listener in Apache and, hence, provides the same scaling capabilities that come with the Apache Web server. The detection is performed by building an internal dynamic hash table consisting of IP addresses and URIs. When a request arrives, it does the following (from the official documentation available at https://github.com/jzdziarski/mod_evasive):

The IP address of the requester is looked up on the temporary blacklist.

The IP address of the requester and the URI are both hashed into a ‘key’.  A lookup is performed in the listener’s internal hash table to determine if the same host has requested this page more than once within the past one second.

The IP address of the requester is hashed into a ‘key’. A lookup is performed in the listener’s internal hash table to determine if the same host has requested more than 50 objects within the past second (from the same child).

If any of the above returns true, the request is blocked and HTTP status code 403 (Forbidden) is sent to the client. In addition, you can also configure this module to run a system command and/or send an email notification when this happens for further actions, like logging a ticket, tweaking a firewall setting by using IPTables, or sending an email to the security teams.

Securing the Apache server using mod_evasive

Let’s jump to some hands-on work with this module. The following steps will show you how to install and configure this module on an Apache 2.x server running on Ubuntu 18.04. Your steps may vary, depending on the distributions and versions you’re using.

2. Install Apache 2.x and configure it to run at system startup:

   sudo apt update
   
   sudo apt install apache2
   
   sudo systemctl status apache2
   
   

3. The default install of Apache 2.x server doesn’t come with mod_evasive. You can install the module using the following command:

sudo apt install libapache2-mod-evasive

You will be prompted to verify the mailing configuration on the host so that the module can send notifications when a DDoS is detected. The installation will also enable the module and you can verify the same, using the following command:

a2query -m | grep evasive

3. The default configuration of mod_evasive comes with a very restrictive configuration. While we will explore the details of the configuration, let’s just see it in action for now:

count=1

while [ $count -le 10 ]

do

  curl -o /dev/null -s -w “%{http_code}\n” http://localhost

  count=$(( $count + 1 ))

done

This little while loop is making a call to the Apache Web server running locally, just for ten times, and displays the HTTP response code as the output. You will most likely see a few ‘HTTP status code 200’ which will be followed by ‘HTTP status code 403’.

Configuring mod_evasive

Apache 2 provides several configuration scripts which are named a2*, such as a2query, a2enmod and more. If you followed the steps above, you have already used them. These scripts primarily work by modifying symlinks inside the /etc/apache2 directory on the host.

When Apache starts up, it looks up the symlinks created in the /etc/apache2/mods-enabled/ directory and loads the module by reading the file names that match the *.load patterns. For mod_evasive, the file will look like what follows:

LoadModule evasive20_module /usr/lib/apache2/modules/mod_evasive20.so

The contents are self-explanatory. You will also see another symlink  in the same directory that points to the mod_evasive configuration called evasive.conf, and the contents of this file will look like what’s shown below:

<IfModule mod_evasive20.c>

    #DOSHashTableSize    3097

    #DOSPageCount        2

    #DOSSiteCount        50

    #DOSPageInterval     1

    #DOSSiteInterval     1

    #DOSBlockingPeriod   10


    #DOSEmailNotify      you@yourdomain.com

    #DOSSystemCommand    “su - someuser -c ‘/sbin/... %s ...’”

    #DOSLogDir           “/var/log/mod_evasive”

</IfModule>

The official documentation (available at https://github.com/jzdziarski/mod_evasive) gives the following details about these parameters.

DOSHashTableSize: The hash table size defines the number of top-level nodes for each child’s hash table. Increasing this number will provide faster performance by decreasing the number of iterations required to get to the record but consume more memory for table space.  You should increase this if you have a busy Web server.  The value you specify will automatically be tiered up to the next prime number in the primes list (see mod_evasive.c for a list of primes used).

DOSPageCount: This is the threshold for the number of requests for the same page (or URI), per page interval.  Once the threshold for that interval has been exceeded, the IP address of the client will be added to the blocking list.

DOSSiteCount: This is the threshold for the total number of requests for any object by the same client on the same listener, per site interval.  Once the threshold for that interval has been exceeded, the IP address of the client will be added to the blocking list.

DOSPageInterval: This is the interval for the page count threshold, which defaults to one-second intervals.

DOSSiteInterval: This is the interval for the site count threshold, which defaults to one-second intervals.

DOSBlockingPeriod: The blocking period is the amount of time (in seconds) that a client will be blocked for, if it has been added to the blocking list.  During this time, all subsequent requests from the client will result in a 403 (Forbidden) error and the timer being reset (e.g., another 10 seconds).  Since the timer is reset for every subsequent request, it is not necessary to have a long blocking period; in the event of a DoS attack, this timer will keep getting reset.

DOSEmailNotify: If this value is set, an email will be sent to the address specified whenever an IP address becomes blacklisted.  A locking mechanism using /tmp prevents continuous emails from being sent.

DOSSystemCommand: If this value is set, the system command specified will be executed whenever an IP address becomes blacklisted.  This is designed to enable system calls to IP filters or other tools.  A locking mechanism using /tmp prevents continuous system calls.  Use %s to denote the IP address of the blacklisted IP.

DOSLogDir: Choose an alternative temp directory. By default, /tmp will be used for the locking mechanism, which opens up some security issues if your system is open to shell users  (http://security.lss.hr/index.php?page=details&ID=LSS-2005-01-01).

In the event you have non-privileged shell users, you will need to create a directory writable only to the user Apache is running as (usually root), and then set this in your httpd.conf.

The parameters for your specific configuration may vary but the default settings are very strict. It blocks requests the moment there are two (DOSPageCount) requests in one (DOSPageInterval) second from the same host. If you ran the commands I have included earlier to test the configuration, you will have seen that we only get two or three cases of ‘HTTP status code 200’ before we start getting ‘HTTP status code 403’. That happens because we hit the threshold in the first few requests.

Those numbers aren’t very practical, and your specific requirements will be different, depending on how busy your website is. You will have to account for how long your pages take to load and how frequently you expect your users to try to reload any page. A DOSPageCount of 50 and DOSPageInterval of 20 seem very generous for the purposes of our article and for testing.

Let’s change those numbers and run the while loop again to see how the behaviour changes. You will have to restart the Apache server with the following command:

sudo service apache2 restart

When you run the while loop, you may have to change the number of iterations to something close to 100 so that you can emulate an attack with the changed mod_evasive configuration:

count=1

while [ $count -le 100 ]

do

  curl -o /dev/null -s -w “%{http_code}\n” http://localhost

  count=$(( $count + 1 ))

done

You may also want to adjust the DOSBlockingInterval for single site configuration, which defaults to 10 seconds before it unblocks the client in question again. If you see a large number of incoming brute force attacks, you may want to increase it to something like 60 seconds to discourage the attackers.

The DOSSiteCount and DOSSiteInterval parameters work like DOSPageCount and DOSPageInterval, but site-wide instead of individual pages, to provide you further flexibility about when they start classifying the incoming traffic as an attack.

The DOSSystemCommand allows you to run commands to initiate further action in case the mod_evasive module detects an attack. You can use this parameter to invoke a script that can perform a firewall rule modification or raise a ticket in your ITSM tool to generate a security incident. This is apart from the notifications that you can send to the email address defined for the DOSEmailNotify parameter.

You can also use DOSWhitelist to whitelist certain IP addresses or ranges that will never be denied responses from your Web server. You can use this parameter multiple times to accommodate multiple IP addresses or ranges.

An example of when this parameter can be used would be a large customer for your SaaS application or a big client that you expect to send heavy traffic to your Web servers.

Augmenting mod_evasive for more security

Busier, larger and mission-critical Web servers will need additional security measures to fend off DDoS attacks, especially with the kind of scale the attackers can engineer with a botnet purchased from the deep corners of the Dark Web. mod_evasive can stand on its own for simpler DDoS attacks but will fall short when attacks do not target the Web server directly or when launched with botnets, as such attacks would either not arrive at the Web server or may escape detection altogether by mod_evasive due to the extremely distributed nature of such attacks.

In such cases, augmenting web server security with host based firewalls, such as IPTables, and network firewalls becomes imperative. You can use the DOSSystemCommand directive to integrate mod_evasive as another signal for these firewalls to strengthen the detection and prevention capabilities. For example, you can use a script to run, which can perform any or all of the following:

• Create an incident for the security team to start investigating the traffic patterns.
• Alter the localhost firewall rules to tighten the host security, such as ‘rate limit’ connections from malicious IP addresses to limit the impact of a DdoS.
• Initiate processes to perform such actions on other services and hosts.

If you are running a Web server that attracts traffic, security needs to be your priority and the mod_evasive module for the Apache Web server is a good tool in your kit. This module works well for the purpose and scales with your Apache Web server; it is meant to be your last line of defence – when the unwanted traffic is hitting your Web host. Please be aware that a DDoS attack is just one of the many types of security threats that you will encounter once you are out there. So a comprehensive network perimeter firewall along with other security practices  should be high on the list of things that you want to implement.