Best Practices in Network Security Monitoring

Network administrators usually rely on generic and built-in monitoring tools for network security. Ideally, the network infrastructure is supposed to have carefully designed strategies to scale up monitoring tools and techniques as the network grows, over time. Without this, there can be network performance challenges, downtimes due to failures, and most importantly, penetration attacks. These can lead to monetary losses as well as loss of reputation. Thus, there is a need for best practices to monitor network infrastructure in an agile manner.

Network monitoring challenges

For starters, each of the seven layers of the OSI networking model has its own responsibilities, which call for separate methods of monitoring and security for each layer. Network monitoring is seemingly simple — but in reality, it’s a very complex process. Mixing traditional network monitoring with security monitoring further complicates things from the design perspective, for network architects, network operations teams, and the systems administrators who manage it.

While there are multiple challenges in network monitoring, the most important one is the vast amount of data gathered by the monitoring tool, and the amount of time required to assimilate the information and apply intelligence to it, in order to achieve actionable decisions.

For example, a simple promiscuous packet capture on a network card for barely ten minutes provides us with a few megabytes of data. Finding out which HTTP request failed, and why, is like finding a needle in a haystack. Another important consideration is to identify key areas, often called choke points. The incorrect selection of a choke point can result in erroneous data that doesn’t accurately reflect the current performance scenario. This leads to incorrect capacity planning and security mapping.

One challenge worth mentioning is caused by the unprecedented growth of a network, a result of the organisation’s growth due to business expansion or company mergers. The bigger the network, the tougher it is to visualise the scale of network infrastructure. This can result in performance bottlenecks as well as security vulnerabilities. Finally, failure to incorporate proper monitoring tools is also a challenge to be addressed by senior IT management staff. It has been observed that relying purely on commercial products actually limits a firm’s ability to bring diversification into the network monitoring process.

The use of appropriate FOSS tools and products is highly recommended wherever applicable. As an example, there is hardly any single commercial tool that can relate Layer 7 network monitoring to the underlying Layer 2 packet and help troubleshoot a performance problem.

Network security challenges

From the infrastructure scaling point of view, irrespective of the size of an organisation, network security is often a complex area to deal with. Since network infrastructure contains components like firewalls, routers, managed switches, etc., the configurations and settings for each of these components further add to the complexity.

Also, when faced with the choices of multiple devices offered by many vendors, it is easy for a network architect to get distracted from considering an appropriate solution customised to the network. As the network grows, it can be more prone to vulnerabilities and loopholes, needing tight security policies and careful designs, using cutting-edge technology devices and solutions.

From the security point of view, a new breed of viruses and spyware has emerged recently, which exploits the operating system as well as the networking device’s vulnerabilities, and can take control to cause enough damage. Though there are multiple security solutions available, hackers are often one step ahead of the cybercops.

It is often the case that an organisation is more prone to internal attacks than to attacks originating from outside the firm’s network infrastructure. Preventing such attacks needs the latest techniques, such as the deployment of intrusion detection systems, unified threat management systems (UTMs), etc.

Devices involved

Network monitoring is the term used for health monitoring, whereby monitoring utilities keep a watch on various networking components, to ensure their uptime and overall performance. Network security monitoring is at a level further down the networking layers, whereby utilities capture each and every traffic packet in a promiscuous mode, correlate it with known and unknown attacks and hacking styles, store the log for further “deep-dive” analysis, and report the alerts.

While there are known components such as intrusion-detection devices, UTMs and firewalls, there are multiple software solutions, both off-the-shelf or FOSS-based, which extend the functionalities of these components, and work on the extensive log data gathered by monitoring appliances and devices. We will discuss these solutions in more detail shortly.

Best practices in network security monitoring

Network security monitoring involves collecting network packet data, segregating it among all the 7 OSI layers, and applying intelligent algorithms to get answers to security-related questions. The purpose is to know in real-time what is happening on the network at a detailed level, and strengthen security by hardening the processes, devices, appliances, software policies, etc.

While there is no single list of practices that can cover all possible situations, we can still enumerate some best practices to implement and follow for any network infrastructure:

1. Perform network performance measurement before deploying the security monitoring solution. This is essential because security monitoring can have its own footprint on the network, especially if the monitoring solution is software-based, running on the servers.
2. If possible and affordable, deploy more than one anti-virus solution. Many anti-virus software solutions don’t offer spyware detection, or don’t do it right; hence, a combination is always helpful.
3. Deploy at least one FOSS packet-capturing software on the network. Though the IDS systems do this job partially, there can be situations where the IDS could be too busy to be used as a packet viewer, and a FOSS utility can come in handy for daily chores.
4. Gather monitoring data at a secure place. It is often a mistake to gather security data on a desktop or a server which is easily accessible, making the network vulnerable at that point. Since security applies to the monitoring process too, the data captured must be stored in a secure manner.
5. Monitor all layers; don’t leave anything to chance. Usually, the data link layer is omitted from monitoring; however, since a new wave of attacks can exploit Ethernet frames too, it is important to take this layer into account. The same applies to the network layer, as most internal attacks can easily use it to exploit vulnerabilities.
6. Deploy the IDS behind the firewall, since the firewall filters out everything that is not meant to enter the LAN. This improves the IDS’ efficiency by keeping the clutter away.
7. Capture VLANs separately. Since VLANs are separate TCP broadcast domains, separately gathering and analysing data for each can help detect internal and external security problems quickly.
8. Consider all protocols. Many firms still use NetBIOS internally along with TCP/IP; such a situation demands monitoring all protocols on the wire. There are a few legacy types of attacks based on the NetBEUI protocol, which could be captured.
9. Enable optimal auditing levels on the monitoring devices. Setting up too many audit event captures can easily confuse monitoring solutions while detecting an anomaly, whereas having very few audit logs can render security monitoring useless.

It is a best practice to have an update in the monitoring process whenever a device is added to the network, removed or changed. Also, even if there is no change in the design, monitoring processes should be reviewed in a timely manner, in order to remove errors, and keep pace with changing security scenarios.

Monitoring network security using FOSS solutions

It is worth mentioning a few commercial products used in network security monitoring, before we talk about the FOSS solutions.

Cisco Security Monitoring, Analysis, and Response System (MARS) is a famous solution that falls in the category of Security Threat Mitigation systems (STM). MARS gathers and stores raw network data, and correlates it with intelligence algorithms. IBM Proventia IDS devices fall under the same category, providing richer features and customisable alerts. All these threat monitoring and mitigation appliances enable us to centralise detection, mitigation and reporting on priority threats by leveraging the network and security devices already deployed in a network.

In the FOSS world, the Backtrack Linux distribution is famous among cyber-security groups, as it allows one to write easy, effective and yet industry-standard utilities quickly. Snorby is an open source suite of network security monitoring utilities, which interacts seamlessly with open source IDS solutions such as Snort, Suricata, etc. Snorby contains the necessary binaries to run on various operating systems, including Microsoft platforms.

Siem-Live is another such distribution gaining support and momentum. A good thing about this distribution is that it comes on a bootable CD, relies on generic IDS systems and log collectors, and applies intelligence on it to create customised reports. The event correlation engine of Siem-Live makes it fit for small- and mid-scale networks.

This article will be incomplete if a famous tool called SGuel is not mentioned here. While maintaining the quality of the intelligence engine, this tool comes with a nice GUI and essential bells and whistles, such as real-time event gathering, raw packet capturing, reporting, etc.

Summary

While there are numerous tools and practices available for monitoring the security of a network infrastructure, it is important to apply the industry’s best practices. It is imperative for IT management teams to know what is happening on the network, in real-time, and accordingly introduce the latest technologies. Since cyber security is a process of continuous improvement, there cannot be a single list of best practices, as the quality of practices evolves with network architecture and design, as well as the devices in use.

Network security monitoring is essential for the IT health of an organisation, and a rock-solid monitoring mechanism can help reduce damages to the business. While there are commercial solutions available for this, FOSS also provides real-time monitoring software and cutting-edge intelligence engines. Senior IT management personnel need to form a security monitoring group within the organisation to deploy the best practices and strengthen the corporate network.