Best Practices for Deploying Intrusion Prevention Systems. A better approach to securing networks

Best Practices for Deploying Intrusion Prevention Systems A better approach to securing networks

A better approach to securing networks Contents Introduction 3 Understanding deployment needs 3 Selecting placement points 5 Choosing the right IPS to meet your needs 5 Tuning and configuring 6 Ready for prevention 8 Other useful tips and practices 8 External notification 8 IPS authorized privileges 9 Retention considerations 9 Summary 9

Best Practices for Deploying Intrusion Prevention Systems Introduction Interest in intrusion prevention has been gaining ground since late 2006. There are a number of reasons for this, not least of which is the thinking that a defensein-depth strategy is essential in any enterprise network environment. Regulatory compliance reports and other requirements are also leading many to consider implementing intrusion prevention systems (IPSs) as a next-generation security technology. Whatever the reason, it is important to not only select the right technology but to deploy this technology correctly. To enable the accomplishment of both objectives, this white paper will outline the criteria for a successful IPS deployment. First, it is important to remember that IPS can refer to network- and host-based IPS, firewalls, and modified networking equipment like routers and switches. Because of the limitations of certain network equipment and IPS solutions that are host-based, this white paper will only focus on the best practices related to deploying a dedicated network IPS. Understanding deployment needs Before deploying an IPS, there has to be an understanding of what is going to be protected. It would be easy to say everything. However, what does everything mean? It certainly refers to applications and servers. But it could also mean printers, desktops, routers, switches, or IP infrastructure like mail, DHCP and DNS servers, and other network-attached devices. The problem is that when everything is protected by IPS, it can set up unrealistic expectations. The key is to not plan aggressively when initially deploying IPS. Additional rules or control that is more granular can always be implemented as security management skill sets and understanding of network and application functionality rise. With IPS, it is best to concentrate at the perimeter and at externally facing services such as FTP, email, and Web services. The protected services and resources should be the most business-critical processes, where relying on a single mode of protection is impractical or insufficient especially in consideration of regulatory compliance mandates like the Sarbanes-Oxley Act, the European Union Directive 95/46/EC and the Gramm-Leach-Bliley Act. Once you know what you want to protect, you can then think about the things you want to protect them from. As an example, you may already have two types of protection for protocol-based vulnerability exploits and Trojans in the form of a firewall and antivirus software. But you may not have the means to protect your critical processes from brute force or application-based attacks or insider attacks, which could represent a targeted internal threat. Successful IPS deployments include being able to define the threats you wish to protect the enterprise from. Don t discount this seemingly simple notion. Understanding the threats you want to protect your environment from has a tremendous impact on your deployment requirements. There are classifications for most exploits, spyware, and malware that could find their way into your environment. It is important to classify threats so that they can be dealt with effectively as a group, whenever possible. Managing threats individually can be daunting. However, at many levels there are often commonalities between threats in how they act, infect, and spread. Check Point Software Technologies, Ltd. 3

A better approach to securing networks A subset of the threat classification may include: Authentication and authoritative issues. This could include: Privileged access acquiring administrative credentials (such as root) without proper authorization User access acquiring the credentials of a user without proper authorization Malware. This could include: Worms matching known service exploits or perhaps acting similar to a known exploit Code execution the execution of arbitrary exploit code on a targeted system that may install unwanted components such as keyboard loggers Denial of Service (DoS) denies service to legitimate uses. This could include: Ping of death Syn flood Best practices violations not malicious activity but something that violates best security practices such as a username with no password or a banner that indicates a vulnerable software version. This could also include: Security policy violations this could be characterized by instant messaging, streaming video, or logging into a system or application from an unauthorized subnet. This could be expanded to seeing traffic that indicates a firewall or other security policy enforcement point has not been configured correctly or has been compromised Password lengths and the proper mix of letters and numbers Information gathering as a prelude to an attack Accessing data or attempting to move restricted files, directories, or data Application-based attacks attempts to exploit vulnerabilities in certain kinds of servers by means of buffer-overflow and injection-attack attempts including: Web-based injection attacks that try to gain access to information or privileges outside the domain of the application Buffer-overflow attacks aimed at general applications or services DNS usurping and spoofing A method classified above is often preceded by a discovery or recon effort by the threat. These efforts will determine for a potential attacker if some part of your infrastructure is vulnerable. It could also allow for the tailoring of exploits for your specific type of system. Many discovery efforts are so elaborate, that they can even determine if your vulnerable component is capable of spreading infection, influence, or control to other systems and components. You will need to invest time tuning your IPS to your specific environment. Of the aforementioned classifications; authentication, malware, and DoS are relatively easy to implement. Keep in mind that establishing best practices by changing user behavior will require some education and is best accomplished in modest segments. Best practices violations and application-level attacks are far more insidious and are very important to catch early. Remember, that every time you patch those applications or make a change to your policies, you may need to retune. 4 Check Point Software Technologies, Ltd.

Best Practices for Deploying Intrusion Prevention Systems Selecting placement points Placement of sensors is vitally important for a successful IPS deployment. Where should you put IPS devices to maximize their effectiveness? Anywhere your infrastructure or applications are unjustifiably at risk these areas would be likely targets. Typically, IPS devices are deployed: 1. Behind firewalls and WAN routers 2. In front of server farms or similar collections of resources 3. At other network access points By concentrating on these critical points, you will reap greater rewards from your initial deployment. The reason for this is that most compliance requirements focus on the ingress and egress points to the network core. Also, deploying IPS at these choke points in the network provides maximum protection opportunities because they involve transporting and enabling the most network traffic. WAN router points are excellent candidates for IPS deployment as they are often the ingress points for exploits from remote sites where you have little direct control and perhaps no authoritative control. If a remote site or business partner site is compromised, you are often defenseless against an infection already running rampant at that location. If extranet or trading partner VPNs are a recurring source of vulnerability, you should review the advantages of a Firewall-integrated IPS function like VPN-1 with SmartDefense protections. In addition to server farms and other hardened access points, a connection from a wireless warehouse application is another type of access area. Blackberry servers or handheld wireless barcode readers are examples of this. These areas are especially vulnerable points within any infrastructure. They often mark boundaries within your network and may represent services and devices that cannot be protected by other methods. These boundaries also represent additional logical and physical responsibilities. These access points signify hard-to-secure applications or services. However, they must be protected. Choosing the right IPS to meet your needs Not all network IPS systems are created equal. With the myriad of vendor claims, confusion can arise from the process of selecting the right IPS for your needs. The following points are key criteria to consider when choosing an IPS: Detection accuracy when considering IPS, it s easy to overlook the fact that to do prevention right, you need accurate, granular detection. You need to pay attention to on-the-wire detection capabilities and detection-test-accuracy scores because unlike intrusion detection systems (IDS) where a false-positive is frustrating, an IPS false-positive can have a direct impact on business Bandwidth requirements instead of getting caught up in speeds and feeds; consider the bandwidth requirements of your network. If the link to your remote site is a T3 line, it does not make sense to place a Multi-gigabit-capable device at that point. However, regardless of advertised bandwidth, always be sure to validate that it meets your need in active inline mode rather than simply passive monitoring mode Check Point Software Technologies, Ltd. 5

A better approach to securing networks Management platform often when evaluating IPS, the focus is on appliances and sensors with no consideration of the overall management platform. The situational visibility necessary to effectively manage network-wide intrusion prevention, provide automated signature updates, deploy upgrades, and configure policies should all be accounted for as part of your evaluation Tuning flexibility it is important that you have power and flexibility when tuning IPS, particularly the ability to tune prevention/blocking to a qualitative or confidence score that will help mitigate concerns regarding false-positives. Review your architectural decision. Will your selected architecture be able to meet the detection and processing requirements of evolving threats? High availability in the IPS model, ensuring appliance high availability is a must. The appliance should have requisite zero-power fail-open options. However, in organizations where security is business critical, you should also give attention to need for high availability throughout your overall IPS architecture. For instance, the server components should offer failover capability Scalability you should weigh scalability based on the size of your environment, and plans for expansion. However, if you are running more than a nominal number of sensors or if you have plans to grow your deployment substantially, you should ensure the overall architecture scales to meet your needs Reporting in view of regulatory compliance requirements, the ability to report on the state of known attacks, protection coverage, remediation, and vulnerabilities has become a critical need If you consider these seven key criteria, you will be able to make the right selection for initial deployment. Tuning and configuring Once your system is installed, you may be tempted to turn on every available inspection method. However, this is not the ideal way to configure your system. Remember the business objectives and the earlier classifications. Enable just one group at a time, starting with the ones that you know are most likely to impact business operations. Then examine the sensor alerts, watching for just those classification exploits. You will likely gain insight to your network that you never had before, even if you regularly run vulnerability scans or penetration tests. Explore these alerts and verify if they are true (positive) or not true (false-positive). If they are not true, tune the entire system or just the specific systems involved with the false-positive. Typically, this can be done by using the IP address of the source and destination systems involved in the alert. You may choose to configure your IPS to ignore the traffic entirely (white-list) or record the event but not report it to the console. You can also directly modify the applications causing the event and thus eliminate the alerts at the source. Once you complete this for one alert classification, you should enable more groups and repeat the process. 6 Check Point Software Technologies, Ltd.

Best Practices for Deploying Intrusion Prevention Systems This process should also account for the confidence score that a particular detected event is actually malicious. Some products may have this capability built in and offer you a more granular level of tuning. This can be tremendously beneficial because it allows you the capability to configure the level of prevention based on the confidence score associated with a given security event. An attack with a known signature match should get a high confidence score while suspicious activity that may be ambiguous in nature will be given a lower confidence score. As the user, you could set the IPS to block attacks that score 90 percent or higher. This ensures that you can prevent serious attacks while not risking the possibility of inadvertently blocking legitimate traffic. Once you have done your initial tuning, and any necessary remediation, you may think you are ready for the next step. But many applications do not run all the time, where they would be seen early and often by an IPS. Many, like backups, run only at night or on odd days. Financial and payroll applications may only run weekly or monthly. Accounting packages may only run at month-, quarter-, or year-end. So, although the bulk of the tuning can be done at the beginning, you will most likely have to revisit this process over the course of the next few days, weeks, or months. A common approach is to tune initially, and then tune again one weekend and one month- or quarter-end later. An easy place to start is with inspection for known malware and/or malicious code execution. This will offer immediate benefits because the IPS immediately will begin mitigating worms and viruses at the point of deployment. It is also important to bear in mind that systems already infected with malware can be carried into your network from the outside. Some consideration should be made to identify internally infected resources. Once you have conducted your advanced tuning regarding external threats, you can then create rules and policies for your IPS to address internal compromised resources. Laptops are prime targets since they are often used outside the protective corporate environment for extended periods. The backdoor communications of these infected systems is what gives them away. Certain malware and spyware have a replication and reporting component where the infected system tries to communicate with a master system while it tries to spread its infection. For example, an exploit may launch its own email server and then emails out its infection to every person listed in your address book. This can be detected by looking for outgoing email traffic coming from an email server not identified as a corporate mail server. It may even be sending over a nonstandard mail IP port number. Check Point Software Technologies, Ltd. 7

A better approach to securing networks Ready for prevention Up to this point, you might think you are in good shape, having deployed an IPS system and tuned it to a high degree of confidence. You should be comfortable that the received alerts are real attacks. As such, you can now take action. It is time to determine how you want to eliminate the offending traffic. Examine and then make your choices for stopping attempts before configuring a preventive response. Your choices usually break down into three approaches: Drop the traffic in this case, the packet is dropped and there is no protocolbased handshake with the participating parties to notify them of the event. This can be good news since it makes it harder for an attacker to figure out what is thwarting his/her efforts. At this point, it is harder for this person to decide how to proceed Blacklist the attacker this means that once an attacking source is identified, it is added to a list that is first examined when a packet shows up for inspection. If it matches a previous entry, no further inspection is required, and the packet can just be dropped. The benefit here is that it is less overhead for the IPS system. This capability is an important layer in your defense against a DoS attack Reset sends TCP resets to the attacker and the intended victim so they both know the connection has been closed. This is the gentlest method and is often used for policy violations. This allows both parties applications to recover gracefully Other useful tips and practices Here are some helpful tips and practices to keep in mind. External notification The first involves automatically externalizing notification. In many small- to medium-size companies, it is impractical to dedicate a single person to continuously watch the console. If a designated security person is available, duties are often widespread and the console may not be kept in constant view. Therefore, you can choose to have the critical notification externalized to a mobile device like a wireless PDA or cell phone. The choice of what should be externalized will be done based on the severity of the attack. To add to that line of thinking, you could also categorize each event into groups based on severity. Attacks of opportunity an attacker, usually an automated process, suspects one of your systems has a vulnerability. This is usually a random shotgun attempt to infect or exploit as many systems and networks as possible, like a worm or Trojan. These can also include the difficult-to-detect blended threats where more than one type of discovery, attack, or threat replication is combined with others Attacks of intent an attacker has focused on a target or enterprise and will keep up the assault until success or the arsenal is exhausted. If your signatures and policies are up-to-date, most attacks of opportunity should be handled automatically and do not require direct notification, unless it is targeting a specific system of high value to your business operations. Attacks of intent are something else. Someone making a deliberate attempt to breach corporate security or violate policies should warrant your immediate notification, especially if the attempted breach involves a business-critical server 8 Check Point Software Technologies, Ltd.

Best Practices for Deploying Intrusion Prevention Systems IPS authorized privileges Another item of importance is related to privileges. IPS is not just an appliance that stops bad traffic: it is a point of protection and policy enforcement. Like with all critical infrastructure components and systems, each administrator should use a separate set of credentials to gain access so that all activity and changes can be logged and traced back to that individual, if necessary. Many IPS systems support a hierarchical approach to managing administrative users that makes this easier. Retention considerations Although this is less of an issue in IPS, the last topic to consider is the retention of alert information. To answer the retention question, start with two pieces of information: Your company s policy on retaining information look at policies that relate to phone records or system log files as a guide The recommended practices or compliance requirements that govern your business Hopefully, a straight-forward comparison to prevailing retention policies and backup procedures will affirm that they align and agree. When considering how you will retain and store this alert information, remember to consider that some jurisdictions will not allow IPS/IDS records to be admitted into evidence at a legal proceeding if they have been altered in any way. If they were compressed, or truncated to save space, they may not be allowed from a forensic perspective. This may be a concern if your organization ever has a need to use this information to prosecute or defend an individual or organization. It is also a good idea to check with your IPS vendor on guidelines for disk space planning. Summary Leading a successful IPS deployment will require the following steps: Understanding your needs for real-time threat protection Selecting the right IPS product for your organization Determining the right placement points for your IPS deployment Taking the time to tune your system right Setting up your compliance-reporting parameters Configuring your IPS for data retention and backup Periodic but necessary evaluation of your overall system use Check Point Software Technologies, Ltd. 9

About Check Point Software Technologies Ltd. Check Point Software Technologies Ltd. (www.checkpoint.com) is a leader in securing the Internet. The company is a market leader in the worldwide enterprise firewall, personal firewall, data security and VPN markets. Check Point s PURE focus is on IT security with its extensive portfolio of network security, data security and security management solutions. Through its NGX platform, Check Point delivers a unified security architecture for a broad range of security solutions to protect business communications and resources for corporate networks and applications, remote employees, branch offices and partner extranets. The company also offers market leading data security solutions through the Pointsec product line, protecting and encrypting sensitive corporate information stored on PCs and other mobile computing devices. Check Point's award-winning ZoneAlarm Internet Security Suite and additional consumer security solutions protect millions of consumer PCs from hackers, spyware and data theft. Extending the power of the Check Point solution is its Open Platform for Security (OPSEC), the industry's framework and alliance for integration and interoperability with "best-of-breed" solutions from hundreds of leading companies. Check Point solutions are sold, integrated and serviced by a network of Check Point partners around the world and its customers include 100 percent of Fortune 100 companies and tens of thousands of businesses and organizations of all sizes. CHECK POINT OFFICES Worldwide Headquarters 5 Ha Solelim Street Tel Aviv 67897, Israel Tel: 972-3-753 4555 Fax: 972-3-624-1100 email: info@checkpoint.com U.S. Headquarters 800 Bridge Parkway Redwood City, CA 94065 Tel: 800-429-4391 ; 650-628-2000 Fax: 650-654-4233 URL: http://www.checkpoint.com 2003 2008 Check Point Software Technologies Ltd. All rights reserved. Check Point, AlertAdvisor, Application Intelligence, Check Point Endpoint Security, Check Point Endpoint Security On Demand, Check Point Express, Check Point Express CI, the Check Point logo, ClusterXL, Confidence Indexing, ConnectControl, Connectra, Connectra Accelerator Card, Cooperative Enforcement, Cooperative Security Alliance, CoreXL, CoSa, DefenseNet, Dynamic Shielding Architecture, Eventia, Eventia Analyzer, Eventia Reporter, Eventia Suite, FireWall-1, FireWall-1 GX, FireWall-1 SecureServer, FloodGate-1, Hacker ID, Hybrid Detection Engine, IMsecure, INSPECT, INSPECT XL, Integrity, Integrity Clientless Security, Integrity SecureClient, InterSpect, IPS-1, IQ Engine, MailSafe, NG, NGX, Open Security Extension, OPSEC, OSFirewall, Pointsec, Pointsec Mobile, Pointsec PC, Pointsec Protector, Policy Lifecycle Management, Provider-1, PureAdvantage, PURE Security, the puresecurity logo, Safe@Home, Safe@Office, SecureClient, SecureClient Mobile, SecureKnowledge, SecurePlatform, SecurePlatform Pro, SecuRemote, SecureServer, SecureUpdate, SecureXL, SecureXL Turbocard, Security Management Portal, Sentivist, SiteManager-1, SmartCenter, SmartCenter Express, SmartCenter Power, SmartCenter Pro, SmartCenter UTM, SmartConsole, SmartDashboard, SmartDefense, SmartDefense Advisor, Smarter Security, SmartLSM, SmartMap, SmartPortal, SmartUpdate, SmartView, SmartView Monitor, SmartView Reporter, SmartView Status, SmartViewTracker, SMP, SMP On-Demand, SofaWare, SSL Network Extender, Stateful Clustering, TrueVector, Turbocard, UAM, UserAuthority, User-to-Address Mapping, UTM-1, UTM-1 Edge, UTM-1 Edge Industrial, UTM-1 Total Security, VPN-1, VPN-1 Accelerator Card, VPN-1 Edge, VPN-1 Express, VPN-1 Express CI, VPN-1 Power, VPN-1 Power Multi-core, VPN-1 Power VSX, VPN-1 Pro, VPN-1 SecureClient, VPN-1 SecuRemote, VPN-1 SecureServer, VPN-1 UTM, VPN-1 UTM Edge, VPN-1 VSX, Web Intelligence, ZoneAlarm, ZoneAlarm Anti-Spyware, ZoneAlarm Antivirus, ZoneAlarm ForceField, ZoneAlarm Internet Security Suite, ZoneAlarm Pro, ZoneAlarm Secure Wireless Router, Zone Labs, and the Zone Labs logo are trademarks or registered trademarks of Check Point Software Technologies Ltd. or its affiliates. ZoneAlarm is a Check Point Software Technologies, Inc. Company. All other product names mentioned herein are trademarks or registered trademarks of their respective owners. The products described in this document are protected by U.S. Patent No. 5,606,668, 5,835,726, 5,987,611, 6,496,935, 6,873,988, 6,850,943, and 7,165,076 and may be protected by other U.S. Patents, foreign patents, or pending applications. May 30, 2008 P/N 503062