PROFESSIONAL SECURITY SYSTEMS

PROFESSIONAL SECURITY SYSTEMS Security policy, active protection against network attacks and management of IDP Introduction Intrusion Detection and Prevention (IDP ) is a new generation of network security systems which implementation is NetScreen IDP (formerly OneSecure ). This is a network security system performing intrusion detection tasks able to block attacks in a real time. IDP as opposed to ordinary IDS systems, which by listening to the network identify events when intruders have already made their attacks, and practically are unable to block them, is able to block attacks efficiently. Even if an ordinary IDS notifies the Firewall about an event, it is too late. IDP system operates in an in-line mode as the active network gateway, directly on the network traffic route. IDP system detects and blocks intruders before gaining access to the protected IT resources IDP makes possible to react at scanning attempts, penetrations and break-in attacks of Exploit type (on network and application level), destructive attacks of (D)DoS type as well as other techniques used by hackers. IDP sensors are delivered as dedicated, ready to deploy Appliances. IDP devices throughput is over 400 Mb/s. They can operate in in-line mode (e.g. network traffic flows through the device) or as a sniffer (a device listening to the network traffic). In the in-line mode of operation, IDP sensors can be deployed as routers or as bridges. CLICO Ltd., Al. 3-go Maja 7, 30-063 Kraków, Poland; Tel: +48 12 6325166; +48 12 2927525; Fax: +48 12 6323698; E-mail: support@clico.pl, orders@clico.pl.; Ftp.clico.pl.; http://www.clico.pl

IDP system architecture IDP security system has complete, three-layer architecture: sensors, management server and GUI interface. It allows for efficient security means deployment and network security management. IDP consists of the following components: IDP sensors - analyze whole network traffic, detect and block attacks and enforce accepted security policy, IDP Management Server - stores and manages all attack signatures, log database as well as security policies set. GUI interface - graphic tools for IDP management which allow administrators to perform their tasks from any point in the network. IDP can be deployed in a distributed (e.g. all the components are separate) or centralized architecture, where the management server and IDP sensor operate on the same device. All the IDP management and log processing is performed on the central management server. Security policies are created on the management server and installed on the IDP sensors in the network. Network communication between all the IDP components is protected by cryptographic means. IDP sensors can operate in in-line or sniffer modes. When operating in in-line mode, IDP sensors can be deployed in a failure resistant HA architecture. 2003 CLICO LTD. ALL RIGHTS RESERVED 2

Attack detection techniques IDP security system utilizes many identification and attack protection methods. Attack detection methods are included depending on the network traffic controlled. At the same time data analysis using individual detection methods is performed (parallel processing). It ensures high detection level of attacks without affecting performance. Multi-Method Detection (MMD) mechanism used in the IDP includes the following detection methods: Stateful Signatures - detection of known attacks based on the signature database. Stateful signatures contain data about attack pattern as well as the communication type, where such an event could take place. Network traffic is exposed to the context analysis, thanks to which most of so-called false positives are eliminated. Attack patterns are searched only in the specific network communication, thus ensuring high control efficiency and security performance. Protocol Anomalies - detection of discrepancies between network traffic and standards of specific protocols (among others RFC). It happens in reality that intruders in order to confuse security means or hide real attacks generate network traffic different from accepted rules and standards. Backdoor Detection - detection of Trojans activities as well as unauthorized access to the protected systems through so-called backdoors. Detection is performed by comparison of network traffic with known intruders activity templates as well as an heuristic analysis of packets transmitted. Traffic Anomalies - identification of network activities regarded as prohibited or suspicious, performed in a form of many different connections (e.g. ports scanning). The subject of an analysis are connections within specific period of time. IP Spoofing - detection of network traffic with forged IP addresses of the packet sender. The IP Spoofing technique is often used by intruders in order to hide a real attack source. IDP detects IP Spoofing by comparing IP addresses in packets with addresses used in internal networks. Layer 2 - detection of attacks and suspicious activities on the 2 layer of the OSI model and MAC addressing levels (e.g. ARP cache poisoning). This is particularly valuable in case of IDP control of internal networks. Denial of Service Detection - detection of destructive and destabilizing DoS attacks (Denial-of-Service). DoS attacks are usually performed through sending to the service server many specifically created requests which cause running out of its resources (e.g. SYN-Flood). Network Honeypot - early detection and recognition of intruders activities by using a honeypot technique. In case of scanning, penetrating or break-in attempt IDP presents to intruders fictitious information regarding services available on servers. 2003 CLICO LTD. ALL RIGHTS RESERVED 3

IDP administrator console IDP system is managed from the central management console (see figure). All the IDP security means in the network regardless of their location operate according to one, coherent security policy of the enterprise. The GUI console allows for maintaining one, central enterprise security policy with regard to intruders detection and active attacks blocking. 2003 CLICO LTD. ALL RIGHTS RESERVED 4

Management tools overview IDP management server is responsible for centralized creation and deployment of enterprise's security policy with regard to detection and blocking attacks and other prohibited activities as well as consolidation of events (logs, alerts) registered on many sensors in the network and storing them in the central database. Administrators can control the whole IDP system from any point in the network using dedicated, graphic tools. The GUI interface consists of the following six components: Security Policy Editor - a tool allowing for creating different rules (e.g. Main, SYN- Protector, Network Honeypot, Backdoor Detection, Traffic Anomalies, Sensor Settings Rulebases) and deploying them on the specific IDP sensors in the network. Dashboard - displays the most important statistics regarding network and security means operation in a real time. It contains the following sections: Host Watch List, Source Watch List, Attack Summary, Reports and Device Status. IDP graphic console presents the current network security status 2003 CLICO LTD. ALL RIGHTS RESERVED 5

Object Editor - object management 1 on which the IDP security system operates. Basic objects include Network Object (e.g. network, host, server, sensor), Service Object (e.g. FTP, HTTP, Telnet) and Attack Object (e.g. attacks and protocols anomalies signatures). Log Viewer - browsing and analyzing of events logged by IDP sensors according to the accepted security policy. Displays record logs in the table format with possibility to define specific data filtering and selection rules. Administrator analyzes events logged by the IDP sensors in real-time. Device Monitor - shows the current state of the IDP sensors and the management server (among others processor, RAM, security processes) and in case of emergency alerts the Administrator. Reports generates different reports based on the events logged by the IDP sensors. 2003 CLICO LTD. ALL RIGHTS RESERVED 6

IDP security policy Currently existed IDS systems are accused of illogically analyzing the attack signature database (e.g. communication to the DNS server is analyzed focusing on HTTP attacks) and sensors detect and log events which are irrelevant from protected network security perspective. Most of the currently available IDS systems were created in 1995-1997. At that time, IDS signature database did not exceed 100 records. Development and complexity of network environment as well as large, constantly increasing number of known attacks, require effectiveness of IDS systems operation. The most complex and time-consuming operation for IDS systems are those which are performed with use of a signature database (among others matching attack signatures to the captured network traffic). The signature database consists of large number of basic attacks definitions (over 1000 records) as well as of different mutations of those attacks. For effective operation of security means, the IDS security policy should unambiguously define what kind of network traffic is a subject of inspections by the specific sensors and what this traffic should be searched for (e.g. what kind of attacks). It became important to create a detailed IDS security policy. Identification of the traffic network itself (e.g. reading the packet headers) is performed much quicker than an analysis of the signature database. The IDP security policy consists of set of rules, which unambiguously determine what kind of network traffic should be inspected by specific IDP sensors, what should be looked for in it (e.g. what kind of attacks) and what kind of activity should be initiated by the system in case when these events are detected. The IDP security policy allows for effectively maintaining their logical cohesion and correctness as well as their efficient management. On the other hand, the security policy of old generation IDS system only makes out what signatures should be included on the specific sensor and what kind of IDS sensor's action should be undertaken for each signature. Maintaining logically correct and efficient security policy of an old generation IDS system, taking into account big volume of constantly increasing signature database (currently about 1500 records) is extremely time-consuming. IDP belongs to the new generation of intrusion detection and attack protection systems. It provides administrators with robust tools for monitoring and control of network traffic which were unavailable in old generation of IDS systems. IDP configuration is directly based on the accepted IT system security policy. IDP has been developed in a way to avoid necessity to tune the security policy to the accepted security technology (e.g. resigning for required security function as a result of security technology limitations). 2003 CLICO LTD. ALL RIGHTS RESERVED 7

IDP security policy consists of set of rules describing security means operation principles. The rules are defined through the graphic Policy Editor. The Policy Editor contains five sections: Main basic rules regarding network monitoring, prohibited and suspicious activity detection as well as attacks elimination, Backdoor Detection network traffic analysis (among others heuristic analysis) focusing on interactive session detection indicating existence of applications of a Trojan or Backdoor types, Network Honeypot presenting of fictitious information regarding IT system services in case of detection of unauthorized access attempts, SYN-Protector protection of servers against attacks of Syn Flood type, Traffic Anomalies - detection of suspicious operation in the network (e.g. TCP and UDP ports scanning). When defining network traffic control rules, the Administrator can choose a configuration mode Basic, which contains only basic settings or Advanced, which describes IDP operation more specifically (View Main Rulebase). What kind of network traffic is the subject of inspection? What should be looked for (e.g. what kind of attacks)? What action should be taken in case of an event detection? 2003 CLICO LTD. ALL RIGHTS RESERVED 8

The Match field unambiguously identifies the network traffic which is subject to control. Who initiates communication (the client)? With whom communication is being performed (the server)? Is the communication controlled against the other security rules after being identified by the IDP? When defining IDP rules in the Advanced mode there is an additional field Service, which determines network protocols and services which are under control. The Look For field identifies events and attacks, which should be detected by the IDP. 2003 CLICO LTD. ALL RIGHTS RESERVED 9

The Action field describes the way IDP operates after the event has been identified. What action is taken by the IDP? How the Administrator is notified? Where the rule is valid? The IDP system after detecting an event can undertake different actions depending on the mode it operates: none no action, ignore the network traffic is ignored, close client & server closing session and sending an RST packet to the client and the application server. close client closing session and sending an RST packet to the application client, close server closing session and sending an RST packet to the application server, (in the in-line mode only) drop packet blocking the packet without sending an RST packet, drop connection blocking the connection without sending an RST packet. 2003 CLICO LTD. ALL RIGHTS RESERVED 10

The IDP Administrator can be notified about the event in the following way: logging storing information about the event in the Log, alarm information about the event is displayed on the console as an alarm (flag in the Log Viewer), session description of the event additionally includes information about the number of packets in session, number of bytes and time of session duration, SNMP Trap the responsive message to the SNMP manager, syslog sending information about the event to the SYSLOG server, send email sending information about the event via e-mail, run script running a specific script or an application, log packets logging packets before the event and/or after the event (i.e. in order to perform a detailed analysis of the network traffic). Settings for different types of IDP notification are performed in the Tools Preferences menu. 2003 CLICO LTD. ALL RIGHTS RESERVED 11

When defining IDP rules in the Advanced mode there are two additional fields: IP Action action is undertaken after detection of the event for next packets within the same communication (e.g. blocking packets from IP address which is the source of the attack for specific time), Severity event priority. Note: IP Action settings should be carefully thought over before they are used. This is a very strong mechanism for the Administrator and its utilization should be specially planned. It allows for blocking any intruders action in case when IDP has detected their unaccepted behavior. First of all, the IP Action should not be used in response to attacks, which might possibly be made from different IP addresses (e.g. IP Spoofing, IP addresses in packets sent by intruders have been forged). 2003 CLICO LTD. ALL RIGHTS RESERVED 12

IDP system perform analysis of specific network traffic performing among others the following: heuristic analysis focusing on interactive session detection, indicating existence on the protected servers applications of Trojan or Backdoor types. For instance, all the sessions with the FTP server except for the FTP protocol are analyzed for existence of applications of Backdoor/Trojan type. In case of detection of unauthorized access attempts to the protected servers, IDP systems present to intruders false information about IT system services available there. For instance, in case of access attempt to the FTP and Web services of Windows 2000 domain controller, intruders get only apparent access to the server and all their actions are monitored and logged. 2003 CLICO LTD. ALL RIGHTS RESERVED 13

Server protection in networks protected against DoS attacks (destabilizing, destructive) using SYN Flood technique. IDP system after noticing the attack can send an RST packet to the TCP server (passive method) or establish temporary TCP session with the server in order to better recognize the attack (relay method). IDP system detects suspicious operations (e.g. TCP/UDP ports scanning) and reacts to them according to the security policy settings. 2003 CLICO LTD. ALL RIGHTS RESERVED 14

Analysis and reporting of logged events IDP system in a real time detects unauthorized and suspicious operations such as scanning, penetration and break-in attempts, attacks of Exploit type (on network and application levels), destructive and destabilizing attacks of (D)DoS type as well as many other techniques used by hackers. Attack detection is being performed using different methods used depending on the type of the traffic analyzed (among others Stateful Signatures, Anomaly Detection, Network Honeypot, Spoofing Detection, Backdoor Detection). Administrator immediately analyzes the security of the IT system using dedicated tools. With Log Viewer he views and selects events logged by individual IDP sensors. Special tools Log Investigator allow for detailed recognition of how attacks are conducted and what is their range. 2003 CLICO LTD. ALL RIGHTS RESERVED 15

The events logged in the IDP log can be stored in the other format (e.g. PDF file). Based on their content administrator can create reports focusing on the analysis of specific behavior or security state (e.g. servers which are scanned most often, attacks most often performed, list of the most dangerous attacks. When needed, the reports can be tuned using factors available for each of them (e.g. time boundaries, type of chart displayed). 1 IDP security policy objects can also be imported from the Check Point FireWall-1 management workstation using CPMI API protocol (OPSEC ). 2003 CLICO LTD. ALL RIGHTS RESERVED 16