Using SYN Flood Protection in SonicOS Enhanced

Transcription

1 SonicOS Using SYN Flood Protection in SonicOS Enhanced Introduction This TechNote will describe SYN Flood protection can be activated on SonicWALL security appliance to protect internal networks. It will also provide a background on this type of attack, how SYN Flood works in SonicOS Enhanced 3.1, and how to properly configure the feature. SYN Flood protection is available in SonicOS Enhanced versions 3.1 and newer. It is not available in any version of SonicOS Standard. Recommended Versions SonicOS Enhanced 3.1 or newer Customers with current service/software support contracts can obtain updated versions of SonicWALL firmware from the MySonicWALL customer portal at Updated firmware is also freely available to customers who have registered the SonicWALL device on MySonicWALL for the first 90 days. Overview What is a SYN Flood? SYN Floods are a common form of denial-of-service attacks launched against IP based hosts, designed to incapacitate the target by exhausting its resources with illegitimate TCP connections. SYN Flood protection helps to protect hosts behind the SonicWALL from Denial-of-Service (DoS) or Distributed DoS attacks that attempt to consume the host s available resources by sending TCP SYN packets with fake IP addresses, or by otherwise creating excessive numbers of half-opened TCP connections. A SYN Flood attack is considered to be in progress if the number of unanswered SYN/ACK packets sent by the SonicWALL (half-opened TCP connections) exceeds the threshold set in Attack Threshold (incomplete connection attempts / second); the default value is 300, the minimum is 5, and the maximum is 999,999. This large range is provided for future scalability and exceeds the practical maximum for existing products; in the current firmware, the maximum you can set is 200, 000. SYN Flood attacks attempt to flood targeted devices/servers with spoofed TCP connection SYNs, such that the targeted device s ability to respond to legitimate traffic is severely degraded. The attacking machine usually produces a TCP packet with random source address and port, making discrimination of SYN flood traffic vs. legitimate traffic rather problematic. SYN Flood attacks are often generated from numerous machines simultaneously usually the product of a widespread virus that has infected an unsuspecting host, or hosts. The method of SYN flood protection employed starting with SonicOS Enhanced 3.1 uses stateless SYN Cookies, which increases reliability of SYN Flood detection, and also improves overall resource utilization on the SonicWALL. A typical TCP handshake (simplified) begins with an initiator sending a TCP SYN with a 32-bit sequence (SEQi) number. The responder then sends a SYN/ACK acknowledging the received sequence (by sending an ACK equal to SEQi+1) along with its own hard-to-predict random 32-bit sequence number (SEQr); the responder also maintains state awaiting an ACK from the initiator. The initiator s ACK should contain the next sequence (SEQi+1) along with an acknowledgement of the sequence it received from the responder (by sending an ACK equal to SEQr+1). The exchange looks as follows: 1 Initiator -> SYN (SEQi= , ACKi=0) -> Responder 2 Initiator <- SYN/ACK (SEQr= , ACKr= ) <- Responder 3 Initiator -> ACK (SEQi= , ACKi= ) -> Responder Because the responder has to maintain state on all half-opened TCP connections (that is, TCP connections that did not transition to an established state through the completion of the 3-way handshake) it is possible for memory depletion to occur if SYNs come in faster than they can be processed or cleared by the responder. When the SonicWALL is between the initiator and the responder, it effectively becomes the responder brokering or proxying the TCP connection to the actual responder (private host) it is protecting. With stateless SYN Cookies, the SonicWALL does not have to maintain state on half-opened connections. Instead, it uses a cryptographic calculation (rather than randomness) to arrive at SEQr.

2 The entire TCP connection sequence for proxied connections is this: 1 Client [SYN] SW 2 Client [SYN/ACK (0 window)]--- SW 3 Client [ACK] SW 4 Client SW ---[SYN(0 window)]--- Server 5 Client SW [SYN/ACK] Server 6 Client [ACK (server window)]--- SW [ACK (clnt window)]- Server Caveats When using the Proxy WAN client connections when attack is suspected mode, these options should be set very conservatively, since they will only affect connections when a SYN-Flood attack is taking place. This ensures that legitimate connections can proceed during an attack. Use of the Proxy All WAN Client Connections mode will cause the SonicWALL to respond to port scans on all TCP ports, as the SYN-Proxy feature forces the SonicWALL to respond to all TCP SYN connection attempts (legitimate or not). When using the MAC Blacklisting feature, it is recommended that the Never blacklist WAN machines option is checked, as leaving it unchecked may interrupt traffic to/from the SonicWALL s WAN port(s). SYN Flood Protection Methods In respect to a firewall, SYN flood attacks may originate from either trusted (internal) or untrusted (external) networks. Attacks from untrusted WAN networks will usually be attacking one or more servers protected by the firewall, or the firewall s WAN interfaces. Attacks from the trusted (LAN/DMZ/ ) networks are usually the result of a virus infection inside one or more of the trusted networks, and will probably be attacking one or more local or remote hosts. To provide a firewall defense to both attack scenarios, SonicOS 3.1 Enhanced provides two separate SYN flood protection mechanisms: Layer 3 SYN Flood Protection (SYN-Proxy) shields inside servers from WAN-based SYN flood attacks, using a SYNproxy implementation to verify the legitimacy of connecting WAN clients before forwarding the connection request to the protected server. Layer 2 SYN Flood Protection (SYN Blacklisting) is used to blacklist individual machines generating (or forwarding) SYN flood attacks. Layer 3 SYN-Proxy is enabled only on WAN interfaces, while Layer 2 SYN Blacklisting may be enabled on any interface. Each mechanism provides several options for customization. In addition, SYN Flood related statistics are gathered and displayed, and detailed log messages are generated for significant events related to SYN-Flood Protection. The internal architecture of both SYN Flood protection mechanisms is based on a single SYN Watchlist. The SYN Watchlist consists of a small dense array containing the Ethernet addresses of the most active machines sending initial SYN packets to/through the firewall. Because this list is based on Ethernet addresses, all SYN traffic is tracked based on the address of the machine forwarding the SYN, regardless of the IP source or destination. Each watchlist entry contains a hit-count. The hit-count is incremented each time an initial SYN is received from the corresponding machine and decremented when the TCP three-way handshake is completed. Assuming a scenario in which no SYN-Flood attack is taking place, the hit-count for any particular machine will equal the number of embryonic half-open connections pending since the last time the hit-count was reset (the hit-count is reset once a second). The thresholds for logging, SYN-Proxy, and SYN-Blacklisting are all compared to these hit-counts when determining if a log message or state-change is necessary. The number of embryonic half-open connections pending at any point in time will vary within a predictable range, depending on the traffic patterns in the associated network. When under SYN-Flood attack, the number of pending half-open connections from the machine forwarding the attacking packets will increase substantially due to the spoofed connection attempts. When the attack thresholds are set correctly, normal traffic flow should produce few, if any, attack warnings or actions, but the same thresholds should detect and deflect attacks before they result in serious network degradation. 2

3 In addition to the SYN Watchlist, the SYN Blacklisting implements a SYN Blacklist. This is similar to the SYN Watchlist, but contains machines that have exceeded, and continue to exceed, the SYN Blacklist attack threshold. Packets from blacklisted machines are discarded early in the packet processing, and so they can be handled in greater quantity, providing a defense against attacks originating on local networks, while also providing a second-tier of protection for WAN networks (when WAN Blacklisting protection is enabled). Machines cannot exist on the SYN Blacklist and Watchlist simultaneously. When blacklisting is enabled, machines exceeding the blacklist threshold are removed from the Watchlist, and placed on the blacklist. Conversely, when a machine is removed from the Blacklist, it is immediately placed back on the Watchlist. Any system whose MAC address has been placed on the Blacklist will be removed from it approximately three seconds after the flood emanating from that Machine has ended. Do I Need SYN Flood Protection? It is entirely possible that your network does not need this option, and can safely leave it off which is the default setting. Many networks never come under a SYN Flood attack, from either an internal or an external source. While SYN Flood Protection is an effective and powerful tool for protecting your networks, it does have some potential performance limitations, which are detailed below in the configuration sections. SonicWALL recommends leaving SYN Flood Protection disabled unless you determine that your network requires it. Configuring SYN Flood Protection The SYN Flood Protection section of the Firewall > TCP Settings page provides the following options, divided into two sections: Layer 3 SYN Flood Protection SYN Proxy The SYN Flood Protection Mode has three drop-down options: Watch and report possible SYN Floods this option is the default recommended setting, and allows the SonicWALL security appliance to monitor SYN connections on all interfaces and log suspected SYN flood activity based upon the specified attack threshold. SYN-Proxy is never turned on, so the TCP three-way handshake is forwarded without modification (other than NAT). Proxy WAN client connections when attack is suspected this option allows the SonicWALL security appliance to trigger SYN-Proxy on WAN interfaces when the specified number of incomplete connection attempts per second is exceeded. This method ensures that legitimate traffic is processed even in the midst of an attack. Proxy-Mode will remain in affect until all WAN SYN-floods have ceased (or have been blacklisted). If your network is having issues with SYN Flood attacks from internal or external sources, this is the recommended setting. 3

4 Always proxy WAN client connections this option sets the SonicWALL security appliance to always use SYN-Proxy. While this method blocks all spoofed SYNs from crossing the SonicWALL, it is an extreme security measure and is not recommended except in high-risk environments. IMPORTANT NOTE: Use of this feature will cause the SonicWALL to respond to port scans on all TCP ports, as the SYN-Proxy feature forces the SonicWALL to respond to all TCP SYN connection attempts (whether legitimate or not). While these ports are not actually open, per-se, it may not be a desirable side-effect in environments that are frequently targeted for attack, or for environments that have scheduled network security audits. If this option is activated, it will also generate false-positives when using port-scan testing software. This automatic response to port scans is not a security vulnerability. In fact, this thwarts port scan attacks by obfuscation. The SYN Attack Threshold subsection has two options. The SonicWALL security appliance gathers statistics on WAN TCP connections, keeping track of the maximum and average maximum and incomplete WAN connections / second and will use statistics to suggest a value for the SYN flood threshold. Checking the Use the value calculated from gathered statistics box will autopopulate the Attack Threshold (incomplete connections / second entry field. This checkbox is used only to autopopulate the field and does not stay checked. The field can also be manually populated by default it has a default value is 300, the minimum is 5, and the maximum is 999,999 (as noted on page one, in the current firmware the maximum you can set is 200, 000). If this feature is used, it s recommended that the SonicWALL run for several days with normal traffic loads so that the device may suggest a threshold based on a complete statistical sample. SYN-Proxy Options When TCP connections are proxied, the firewall responds to the initial SYN with a manufactured SYN/ACK reply, waiting for the ACK in response before forwarding the connection request to the inside server. Machines attacking with SYN- Flood packets will not respond to the SYN/ACK, and their spoofed connection attempts will be blocked by the firewall. With SYN-Proxy, the firewall must manufacture the SYN/ACK response, without knowing how the server will respond in regards to the TCP options normally provided on SYN/ACK packets. Of particular significance are the maximum TCP MSS and SACK option. To provide more control over the options sent to WAN clients when in SYN-Proxy mode, the user may control these two options with the following: The All LAN/DMZ servers support the TCP SACK Option setting, when checked (and when WAN clients include this option on their initial SYN requests), will force the SYN-Proxy to include the SACK option in response to those clients. This box should only be checked when it is known that ALL servers behind the SonicWALL accessed from the WAN support the SACK option, as the SonicWALL has no way to determine that the systems it is proxying the connection for are capable of supporting this option. The Limit MSS sent to WAN clients (when connections are proxied) setting provides a limiting MSS to be sent to WAN clients when connections are proxied. This prevents WAN clients from sending TCP segments that may be too large for a targeted server. For instance, if the inside server is an IPSec gateway, it may need to limit the MSS it receives to a provide space for IPSec headers when tunneling traffic. As with the SACK Option setting, the SonicWALL cannot predict the MSS value that will be sent by the Server when it responds to the SYN manufactured during the proxy sequence. So this option lets network administrators control the manufactured MSS value sent to WAN clients. The Maximum TCP MSS sent to WAN clients field is used to enter the max MSS described above. If the user specifies an override value, that value, or something smaller, will be sent to the client in the SYN/ACK cookie. This should be a worst-case value, since it is global for all proxied connections. Please use caution with this setting, as setting too low or too high a value will cause performance issues. Setting this value too low can decrease performance only when SYN- Proxy is always enabled or triggered by the threshold. Setting this value too high can break connections if the server subsequently responds with a smaller MSS value, and the associated TCP segments cannot be fragmented. IMPORTANT NOTE: When using the Proxy WAN client connections when attack is suspected mode, these options should be set very conservatively, since they will only affect connections when a SYN-Flood attack is taking place. This ensures that legitimate connections can proceed during an attack. 4

5 Layer 3 SYN Flood Protection SYN Proxy SYN Blacklisting may be enabled or disabled, regardless of the SYN Proxy configuration, by checking or unchecking the Enable SYN Flood blacklisting on all interfaces option. The threshold for SYN flood blacklisting (SYNs / Sec) entry should be quite a bit larger than the SYN-Proxy threshold, since blacklisting is intended to thwart more vigorous local attacks, or particularly severe attacks from a WAN network. By default it is set to a value of The Never blacklist WAN machines option ensures that WAN-side systems are never added to the SYN Blacklist. This option is recommended, as leaving it unchecked may interrupt traffic to/from the SonicWALL s WAN port(s). For example, if a system on the public Internet launches a SYN flood against a target behind the SonicWALL, and that attack exceeds the SYN flood blacklisting threshold, the SonicWALL will immediately block the MAC address of the source, which most of the time is going to be the SonicWALL s upstream gateway. If it does this, then the SonicWALL will no longer pass traffic to/from this device effectively ceasing communications to/from the public Internet until the device is removed from the MAC Blacklist. The Always allow SonicWALL management traffic option causes IP traffic from a blacklisted machine targeting the SonicWALL s WAN IP address(es) to not be filtered. This allows management traffic, and routing protocols, to maintain connectivity through an otherwise blacklisted machine. This setting is particularly useful in environments where the SonicWALL is managed by SonicWALL s Global Management System (GMS). 5

6 SYN Flood Statistics The TCP Traffic Statistics section on the Firewall > TCP Settings page has a number of entries related to the SYN Flood feature: Max Incomplete WAN Connections / sec This is the maximum number of pending embryonic half-open connections recorded since the firewall has been up (or since the last time the TCP statistics were cleared). Average Incomplete WAN Connections / sec This is the average number of pending embryonic half-open connections, based on the total number of samples since boot (or the last TCP statistics reset). SYN-Floods in Progress The number of individual forwarding machines that are currently exceeding either SYN-Flood threshold Total SYN-Floods detected The total number of events in which a forwarding machine has exceeded the lower of either SYN-Flood threshold. TCP connection proxy-mode (WAN only) Indicates whether or not Proxy-Mode is currently on for WAN interfaces. Current SYN-Blacklisted Machines The number of machines currently on the Blacklist. Total SYN-Blacklisting Events The total number of times any machine has been placed on the Blacklist. Total SYN Blacklist Packets Rejected # of packets dropped due to the Blacklist. Created: 03/10/2005 Updated: 05/14/2008 Version 1.2 6