What's in the data bucket?

Transcription

1 What's in the data bucket? Event Correlation and SIEM Vendor Approaches STI Joint Written Project Authors: Brough Davis, Jim Horwath, John Zabiuk Advisor: Stephen Northcutt Accepted: April 12, 2010 Abstract As technology progresses IT professionals and Security analysts are presented with an ever increasing volume of data to parse through to find evidence of security events. Many companies rely on disparate logging architectures that split network, server, and application logging. Each of these logging architectures are often isolation from each other. The security threats to a company are always increasing and it is becoming imperative for a company to have a well designed logging infrastructure that consolidates, archives, and correlates all useful logging information from as many parts of a company's network as possible. Not only is it becoming imperative to stop intrusions from both external and internal attackers but it is crucial for protecting critical information from getting into the wrong hands. Many vendors are starting to offer appliances called SIEM's (Security Information Event Management) that claim to provide these functions. Are these appliances a silver bullet to help protect companies? STI Joint Written Project - May

2 1. Executive Summary This purpose of this paper is to discuss how a Security Information and Event Management system can prevent or mitigate data loss by detecting both network intrusions and extrusions. Such losses of data can cost organizations in the millions of dollars to recover from some organizations may never recover. Several key area will be examined with the intent to discuss how different vendors SIEM products can mitigate these areas. The areas discussed are: Identification of infected systems trying to exfiltrate information Countermeasures to detect attempts to infect internal systems Detection of outbound sensitive information Mitigation of the impact of infected systems In order for an organization to benefit the most from a SIEM, a great deal of data must be gathered from numerous devices and applications throughout the network. These devices could range from a simple router to a complex business-critical application. This paper will discuss these various sources of log data and the service used to collect that data. Once data has started being collected, the next task that an organization must tackle is how to correlate the different logs in order to identify where and when an attack is occurring or has occurred. Without additional log analysis software, this can be a daunting, if not impossible task. To that end, the use of a SIEM to detect and prevent attacks will be examined in detail. Several different SIEM vendors have agreed to share the approach they take to address the identified key areas for discussion. The vendors providing input to this paper are, in alphabetic order: LogRythm Nitro Security Prism Microsystems QRadar (by Q1Labs). The methods each vendor solution deals with different attacks will be discussed in detail. Finally a summary will be provided in the form of a comparison chart outlining the ability of each solution to handle specific issues. As well, a relative comparison of cost will be included in the comparison. STI Joint Written Project - May

3 2. Problem Description Over the past decade, network intrusions and subsequent data loss have become more and more frequent. The cost of data loss to an organization may be small, but it can also be very large. The Ponemon Institute produces an annual report, The US Cost of Data Breach, which outlines various aspects of the costs associated with data breach and data loss. According the 2009 US Cost of Data Breach Study, the average cost of insider data breaches in 2009 was $6.7 million per incident. This is an increase from $6.65 million in The cost per customer record rose from $202 to $208. The most expensive data breach included in the 2009 report cost nearly $31 million to resolve. The lease expensive data breach cost $750,000 to resolve. Not only will such data breaches impact an organization in present day costs, but also in terms of future revenue. According to the Ponemon Institute, 20% of consumers terminated a relationship with a company after being notified of a security breach. The purpose of this paper is to discuss how a Security Information and Event Management (SIEM) systems can be utilized to detect intrusions and extrusions, or data loss. It will provide guidance that organizations can use to accomplish actionable results using a SIEM. In specific, the following areas will be discussed: Identification of infected systems trying to exfiltrate information Countermeasures to detect attempts to infect internal systems Detection of outbound sensitive information Mitigation of the impact of infected systems To assist in the discussion of this topic, the assistance of four SIEM vendors has been enlisted. A discussion of how each vendor s solution addresses each of the afore mentioned areas of discussion will be provided. The vendors assisting with this paper, in alphabetical order, are: LogRythm ( Nitro Security ( Prism Microsystems ( QRadar ( Network Example A typical network design used by small to medium businesses was used as the basis for this report. The network consists of a Microsoft Active Directory environment in two sites with approximately 35 workstations at the primary site, and 25 at the secondary site. All STI Joint Written Project - May

4 workstations are running either Windows 7 or Windows XP. Within the DMZ for this network, web, , and DNS services are provided. With the exception of the web server and the Syslog server, which are running Red Hat Linux, all servers are running either Windows Server 2003 or Each site is utilizing either a Cisco 2800 or Cisco 1800 as the perimiter gateway and first line of defense. Beyond each site perimeter, a Cisco ASA 5510 firewall appliance is used to provide more in-depth protection. To communicate between sites, an IPSec VPN is established between the perimiter router at each site. Although the primary use of this network is for static users, there are several that require VPN access from remote locations. This is accomplished with the Cisco VPN client connecting to the ASA The following is a diagram of this network. 3. Logging Sources In order to get the greatest benefit out of a SIEM system it's important for a company to review all the different logging sources across Network devices, Servers, Workstations, and business critical applications. The following sections review common configurations for enabling logging from each of these four areas. STI Joint Written Project - May

5 Importance of Time Before logging configurations are described it's important to note that in order to accurately correlate multiple logs from different sources the time stamps in the logs must be consistent between each other. For example, an attacker may find an exploit on a DNS server and then create user accounts and start downloading attack tools. These actions could be tracked in netflow data, syslog data, and IDS log data. If the clocks on the logging sources were not consistent then then comparing which event happened first and how much time passed between events would make correlating the events very difficult. In effect bad time sources could destroy any correlation attempts and treat each event as singular non-related event. This is especially bad if each event by themselves is not considered a cause for alarm. To ensure that all logging sources have identical clock times it is highly recommended that NTP (Network Time Protocol) be used across all devices. Using an internal GPS/CDMA based stratum 1 appliance is recommended in medium/large networks. Using ntp.pool.org systems can help in small/medium size networks. Monitoring the NTP state on systems is also recommended to ensure devices maintain connectivity to their NTP servers. Timezone It's common to configure a devices Timezone to the local time the Network/System Admins work in but this may create problems if not all the devices are configured with the same timezone. If devices are putting timestamp information in the logs this may create correlation problems on a central logging or SIEM system. If the SIEM system can not convert the timezones then the log files can not be successfully correlated. This problem can be handled in different ways; the device sending logs does not send timestamp information and the logging server itself timestamps the message when recieved, or one timezone is used across all equipment (eg. UTC). Using UTC as the standard timezone is sometimes preferred as all other timezones shift from UTC which is easier to reference in a global company. UTC also avoids daylight savings time (DST) software bugs that commonly surface from time to time. Syslog and SNMP Trap Many of the events are sent to both syslog and snmptrap. SNMP Trap logs have a long history with being integrated into alarm management to alert to specific conditions network administrators want to be alerted to. Syslog is good at sending all event information in order to help provide a good picture of the condition of a network device at a given time. SNMP Trap configurations traditionally also have related RMON threshold configurations that allow network admins to have SNMP Trap messages sent when certain conditions (eg. CPU Utilization) meet a threshold value (eg. 75%). Having both SNMP Trap and Syslog STI Joint Written Project - May

6 messages being sent from network devices allows for both good device state information as well as specific alarm criteria in order to help build a better message archive that can be used to correlate against to weed out possible malicious activity. Network One of the first steps a company must take in the pursuit of a robust logging infrastructure is to audit the network perimeter around criticial systems. Having a good understanding of all the network devices allows for better deployment and configuration management of logging. By deploying consistent logging configurations across all equipment a company can get the most benefit out of log correlation. Common network devices on a network may include but are not limited to the following: Router Switch Firewall Modems IDS/IPS Wireless AP Vulnerability Scanner Some examples of these device and their logging configurations will be described below. Cisco IOS Cisco Firewall PIX/ASA/FWSM Snort Intrusion Detection System (IDS) Cisco IOS Folloing configurations are identical across most Cisco devices running IOS code 12.x such as the 3750 switch, 2900 Router,1800 Router, and Airopoint 1400 Wireless Access Point. Cisco IOS can provide syslog, snmp trap, and netflow Syslog Configuring syslog on Cisco devices running IOS 12.x is a pretty standard configuration. Some configuration options are useful if you want to enable which severity level, including the hostname and device timestamp in the logs. Warning: Enabling debug level syslog messages to console may cause high CPU utilization on a device if the logs are verbose which may render a device unusable. Making sure console level logging is low or turned off is a good idea during standard operation. STI Joint Written Project - May

7 no logging console logging trap debugging logging source-interface Loopback0 logging SNMP Trap Cisco snmptrap configuration is fairly straightforward. Many events that are logged in syslog events has an equivalent snmptrap event. A common mistake is to enable all snmptraps. This ends up sending every possible event to a monitoring system which across a large infrastructure may overload a monitoring system or even cause network congestion on 'bottle neck' segments. Knowing the type of events to alarm on is not only conserves resources but also helps focus the type of events that are critical to the network and the company. The below example sends snmptrap messages to for config changes, bgp events, and AAA events. Note that global traps are enabled for BGP and config events but the AAA events are specific to the host only. snmp-server host public config bgp aaa snmp-server enable traps config snmp-server enable traps bgp Netflow Netflow data will provide an important view into operations on your network. Netflow data, also known as session data reports on conversations between two systems. Netflow data will show connections to malicious sites, conversations using protocols that violate security policies and/or best practices, along with conversations of long durations. In an environment leveraging a SIEM, correlation using Netflow data can improve monitoring, alerting, and reporting capabilities. Netflow data can help analysts identify targets of attacks, and identify malicious sites targeting your network. The following example configures Netflow v9 to export ingress traffic on interface serial 3/0/0 to the Netflow collector( ) on UDP port ip cef ip flow-export version 9 ip flow-export destination interface serial 3/0/0 ip flow ingress NTP STI Joint Written Project - May

8 NTP configuration for the Cisco IOS can be easily enabled but has additional security options that are sometimes over looked. Basic NTP messages can be easily spoofed, easy to enumerate information such as internal NTP servers, and be open to DoS attacks to the NTP service or router itself. Enabling NTP key authentication, restricting NTP servers and clients with access-lists, and disabling the NTP service on public interfaces will reduce the possiblity of NTP manipulation from attackers. The below example has a router configured to use 3 ntp servers, restrict any ntp updates from only those 3 servers, allows the network /24 to use the router as a NTP server, source NTP messages from the Loopback0 interface, disables NTP services on public serial 0/0 interface, and has NTP authentication enabled. Router(config)#ntp server Router(config)#ntp server Router(config)#ntp server Router(config)#access-list 20 permit Router(config)#access-list 20 permit Router(config)#access-list 20 permit Router(config)#access-list 20 deny any Router(config)#ntp access-group peer 20 Router(config)#access-list 21 permit Router(config)#access-list 21 deny any Router(config)#ntp access-group serve-only 21 Router(config)#ntp source Loopback0 Router(config)#ntp authenticate Router(config)#ntp authentication-key 10 md5 MySecretKey Router(config)#ntp trusted-key 10 Router(config)#interface serial 0/0 Router(config-if)#ntp disable WARNING: Configuring NTP authentication does not require all clients to use NTP authentication; it enables clients to use authentication. Your router will still respond to unauthenticated requests, so be sure to use ACLs to limit NTP access. Cisco PIX/ASA/FWSM The Cisco PIX/ASA firewall appliances and Catalys 6500 service modules have basic syslog and advanced syslog configuration options. Basic SNMP trap configuration options are also available. Basic Syslog There are a handful of syslog options that need to be taken into account such as including timestamps, host-id, and standby log information in addition to the standard logging STI Joint Written Project - May

9 serverity and syslog host. The below example enables syslog, includes timestamp, includes hostname, and sends debug level messages to facing the inside interface. Also note that notification severity logs are sent to the system buffer which will not include verbose connection entries but useful access-list deny log messages. logging enable logging timestamp logging standby logging buffered notifications logging trap debugging logging device-id hostname logging host inside Warning: Debug mode will log every connection traversing the firewall which will drastically increase the amount of syslog messages going to the syslog server. Proper disk storage capacity and I/O speeds will have to be in place in order allow for enabling debug level logging. Debug level logs may also increase CPU utilization on the firewall as well as additional bandwidth resources to the syslog server. It is recommended to use the advanced syslog option if specific debug level output is needed without sending all debug logs. Advanced Syslog PIX/ASA 7.0 provides several mechanisms that enable you to configure and manage syslog messages in groups. These mechanisms include message severity level, message class, message ID, or a custom message list that you create. With the use of these mechanisms, you can enter a single command that applies to small or large groups of messages. When you set up syslogs this way, you are able to capture the messages from the specified message group and no longer all the messages from the same severity. The below example captures all VPN (IKE and IPsec) class system log messages with debugging level or higher in addition to sending all notification level events. hostname(config)#logging enable hostname(config)#logging timestamp hostname(config)#logging list my-list level debugging class vpn hostname(config)#logging list my-list level notification hostname(config)#logging trap my-list hostname(config)#logging host inside SNMP Trap Cisco firewall SNMP traps typically are only used for standard traps: authentication, cold start, link up and link down. While these alarms are typically only used for hardware STI Joint Written Project - May

10 malfunctions it can also be useful in situations where a security event maybe related(eg. attacker brute forcing an ssh login). The below configuration example sends SNMP traps from the inside interface to a SNMP trap server with the SNMP community string of 'supersecret' snmp-server host inside trap community supersecret NTP PIX/ASA devices can only be configured as NTP clients and are not able be used as a NTP server time source. The following configuration example has the PIX/ASA using the NTP server facing the inside interface using a md5 hashed passphrase of 'supersecret' for authentication. ntp authentication-key 1 md5 supersecret ntp trusted-key 1 ntp server key 1 source inside Snort IDS/IPS Intrusion Detection Systems are a valuable source of logging information for adding value for log correlation. A common opensource IDS is called SNORT. Snort has many different logging outputs. To send alerts to syslog, use the -s switch or by using the alert_syslog output variable in the snort.conf file. The default facilities for the syslog alerting mechanism are LOG AUTHPRIV and LOG ALERT. The below snort.conf example configuration file wil send snort alerts to the local syslog local6 facility. The syslog or syslog-ng configurations can then send the alert messages to a remote server....#end of file output alert_syslog: log_local6 log_alert output alert_fast: alert Servers/Workstations The majority of workstations common to many business environments are using Microsoft Windows Operating Systems such as Windows XP, Vista, and now Windows 7. Enterprise Linux as well as Microsoft Windows 2003/2007 Server distributions are popular in many server environments. STI Joint Written Project - May

11 Enterprise Linux 3/4/5 Enterprise Linux distributions such as Red Hat Enterprise Linux and Centos 3,4,5 are common Linux server operating systems. System messages can be sent to a central logging system using common opensource logging applications such as syslog, syslog-ng, rsyslog, and snmptrapd. Syslog The syslog application found on most Linux systems is called syslog. The below syslog.conf file entry allows a server to send all severity messages to the remote host named 'logserver'. 'logserver' would be defined in dns or the /etc/hosts file. Syslog-ng Syslog-ng (syslog next generation) was an application that extended syslog with additional features such using TCP and making more advanced filtering options. The below example of the syslog-ng.conf file shows how to send all messages to a remote syslog-ng logging server named 'logserver'. source from_local { unix-stream ("/dev/log"); pipe ("/proc/kmsg" log_prefix("kernel: ")); internal(); }; destination central_log { }; tcp("logserver" port(514)); log { }; source(from_local); destination(central_log); rsyslog rsyslog is one of the more recent logging applications and is the current standard logging application on enterprise linux distributions. The following rsyslog.conf configuration will forward all system messages to a remote server named logserver over TCP port 514. UDP and RELP('reliable protocol') are also optional protocols for sending messages via rsyslog. #TCP STI Joint Written Project - May

12 NTP The NTP application service comes standard on most Linux distributions and is usually enabled to point at ntp.pool.org systems or other common distribution specific NTP servers on the Internet. The following example ntp.conf file shows time being received from the NTP server ntp.pool.org systems. Additional security options to restrict NTP queries and changes via NTP are also included. driftfile /var/lib/ntp/ntp.drift server 1.north-america.pool.ntp.org server 2.north-america.pool.ntp.org server 3.north-america.pool.ntp.org # By default, exchange time with everybody, but don't allow configuration. restrict -4 default kod notrap nomodify nopeer noquery restrict -6 default kod notrap nomodify nopeer noquery # Local users may interrogate the ntp server more closely. restrict restrict ::1 Microsoft Windows Windows 2003/2008 Server Windows XP/Vista/7 Microsoft Windows systems typically use third party applications to send event log information to a remote server. One common application used on many Windows Operating Systems is called SNARE. Below is a picture of network configuration section for SNARE in which local logs can be sent to a remote server(snare,syslog,syslog-ng,etc). STI Joint Written Project - May

13 Snare Agent includes a MSWinEventLog tag before each message. Match this tag with a program filter in syslog-ng.conf, and use flags(final) to route the logs from Windows into a custom file before any subsequent destination. filter windows { program(mswineventlog); }; destination windows { file("/var/log/archive/windows/$r_year/$r_month/$r_year-$r_month-$r_day" template("$isodate <$FACILITY.$PRIORITY> $HOST $MSG\n") template_escape(no) ); }; log { source(local); filter(windows); destination(windows); STI Joint Written Project - May

14 flags(final); }; NTP Windows desktops can be configured to get time information from NTP services through the Control Panel->Date Time settings as seen in the below example. Applications Many companies have a difficult time tracking all the different applications that run in their network. It's important that a company understands the applications that are critical to the business in order to assess what logging can be beneficial from those applications. Below are some commonly found applications and their respective configurations in order to enable logging to a remote syslog server. Apache BIND Microsoft Exchange Microsoft SQL Tripwire STI Joint Written Project - May

15 Apache Apache is a common web server application found on many different Operating Systems. The main configuration file used for apache is httpd.conf. Error events and Access events are treated separately and usually log to different locations. Error Logs The error log is usually written to a file (typically error_log on UNIX systems and error.log on Windows and OS/2). On UNIX systems it is also possible to have the server send errors to syslog or pipe them to a program. ErrorLog syslog:local1 Sending apache logs to syslog requires a local syslog daemon to relay the syslog messages to a central syslog server. Access Logs Unfortunately, only the error log has this feature built in. It's extremely useful to also have your access logs logged to a remote server, for the reasons described earlier. There is a technique to allow you to log your access log to syslog as well. However, as with any other article of this nature, we encourage you to check the Apache web server documentation site because this feature may be built-in at some point in the future, rendering this technique obsolete. At the moment, here's what you need to do. This is a two-step process. First, create a script that is capable of sending entries to syslog: #!/usr/bin/perl use Sys::Syslog qw( :DEFAULT setlogsock ); setlogsock('unix'); openlog('apache', 'cons', 'pid', 'local2'); while ($log = <STDIN>) { syslog('notice', $log); } closelog Second, point your access log at this script using the piped logfile syntax: CustomLog /usr/local/apache/bin/apache_syslog combined STI Joint Written Project - May

16 BIND BIND (Berkeley Internet Name Daemon) is a common DNS application service. BIND can log events to syslog fairly easily by adding the following options to the named.conf configuration file. logging { channel syslog_chnl { syslog local1; severity info; }; }; Sending bind logs to syslog requires a local syslog daemon to relay the syslog messages to a central syslog server. Microsoft Exchange Microsoft Exchange server logs can be sent to remote logging server using many different applications available. A common application used is called Snare. Snare Epilog for Windows is a program that facilitates the central collection and processing of Windows text-based log files. Epilog for Windows also supports date stamped log files such as IIS, ISA, SMTP and Exchange message tracking logs. Log information is converted to tab delimited text format, then delivered over UDP to a remote server. The following picture describes where the log format options are configured in snare. STI Joint Written Project - May

17 Microsoft SQL One method for sending Exchange and MSSQL event logs to a remote server is to use the SNARE application for MSSQL. The development of 'SNARE for Microsoft SQL Server' allows events generated by MS SQL to be collected and forwarded to a remote audit collection facility. The SnareMSSQL service can be configured to monitor a variety of MSSQL configurations. The default is to monitor the master database within the default local MSSQL instance. This can be modified on a per objective basis to specify a named MS SQL instance and a database within that instance. The below example shows the configuration page for setting which database instance and logging information to send to the remote logging server. TripWire Host intrusion detection is an important piece to every network that deploys a defense in depth methodology. Effectively correlating log events requires using both network events and host based events. There are different host intrustion detection system (HIDS) applications for both Windows and Linux operating systems. One application that is common to both Windows and Linux is called Tripwire. STI Joint Written Project - May

18 To enable tripwire to log to syslog requires changing the "SYSLOGREPORTING" variable in the twcfg.txt configuration file. The variable is describe in more detail below. SYSLOGREPORTING If this variable is set to true, messages are sent to the syslog for four events: database initialization, integrity check completions, database updates, and policy updates. The syslog messages are sent from the "USER" facility at the "NOTICE" level. The following log messages are examples of 4 different types of logging messages found in the current Linux opensource Tripwire 2.3 version. Jun 18 14:09:42 lighthouse tripwire[9444]: Database initialized: /var/lib/tripwire/test.twd Jun 18 14:10:57 lighthouse tripwire[9671]: Integrity Check Complete: TWReport lighthouse V:2 S:90 A:1 R:0 C:1 Jun 18 14:11:19 lighthouse tripwire[9672]: Database Update Complete: /var/lib/tripwire/test.twd Jun 18 14:18:26 lighthouse tripwire[9683]: Policy Update Complete: /var/lib/tripwire/test.twd The letters in the Integrity Checking log correspond to # of violations, maximum severity level, and # of files added, deleted, and changed, respectively. With any value other than true, or if this variable is removed from the configuration file, syslog reporting will be turned off. Tripwire 4.x and 7.x versions can be installed on windows operating systems as well as AIX. The following are some examples of Tripwire 7.x logs. May 3 10:23:17: TE: Information HostName=tripwire LogId= under Policy 'IBM AIX 5.1 Benchmark - CIS v1.0.1' changed from 'Failing' to 'Gold or bette May 3 14:02:52: TE: Information HostName=tripwire LogId= AssociatedObjects= : , : Msg="Deleted 79 test results and 0 waivers affecting 8 policies[lf]policy MS Windows Serve old.[lf]policy IBM AIX 5.1 Benchmark - CIS v1.0.1: Deleted 104 test results for this polic 3 test results for this policy that were more than 31 days old.[lf]policy MS Windows Serve Windows Server 2003 DC Enterprise Benchmark - CIS v2.0: Deleted 3 test results for this po STI Joint Written Project - May

19 May 3 14:43:56: TE: Information HostName=tripwire LogId= Logging Services SYSLOG Network devices can produce large quantities of log messages in a very short period. Without additional configuration, these messages are displayed on the locally attached console only. Some of these messages are routine, while others may indicate that the device is about to fail. Unless someone is consistently watching the console for these messages, they are lost. This problem is compounded further with each additional device on a network. To help reduce the loss of important log messages, most network devices allow for the forwarding of these messages to a syslog server over an IP network. Developed in the early 1980 s by Eric Allman, syslog was originally designed to work exclusively with Sendmail. Since then, its popularity has increased exponentially and is implemented in nearly all network devices. The current documentation for syslog resides in RFC5424. The syslog facility allows devices to forward log messages over an IP network to a data store on a remote host. Many devices that do not have any other communication means can use this functionality to notify administrators of error conditions. Prior to syslog, each device and application would handle log messages differently. Messages could be written to STDERR, to a file, or to a pipe. There are many uses for the syslog facility ranging from basis log message aggregation to network management and security auditing. The syslog facility is cross-platform based, meaning that non-homogeneous devices can send log messages to a single repository. Syslog provides for the ability of messages to be sorted, either by their severity level or by their source. Messages can be sent to a variety of destinations including log files, users terminals, or remote systems. Once these log messages are stored in a single repository, they can be analyzed by a variety of open source and third party applications. There are numerous implementations of syslog. This section of the report will discuss three of these implementations syslogd, rsyslog, and syslog-ng. Table 1 - Syslog facility names Facility Program using the facility * All facilities except mark STI Joint Written Project - May

20 auth authpriv cron daemon ftp kern local0-7 lpr mail mark news syslog user uucp Security and authorization-related commands Sensitive/private authorization messags cron daemon System daemons ftpd kernel Eight flavors of local message Line printer spooling system sendmail (and other mail applications) Timestamps generated at regular intervals Usenet news syslogd internal messages User processes Obsolete Table 2 - Syslog severity levels (descending severity) Level Meaning emerg Panic situation alert Urgent situation crit Critical conditions err Other error conditions warning Warning messages notice May be worth investigating info Informational message debug For debugging only SYSLOGD Syslogd is the basis for input that many log management systems use. The origins of syslogd begin in BSD, however many other syslog implementations are based on syslogd. The syslog architecture consists of three parts: o Syslogd The logging daemon o Openlog Library routines that submit messages to syslogd o Logger a user-level command to submit log messages from the shell Syslogd is a daemon that runs continuously on a system and is started at boot time. It is not controlled by inetd. Messages by applications are sent to a special file called /dev/log (a UNIX domain socket). Syslogd reads messages sent to /dev/log and, based on its configuration (syslog.conf) file, routes them to the defined destination. STI Joint Written Project - May

21 The syslog.conf file controls the behavior of syslogd. Syslog.conf is a simply formatted text file with the basic format being: selector <Tab> action or facility.level <Tab> action example: mail.info /var/log/maillog This would cause all informational messages from the mail system to be saved in the file /var/log/maillog. Syslogd produces timestamp messages, which are logged if the mark facility appears in syslog.conf to specify a destination for them. These timestamps allow network administrators the ability to establish exact time of an issue. SYSLOG-NG Syslog-ng is an open source implementation of syslog which extends the original syslogd model with content-based filtering, rich filtering, additional flexibility in configuration options, and adds additional features. One such feature is the ability to utilize TCP for the transport protocol to provide reliability. Syslog-ng was introduced in 1998 by Balazs Scheidler as a project to port the existing nsyslogd code to Linux. Of the various extensions to syslogd are the following: ISO 8601 timestamp with millisecond granularity and time zone information Addition of the name of relays in the host fields to allow tracking of the path a message has traversed Reliable transport using TCP TLS encryption Syslog-ng offers much wider functionality that transporting syslog messages. It also provides the following features: Ability to format log messages using variable expansion Use of viable expansion when naming files Ability to send log messages to local applications Ability to message flow-control in network transport Logging directly to a database Rewrite portions of the syslog message with set and substitutable primitives Classify incoming log messages and at the same time extract structured information from the unstructured syslog message Generic name-value support STI Joint Written Project - May

22 Ability to process structured message formats over syslog RSYSLOG Rsyslog is an enhanced version of syslogd. It is licensed under GPL (General Public License). Introduced in 2004 by Rainer Gerhards, the goal of the rsyslog project is to provide a more feature-rich, reliable syslog daemon. Reliability is accomplished through the use of TCP as the transport protocol. Initially, rsyslog did not support reliable-syslog or TCP. Instead, rsyslog supported database integration and enhanced configuration. One of the design goals of rsyslog is to act as a direct replacement for the syslogd daemon. ALTERNATIVES TO SYSLOG Although syslog is essentially ubiquitous among network devices, it is not natively support by Microsoft Windows, however there are numerous 3 rd party applications that allow the use syslog. As an alternative to syslog, two options worthy of note are SNMP and Microsoft Operations Manager. Of these two, only SNMP will be discussed as this is an alternative that can be utilized without the addition infrastructure to the sample environment. SNMP SNMP or Simple Network Management Protocol is an Internet standard protocol designed to facilitate the management of devices on IP networks. SNMP is supported by virtually all network devices, including routers, switches, printers, workstations, servers, modems, uninterruptable power supply (UPS) systems, VOIP phones, HVAC systems, many cellular smart phones, and more. SNMP can be used to simply monitor the health of a network device, or it can be used to even manage and control network devices. SNMP can gather information ranging from basic device status or traffic statistics to device specific information such as air temperature and humidity inside a switch. SNMP is supported natively by MS Windows. SNMP is utilized through a relatively simple set of operations. These operations allow devices to be queried for specific information or for specific parameters to be modified on a device. Through SNMP, for example, one can determine the operating characteristics of a switch interface by querying the switch. One could also use SNMP to shutdown or activate a switchport. The commands that SNMP uses are as follows: get getnext STI Joint Written Project - May

23 getbulk (SNMPv2 and SNMPv3) set getresponse trap notification (SNMPv2 and SNMPv3) inform (SNMPv2 and SNMPv3) report (SNMPv2 and SNMPv3) There are three versions of SNMP. SNMPv1 is the original version of the SNMP protocol. It is defined in RFC SNMPv2 extended the type of information that could be gathered by SNMP and is defined in RFC 3416, RFC 3417, and RFC SNMPv3 is the most recent version of SNMP. SNMPv3 is the first implementation of SNMP that introduces strong security. Whereas SNMPv1 and SNMPv2 pass all information in clear text, SNMPv3 can be configured to encrypt all SNMP packets. SNMPv3 is defined in RFC 3410, RFC 3411, RFC 3412, RFC 3413, RFC 3414, RFC 3415, RFC 3416, RFC 3417, RFC 3418, and RFC SNMP has two components a manager, and an agent. Agents are configured on the devices to be monitored. The manager is the system, typically a Network Management System (NMS) that generates requests for information and receives information being sent from the various devices. SNMP can also be used to send traps. Traps are, essentially, alerts sent from a device to a manager indicating that something has gone wrong. The trap message will include specific information about the fault, allowing the network administrator to more easily resolve the issue. Traps can be equated to syslog messages. There are seven generic traps, however each device has the capability to send more specific traps, as defined by the developer or manufacturer. Table 3 - SNMP Generic Traps Generic trap name (and Meaning number) coldstart (0) Indicates that the agent has rebooted warmstart (1) Indicates that the agent has reinitialized itself linkdown (2) Indicates that an interface on a device is gone down linkup (3) Indicates that in interface on a device has gone up authenticationfailure (4) Indicates that an attempt to query a device with an incorrect authentication credential egpneighborloss (5) Indicates that and EGP neighbor has gone down enterprisespecific (6) Indicates that a trap is enterprise-specific. In other words, it is a trap specific to the device as defined by the developer or manufacturer STI Joint Written Project - May

24 5. Using/Correlating Data Logging Attacks In order to see the value from correlating events it's important to look at an attack from a broader view to understand the goals of an attack. A single event may be only one small piece of an overall attack strategy of an attackers goal. Below are some examples of single events that can be seen across different logging and alarming systems are shown below. Apache Log - worm The following is an example Apache log entry of an attack by the Lupper worm, against the AWStats command-injection vulnerability: [24/Dec/2005:13:02: ] GET /cgi-bin/awstats.pl? configdir= echo;echo%20yyy;cd%20%2ftmp%3bwget%20xx%2eyyy%2ez%2e216%2fnikons%3bchmo d%20%2bx%20nikons%3b%2e%2fnikons;echo%20yyy;echo HTTP/1.1 Certain versions of the awstats program would execute the code "echo%20yyy;cd%20%2ftmp%3bwget%20192%2e168%2e1%2e216%2fnikons%3bchmod%20%2bx%20n in response to this request. This would cause the file at ' /nikons' to be downloaded and stored in the /tmp directory. Then it would be made executable using the 'chmod +x nikons' and finally it would be executed. DLP Below is an example of a SNORT Rule that looks for credit card numbers being transferred in clear text. #alert ip any any -> any any (msg:"et POLICY Credit Card Number Detected in Clear (15 digit dashed)"; pcre:"/ (3[4 7]\d{2} )-\d{4}-\d{4}-\d{3} /"; reference:url, classtype:policyviolation; reference:url,doc.emergingthreats.net/ ; reference:url, POLICY_Credit_Card_Numbers; sid: ; rev:12;) Unusual Traffic Patterns The following Netflow traffic patterns have outbound UDP port 53 traffic that does not match common traffic patterns for the local network. This could be a compromised system sending information out of the company network or it could be normal traffic. STI Joint Written Project - May

25 router#sh ip cache flow IP packet size distribution (435092M total packets): Protocol Total Flows Packets Bytes Packets Active Idel Flows /Sec /Flow /Pkt /Sec /Flow /Flow... TCP-other UDP-DNS UDP-NTP Correlating The above events looked at individually may not raise a high priority alarm. Each event could be treated individually with specific mitigation techniques or even possibly ignored. Looking at each event individually makes it difficult to assess if a strategic attack is occurring and what an attackers goal may be. These combined events together should be treated differently then each event by themselves. Correlating the events gives a company a better picture of the an overall attack strategy used by an attacker. This knowledge can allow a company to quickly adopt an overall defense strategy very quickly. Hopefully the many products offered by SIEM vendors can effectively leverage event correlation and help a company better protect itself. Before We Begin This paper assumes you the user have already done your due diligence regarding the selection of a SIEM. The SIEM technology meets your business needs, and you already have use cases you need to address with the technology. There is one item to verify before we go any further with the discussion of using and correlation of data. A company needs to have a corporate policy defining appropriate and inappropriate computer use. A behavior in one environment may violate company policy or federal regulations, but in another, the policy may be less stringent and allow questionable actions. The typical college campus comes to mind. A sound corporate policy will drive many practical uses of a SIEM. This section assumes you already have a company security policy that your SIEM will support. SIEM correlation rules apply logic to data and make decision based on the logic. Without STI Joint Written Project - May

26 applying context, content, and other external factors does not always derive the correct answer. This is why the analyst plays an important role. What is a SIEM? A SIEM is a huge bucket of data available for searching and reporting of normalized data. A SIEM is a huge repository of data stored in a normalized format. Normalization of data is the process of converting log data from its native form into a common format. This common format empowers users the ability to search for common events across disparate systems. Being able to make sense and business use of the data is the magic of a SIEM analyst. This section of the paper will discuss how to leverage log data to address the following use cases: Identify common attacks and what logs entries they produce Prevention of in-bound attacks (both external/internal) Prevention of out-outbound data loss (DLP) Identifying Infected systems Mitigating against infected systems Use and misuse of privileged accounts Access to sensitive information (database information such as payroll) Access to Enterprise-wide Data and Events One of the strengths of the SIEM is the ability to quickly view, navigate, find and report on events in an environment. Collection of data into a common system with the ability to view and prioritize events can solve many security challenges facing organizations today. Below is an example of a view from a SIEM. There is a wealth of information waiting for the analyst to use. STI Joint Written Project - May

27 Overview screen from QRadar Vendor Setups When preparing for this section the team collaborated with four vendors to address the use cases and provide the team with additional council on how to best leverage a SIEM in the fictional environment. The team prepared a survey to gather information from the vendors concerning their products. As part of the survey, the vendors made recommendations of how much product would address the company s business need and how their product would fit into a fictional company network. The team allowed the vendors to configure their products in the fictional company to produce optimal results while using a reasonable budget. The vendors collaborating with the teams are: Log Rhythm ( QRadar ( Prismmicrosystems ( Nitro Security ( STI Joint Written Project - May

28 Proceed with Caution A strength and curse of a SIEM is the deep insight into an environment that before SIEM implementation was not possible. The SIEM allows the collection and viewing of data that most environments rarely had access to before the arrival of a SIEM. Unless your environment is unique, the first thing you will notice about your environment is the amount of operational crud that needs attention and correction. Most companies uncover a plethora of broken process and policy violations. This operational noise provides the perfect cover for nefarious activities to fly under the radar. Although the cleanup of these policy violations and broken processes can be arduous, it is well worth it. Operational issues can quickly become outages and resource drains. The next warning concerns sales staff who may tell you a SIEM runs itself. Although our team did not hear this from any of the vendors participating in our study, there are many horror stories from people who were told, Turn it on, point log sources at it and it will run itself. A SIEM or any tool is only as good as the person using it. The analyst applies business knowledge and context to events in the environment. Finally a SIEM can quickly overwhelm an analyst once the data starts flowing. The temptation to send everything to a SIEM is common, however, after a short period most users realize they need to prioritize the data. One of the collateral effects of sending large amounts of log data to a SIEM, is having to weed through volumes of meaningless information. With tuning, business knowledge, corporate policies, and product knowledge, the analyst starts to recognize the useful data from noise. Identify common attacks and what logs entries they produce Brute force login attempts are an old school way of breaking into a computer system. There are numerous studies concerning the dangers of weak passwords chosen by users. Many users prefer convenience over security, a practice that places organizations at risk. The report below shows the server at IP address having 343,906 failed logins from the user baduser. This report is for the period spanning April 30 to May 5. Further investigation would reveal either a broken process trying to connect to the server, or a malicious user trying to connect to the server by brute force password guessing. Regardless of the root cause, this event requires further investigation and resolution. This report may also reveal a policy violation of an account that does not have a lockout threshold after so many failed attempts. STI Joint Written Project - May

29 Nitro Security login failed report Identify common attacks and what logs entries they produce The view below shows a correlated incidents dashboard summarizing potential attacks (by the correlated events they generate), bound to the source and destination IPs associated with the events, and then allowing the user to see the individual log events in the Events window at the bottom. Nitroview employs a technique called data binding which allows live linking of queries between component windows to allow on-demand forensic investigation. STI Joint Written Project - May

30 Nitro Security Common Attacks Identify common attacks password guess/login failure, alerting via STI Joint Written Project - May

31 Inbound attack (captured via snort integration) Firewall blocks (summary showing time, destination and source) Software install (Office 2007 by user=jcarlson) STI Joint Written Project - May

32 Abnormal IP address access to network by Prism Microsystems common attacks Prevention of in-bound attacks (both external/internal) Prevention of in-bound attacks (both external/internal) The view below offers a summary of events headed into the JWP private IP space ( /8). Through the use of baselining, the customer can compare current event activity by destination IP to the historical normative behavior. This allows a contextual understanding of how events are changing in time, helping pinpoint anomalous activity into the JWP network. STI Joint Written Project - May