SIEM 2.0: AN IANS INTERACTIVE PHONE CONFERENCE INTEGRATING FIVE KEY REQUIREMENTS MISSING IN 1ST GEN SOLUTIONS SUMMARY OF FINDINGS

Transcription

1 SIEM 2.0: INTEGRATING FIVE KEY REQUIREMENTS MISSING IN 1ST GEN SOLUTIONS AN IANS INTERACTIVE PHONE CONFERENCE SUMMARY OF FINDINGS OCTOBER 2009

2 Chris Peterson, LogRhythm CTO, Founder Chris brings a unique and diverse background in information security, audit, product development, and product management to his role as Chief Technology Officer/Founder of LogRhythm. Chris has spoken at numerous conferences, been frequently quoted in industry publications, and was a faculty member with IANS. Chris has a degree in Accounting/ Information Systems from Colorado State University. Raffael Marty IANS Faculty Raffael is a member of the IANS faculty. He is an expert and author in the field of data visualization and has additional expertise in log management, log analysis, industry standards, and intrusion detection. His previous positions in the log management space include roles at Splunk, ArcSight, and IBM Research. Briefing Summary The first SIEM solutions (SIEM 1.0) had many limitations including a narrow focus on security and data reduction; limited forensics and contextualization; unfulfilled promises on correlation; and high cost. Many organizations with a SIEM are not realizing its full value, as SIEM is often used for compliance but not much else. Collecting app logs is increasingly important but can be difficult. Various approaches to get at these logs can be tried. There is a role for both agentless collection and agents. Overview Because of large volumes of data, SIEMs must have a scalable architecture. LogRhythm scales horizontally which allows growing incrementally and adding capacity over time. SIEMs must support the work of analysts through filtering and visualization tools as well as through automation. SIEM 2.0 will provide better environmental awareness (which includes network, host, and data awareness) and superior forensics capabilities, and will be deployable incrementally. SIEM 2.0 will give users the tools to help them comply. In this Interactive Phone Conference, IANS faculty member Raffael Marty moderated a conversation with LogRhythm CTO/Founder Chris Peterson about the limitations of SIEM 1.0 solutions, the evolution to SIEM 2.0, what SIEM 2.0 will look like, and how LogRhythm fits in SIEM 2.0. SIEM is in the process of evolving from version 1.0 to 2.0. SIEM 1.0 had many limitations and didn t fulfill its promise. The limitations included a narrow focus on security and data reduction (to the exclusion of all else), limited forensics capabilities, over-promising on correlation, and high costs. With forthcoming SIEM 2.0, this is changing. SIEM 2.0 will have a broader focus than merely security and data reduction. It will pull in more data and in doing so will provide better environmental awareness and contextualization. SIEM 2.0 will provide better forensics and will be deployable incrementally. It will be more easily scalable and will have better capabilities to help organizations with compliance. LogRhythm is leading the way in the development of SIEM 2.0. LogRhythm has a scalable architecture, excels at both data collection and analysis (including automated analysis capabilities), and provides significant compliance coverage. To learn more, visit Key Points The first generation of SIEM products (SIEM 1.0) had many limitations. Based on his extensive experience in this market, Chris has identified five limitations of SIEM 1.0. They are: Produced for the by IANS. All Rights Reserved. For additional information contact LogRhythm or IANS. Page 1 of 6

3 The initial SIEM systems didn t bring in data to contextualize; they lacked visibility. Instead of reducing data we actually needed more data for forensics and for contextualization. 1. A security-centric focus. When the initial SIEM solutions came to market, they were solely and narrowly focused on security. This was a shortcoming as these solutions weren t architected for use outside of security. The initial SIEM solutions couldn t be used for operations, compliance, or audit use cases. 2. Too much attention on data reduction. The purpose of the initial SIEM products was to reduce the amount of data and to focus exclusively on high-quality events. But in the process of focusing on data reduction, important data was lost. 3. Limited forensics and contextualization support. Because of the focus on data reduction, forensics data and other contextual data was lost. Moreover, other important contextual data elements were not brought in, such as asset value and network flow data. 4. Over-promising on correlation functionality. The initial SIEMs made bold promises about their ability to correlate. But correlation is hard. It requires known patterns, which may not exist. 5. Overly complex and expensive. SIEM 1.0 solutions were extremely complex and expensive. In many instances, the professional services were as expensive as the software. A solution might cost $500,000 and might require another $500,000 in professional services. Many organizations don t fully utilize their SIEM. Over the past few years, in many instances the primary driver for a SIEM project was compliance. Compliance drove the budget and was the focus of the implementation. In such situations, a SIEM is probably not being used to its full potential. LogRhythm s founders believe that log and event management can provide value beyond just compliance for multiple constituencies: information security, audit and compliance, and operations. But in many organizations, just one of these constituencies is using the SIEM, which limits the value that is derived. How and what SIEMs correlate has evolved. Previously, most SIEMs had a network focus, correlating information such as firewall logs or IDS information. But with the evolution to SIEM 2.0, the scope of SIEMs has broadened. Now, more data is being collected from more places, such as the application layer. This provides more robust and valuable correlation. Application logs will play a more important role, but many applications aren t instrumented for this purpose. More information with deeper substance is needed from the application layer, information which is generally housed at the application. What is Produced for the by IANS. All Rights Reserved. For additional information contact LogRhythm or IANS. Page 2 of 6

4 needed is to bring logs into the SIEM from the application to better understand what is happening at the application layer. However, most applications aren t instrumented to generate the types of logs that are needed. In those instances, there is a need to augment what the app layer provides. Ways to address this include augmenting the audit trail to provide more data around the environment being monitored, and pulling in data that the app may not provide. This may come by monitoring the database to see what data is being moved, and where it is being moved to/from. It would be beneficial if app vendors would log more data, but customers don t buy an app based on its logging capabilities. You need a flexible, hybrid collection infrastructure with multiple interfaces... in an enterprise class solution, you want both of those capabilities [agent and agentless] from one vendor, that are centrally managed and reliable. Because of the huge amounts of data, you ve got to have a scalable architecture.... You also need a scalable archiving infrastructure. In collecting data, there is a role for agentless collection as well as agents. There are challenges associated with collecting logs across the IT stack. Organizations need a flexible IT infrastructure. This entails a combination of robust agentless collection capabilities and agents when appropriate. Many solution providers say that their SIEM is agentless. But in certain cases, you can t get around using an agent, and agents have certain advantages. For example, an agent can be on a high volume web server to pull flat files, collect data in batches, or encrypt data. An enterprise class solution should have both agent and agentless capabilities. Because of the massive amounts of data, SIEMs must have a scalable architecture. Dealing with high amounts of volume and scalability has been a limiting factor for some SIEM 1.0 solutions and is critical in SIEM 2.0. For SIEM 2.0 vendors, a starting point is acknowledging the huge amounts of data. Some approaches for making a solution scalable include: Scalable architecture. Solutions have to be scalable. The approach adopted by LogRhythm allows customers to easily expand their deployment to fit their ever-increasing requirements. Log Deduplication. Apps need intelligence to be able to reduce the data to make storage and analysis more efficient. LogRhythm has a Log Deduplication capability that decreases the cost of storing data and makes analysis and reporting more efficient. Scalable archiving. The infrastructure must support scalable archiving so that archiving is efficient, compressed, and accessible. Enterprise data management. Organizations must make decisions on how to treat their data. This includes determining what must be available in real time and what data can be stored separately for purposes such as forensics. Produced for the by IANS. All Rights Reserved. For additional information contact LogRhythm or IANS. Page 3 of 6

5 There is a lot of opportunity to automate the analysis function so that analysts just deal with the highest risk events. SIEM 2.0 will have environmental awareness where users will have information about their environment at their fingertips. For SIEMs to deliver value they must support the work of analysts. For a SIEM to collect a lot of data isn t good enough; it must be able to present information in a useable form. SIEMs need to perform in terms of both collection and analysis. LogRhythm has created a client interface that enables filtering and drilling down, and provides good visualization tools. A SIEM needs to provide tools that scale, provide quick analysis, and enable seeing all logs in a single pane of glass. In addition, analysts need tools that are automated and that do much of the work for them. This includes pattern-based approaches to correlation, behavioral-based detection, statistical analysis, and prioritization. SIEMs have a long way to go on this front. SIEM 2.0 will have a few critical, defining characteristics. While no solutions yet deliver on the potential for SIEM 2.0, solutions such as LogRhythm are close. Some of the key characteristics of SIEM 2.0 are: Environmental awareness. In SIEM 2.0, solutions will provide more information about the environment. This will include: - Network awareness. This includes information such as the risks, threats and information about the activity occurring on the network. This will bring a great level of awareness about attacks. - Host awareness. This includes information about what the host is; what its business function is; what its vulnerabilities and levels of risk are; what data it is storing; and who is authorized to use it. Automated analysis can yield much of host awareness. - Data awareness. This includes knowing where data is and where it is moving to. Having environmental awareness within a SIEM requires integration with other products, such as products that can bring in flow data and data loss prevention (DLP) information. Immediate forensic search. SIEM 2.0 will have the capability to bring in a lot of data that can be used for forensic purposes. SIEM 2.0 will create a forensic information base. This will include log data, flow data, and much more. SIEM 2.0 will provide an immediate search capability and will give users the ability to draw on the forensic information base when needed. Incremental deployment and scalability model. In SIEM 1.0, SIEMs were deployed enterprise wide and top down. With SIEM 2.0, organizations will be able to deploy a SIEM incrementally. For example, an organization can first address the log management use case and can then expand the use over time to correlation, forensics, and more. Produced for the by IANS. All Rights Reserved. For additional information contact LogRhythm or IANS. Page 4 of 6

6 SIEM 2.0 gives you the tools to help you comply. SIEM 2.0 will help organizations with compliance in more ways than a conventional SIEM. Just having a SIEM doesn t make an organization compliant, but a SIEM can perform certain functions that can help an organization comply. For example, a SIEM can centrally collect logs, which is a requirement for compliance. And a SIEM can perform automated analysis, which is also related to many compliance standards. Additionally, a SIEM can monitor access and use by privileged users. An important attribute of SIEM 2.0 is figuring out who the user is. An advantage of SIEM 2.0 is bringing in the audit trail to see what systems users have logged into. If an attack is from an internal user, the SIEM should be able to infer the user from the other data that is collected. Other Important Points In discussing the above key points about the evolution of SIEM technology, a few related themes were highlighted by the event participants. Normalization. A failing of SIEM 1.0 was having to understand and normalize data. SIEMs should separate the collection and normalization of data; they should be able to collect any data, normalizing the time of occurrence. Then, an additional layer of normalization can take place to classify logs, ID logs by type, and pull out relevant metadata. Visualization. Visualization looks good and can help sell a SIEM. But as a user, the purpose of visualization is to see something that wouldn t otherwise be seen, or see it earlier. You want visualization to tell you something. A value of visualization is to see relationships ( relationship visualization ). This includes mapping and diagramming which hosts have sent and received data, and how much data they have sent/received. Databases. When buying a SIEM, a purchaser shouldn t have to worry about the database. Some SIEM vendors have a proprietary database and others, like LogRhythm, have hybrids. These are commercially available databases with proprietary data structures. When users are setting up a database for a SIEM, it is common to want to design an audit trail. The first step in doing so is to create a date and time stamp, which is best written in UTC. In addition, it is necessary to have an event ID structure and to document this structure. Lastly, for any relevant metadata that is to be captured which should be part of an audit trail, this metadata should be put in the database in a separate column. Produced for the by IANS. All Rights Reserved. For additional information contact LogRhythm or IANS. Page 5 of 6

7 About LogRhythm LogRhythm provides a fully integrated log & event management, file integrity monitoring and endpoint monitoring & control solution that empowers organizations to efficiently comply with regulations, secure their networks, and optimize the availability of their IT infrastructure. By automating the collection, organization, analysis, archival, and recovery of all log data, LogRhythm enables enterprises to easily comply with log data retention regulations while simultaneously gaining valuable, timely, and actionable insights into security, availability, performance, and audit issues within their infrastructure. LogRhythm solutions are noted for their completeness, useful analytics, ease of use, and rapid time to value. LogRhythm customers range from Fortune 500 organizations and government agencies to community banks and credit unions. SC Magazine gave LogRhythm 4.0 a perfect, 5-star rating and their coveted "Best Buy" designation. SC Magazine readers selected LogRhythm as the best SIEM (Security Incident/Event Management) solution. Learn more at About IANS IANS is the premier membership organization for practicing information security professionals. IANS mission is to provide key technical and business insights to help members solve their most pressing professional challenges. IANS achieves this mission through a broad offering of services provided to its members insightful events, thought-provoking publications, best-practice research, and unique networking opportunities. Learn more at Produced for the by IANS. All Rights Reserved. For additional information contact LogRhythm or IANS. Page 6 of 6