MASSIF: A Highly Scalable SIEM Ricardo Jimenez-Peris Univ. Politecnica de Madrid (UPM) rjimenez@fi.upm.es DEMONS Workshop Berlin, April 25 th 2012
MASSIF in a Nutshell MASSIF aims at developing the next generation of SIEMs (Security Information and Event Management Systems). MASSIF event correlation is based on complex event processing technology. One of the main innovations in the project lies in that the event correlation engine is highly scalable (able to process millions of events per second) and elastic. A main application is security for cloud applications that are scalable and elastic and require scalable and elastic security technology.
Complex Event Processing: A Primer Complex event processing provides an in-memory version of database queries over streaming events. Each event stream can be considered as an infinite table. Queries over event streams are similar to database queries but adapted to the continuous nature of event streams. Event queries are based on the window sliding model in which the query process the events received in a temporal window, e.g., number of IP packets per IP destination address in the last hour.
Complex Event Processing: A Primer Sample security event schema: DevID: ID of the device generating the event. EvID: ID of the kind of event. SrcIP: Source IP address. DstIP: Destination IP address. Ts: Timestamp. Queries consist of interconnected operators that can be stateless or stateful.
Complex Event Processing: A Primer Event Transformation Example: Extracting attributes of interest from incoming events
Complex Event Processing: A Primer Event Filtering and Routing Example: Route Snort and CISCO Pix events, discard others
Complex Event Processing: A Primer Event Merging Example: Merge Snort events from two different instances
Complex Event Processing: A Primer Event Aggregation Example: Count open connections per minute for each server
Complex Event Processing: A Primer Event Correlation Example: Correlate Snort and Cisco Pix events with the same SrcIP
MASSIF Correlation Engine: Innovations Based on complex event processing (CEP). Main innovations: Highly scalable thanks to parallel-distributed event processing. Elastic thanks to an elastic provisioner and online reconfiguration protocols. Scalability enables to aggregate the resources of many nodes (100s) to enable the processing of millions of events per second. Elasticity enables to use solely the required resources to process the incoming load.
MASSIF Correlation Engine: Ease of Use Transparent parallelization of CEP queries. Syntactic and semantic transparency: Applications remain unchanged. The behavior remains unchanged. Automatic translation of security directives into CEP queries: OSSIM SIEM directives are translated directly into CEP queries. SIEM security experts do not need to learn CEP, they can still continue using the current SIEM directives.
Parallelization of CEP Queries
Elasticity and Load Balancing Elasticity: the system scale-up and down the number of nodes to the current workload avoiding over-provisioning and preventing under-provisioning. Dynamic load balancing: the system continuously balances the load among nodes. Non-intrusive online reconfiguration: the system reconfigures itself without disrupting ongoing processing and keeping the maximum throughput of the system.
Elastic Management
Other Security Application of the Scalable and Elastic CEP Technology Application to detection and mitigation of distributed denial of service attacks (DDoS). The scalability CEP parallel technology enables to characterize in an online manner the traffic from legitimate clients. During a DDoS attack the characterization of the traffic changes significantly. Thanks to the elasticity of the CEP technology, informed filtering of the incoming traffic can be performed based on the characterization of legitimate traffic.
Conclusions MASSIF is developing the next wave of SIEM technology. One of the main innovations lie in scalable and elastic CEP technology used for the event correlation engine. This security support is especially interesting in cloud platforms that are scalable and elastic and require scalable and elastic security as well. This CEP technology is also being applied to other security applications such as mitigation of DDoS attacks.