Security Data Analytics Platform Figure 1 - Global Search Dashboard "The Data Analytics Platform has revolutionized the way we handle data from our Security monitoring infrastructure to our developers and system administrators tuning performance and tracking resource consumption. By combining best of breed open source products into an analytics ecosystem we reap the benefits of lowered cost and increased flexibility." Introduction - - Bob Grant Chief Technology Officer, UC Riverside IT security challenges facing higher education institutions are becoming increasingly complex. Major security breaches in 2014 provided examples of disturbing attack trends involving malicious actors breaching systems and exploiting users. In response, UCR developed innovative methods for monitoring and protection of a growing number of IT resources and a large population of dynamic user accounts. With hundreds of servers, workstations, embedded systems and in- house applications, it is important to have a flexible and scalable solution capable of providing real- time analysis of massive amounts of data. UCR built a security data analytics platform to combine the event data of many disparate systems into a comprehensive, unified enterprise solution that greatly enhances the response to security threats by providing real- time discovery and analysis of network, system and user account activity. Business Need Campus IT services are producing terabytes of data on a daily basis making it incredibly difficult for security teams to discover and respond to relevant security threats. Additionally, user accounts may be 1
compromised through phishing or other by means, making these incidents difficult to detect. Disparate systems and applications with dissimilar logging and auditing formats add additional complexity to understanding enterprise activity and making sense of enormous amounts of data. Resource constrained security teams spent too much time sifting through irrelevant noise and not enough time focusing on meaningful security events and behavior requiring immediate attention. A strategic initiative was launched in 2014 to change how central computing teams were conducting security data analytics across a multitude of campus systems, services, and applications. A new solution was designed to meet the following objectives: Utilize free or low- cost software to avoid vendor lock in Utilize low- cost commodity hardware Integrate with existing campus security systems (e.g. SecTools) and provide web services for exchanging data Reliable and easily scalable to meet increasing demands Implementable by other departments or institutions using common architectural patterns Provide staff with real- time correlation and analysis of events Capable of processing, indexing, and storing terabytes of event data from hundreds of sources Provide flexibility in handling frequent environment changes and evolution of new sources of security data Dashboards, data sharing and user collaboration Features and Highlights In an effort to address the security needs expressed above, UCR designed and built a brand new data analytics platform. The platform is a collection of technologies, which contains the following features: Built entirely with free and open source technologies Virtually the entire technology stack is sharable with others Provides a unified application portal with many dashboards for monitoring and responding to events across a multitude of systems, services and applications Eliminates the development of dashboard user interfaces and visualizations of data models (such pie charts, histograms, table pagination). Developers can focus on the collection and modeling of data and not the complex UI interactions. Dramatically reduces time in analyzing large quantities of security event data through powerful clustered indexing systems allowing sophisticated data mining Web services architecture (RESTful) makes it easy for storing, distributing and analyzing event data. Readily integrates data with other systems. Customizable dashboards provide real- time analysis. Dashboards are easily shared with other staff via unique URLs and can be created ad hoc. Centralizes log collection and indexing across many campus servers, as well as critical services such as CAS, DNS, Wireless, RADIUS, E- mail, Firewalls, campus VPN, etc. Enhances capability for tracking security incidents such as DMCA violations by providing dashboards that display information collected from internal ticketing systems 2
Integration with campus security systems including host/network intrusion detection systems and vulnerability scanners. Host vulnerability information is immediately available in the system. Log analysis provides customizable rules and decoders allowing virtually any system or application that produces log files to be monitored Provides security controls and separation of duties so users are only able to access dashboards, tools and event data for which they re authorized Meets security compliance objectives of data security standards (e.g. PCI DSS) by providing real- time monitoring, alerting, incident response, centralization of logs and authentication/authorization controls Figure 2 shows an example of an actual dashboard used by central computing for monitoring campus network traffic and intrusion detection systems. Figure 2 - Network Intrusion Detection Monitoring The new platform provides an innovative, low cost approach for data collection and analytics. It was intended that this platform have wide applicability, and as the system evolved, other business units outside of security have expressed interest. In April 2015, security teams worked with enterprise application developers to centralize application server logs to provide data analytics capability for developers. The system is now providing monitoring of application events via the exact same architecture used by the security team. Newly provisioned systems are automatically monitored and events collected without any user intervention. Beginning in summer of 2015, the analytics platform will also provide statistical analysis and data mining capability for UCR campus web portals used by students, faculty and staff. Figure 3 shows an example of portal analytics showing user clicks categorized by graduate level and class, all collected by the analytics platform. 3
Figure 3 - Web Portal Analytics Proof of Concept The Process: Technology and Implementation While built on commonly available components, this combination of tools makes for a powerful platform that easily serves the analytics needs of multiple business functions. At a high level, all event data including local logs for systems, services and applications are collected by host and network intrusion detection systems (OSSEC and Bro- IDS). This data is then sent to a central collection system (Redis and Logstash) where event data is normalized before being shipped to the Elasticsearch cluster. The SecTools and Kibana dashboards display the data to users. The entire process of log collection, analysis, correlation, indexing and availability for user dashboards is near real- time, making all information available within seconds. Figure 4 provides a high- level workflow overview of the platform. Figure 4 - High Level Data Flow 4
Testimonials Student Affairs Technology Services is responsible for protecting data integrity that is shared among more than 300 systems. What makes this responsibility even more critical is that these systems can be restricted or non- restricted in nature. Our network suffers literally hundreds of attacks each minute, attempting to gain access to secure data. The systems governed by Student Affairs are actively monitored and protected from these attempts. The implementation of the Security Data Analytics Platform tools by UCR C&C has broadened the scope of our proactive security response to the UCR campus footprint. This is a vital component in our efforts to protect our students, faculty, and staff." - - Deborah Enright, Senior Director (interim) - Student Affairs Technology Services, UC Riverside Timeline August 2014 October 2014 November 2014 December 2015 April 2015 July 2015 September 2015 Project initiation and revamp of original SecTools system New platform designed, built and delivered to production Delivery of new dashboards and data models Integration with campus network security scanners, host and network intrusion detection systems Provision of logging and data analytics to C&C s enterprise developers (Planned) System to provide UCR web portal analytics (Planned) Every critical campus service and system monitored and available for security analytics Team Members Nicholas Turley Jonathan Ocab Vasken Houdoverdov Dept., Org., Partners, etc. Submitted By Nicholas Turley Manager of Security nick.turley@ucr.edu (951) 827-3070 5