National Security and Cyber Defense with Big Data

Transcription

1 National Security and Cyber Defense with Big Data Tomasz Przybyszewski Big Data Solutions Lead ECE Region Sept 2015 Tomasz Przybyszewski Copyright 2014 Oracle and/or its affiliates. All rights reserved.

2 What is Big Data and why to use it Gart er s 3 V s of big data High Volume with an estimated 2.5 quintillion bytes of data created every day. Comes in a Variety of for ats te t stri gs, i ages, e logs, do u e ts, u eri data et. a ross a diversity of formats and sources. This is data in motion, constantly changing high Velocity the relevance of which can decay rapidly Digital data is diverse which makes it difficult for most traditional technologies to enable capture, storage & analysis 2,500 exabytes of new information in 2012 with digital content as the primary driver Digital universe grew by 62% last year to 800K petabytes and ill gro to. zetta tes this year New world requires a different approach: Un-aggregated, lowest level data Ad a ed a al ti s to e a le dis o er & enable complex queries Fast, real time processing capability Copyright 2014 Oracle and/or its affiliates. All rights reserved.

3 Big Data Analytics Moving to the Next Generation of Analytics to Predictive Analytics Current Intelligence Generation Tomasz Przybyszewski Next Generation Big Data Copyright 2014 Oracle and/or its affiliates. All rights reserved.

4 Big Data for National Security Cyber Intelligence Social Media Intelligence Miltary Defence Video Analytics Better Low Level Data Analytics = Deeper, More Actionable Insights = More Informed Decisions Data fusion eg: External data with own data and stakeholder data Appl i g a risk ased i tellige e apa ilities to cope with the massive Volume, Velocity and Variety of data Copyright 2014 Oracle and/or its affiliates. All rights reserved.

5 4th Generation Oracle Data Architecture for Big Data Data Services APIs Business Data Data Streaming Data Streams Social/Log Data Enterprise Data Execution Innovation Other Data Sources Data Platform Reservoir Data Factory Analytics Warehouse Telematics Industry Services Internet of Things Sentiment Reports Model First& Analytics Dashboards Reporting-oriented Often enterprise wide in scope, cross LoB you know the questions to ask Data First Discovery Analytics Discovery Lab Data Exploration Highly visual and/or interactive you don t know the questions to ask Copyright 2014 Oracle and/or its affiliates. All rights reserved. #StrataHadoop - Oracle Big Data Architecture

6 Integrated Oracle Systems for Big Data Data Services APIs Business Data Data Streams Social/Log Data Data Streaming Data Platform Analytics Execution Innovation Other Data Sources Reports Model First& Analytics Dashboards Reporting-oriented Often enterprise wide in scope, cross LoB you know the questions to ask Search Enterprise Data Telematics Industry Services Internet of Things Sentiment Transformation Script Data First Discovery Analytics Discovery Lab Data Exploration Highly visual and/or interactive you don t know the questions to ask Copyright 2014 Oracle and/or its affiliates. All rights reserved. #StrataHadoop - Oracle Big Data Architecture

8 Big Data Cyber Intelligence Israel Government The Israel Government with Ministry of Defence and National Cyber Bureau have national responsibility for all aspects of cyber security for meeting current cyber threats and to develop capabilities to combat the next and future generation of cyber threats Business Drivers Cyber platform and tools for monitoring, organising and investigating current and next generation cyber threats Open platform for monitoring and forensics Real time decision making and situational awareness for security and operational professionals Investigate network behaviour and automate detection of threats using sophisticated algorithms and predictive engines

9 Cyber Solution Overview Internal Network Collection Discovery Port Mirror Security Officer All incoming and outgoing data packets Aggregation & Analysis BigSearch Data Discovery Configurations, logs & ex. resources Oracle Big Data Platform 9 Transformation Script

10 2nd area of Big Data References * National Security incl: - Social Media Intelligence - Miltary Defence - Video Analytics * Currently Implementing Copyright 2014 Oracle and/or its affiliates. All rights reserved.

11 US Customs & Border Protection Automated Targeting System (ATS) Rules based decision support system National Targeting Center Data sources: government and public Historical data and trends analysis Deployed for air, land and sea travel Massive volume of data Identify high risk targets Faster clearance for low risk traveler/cargo 11 1

12 Allied Nation Intelligence Service Oracle Spatial and Graph: Social Analysis Objectives Benefits Profile suspects through telephone, Standards-based tools: W3C RDF & SPARQL and social network communications Produce data products for analysts Semantic tagging for 600 TB / 10b triples graph Solution New discovery on ~100 million triples / month RDF Graph modeling of the social network: people, groups and places of interest Inferencing & graph analytics discover relationships among individuals & meaning of pseudonyms, aliases, codes, terminology 12 Top-secret, compartmented security for data Find & label same-as relationships

13 Italian Ministry of Interior Predictive Analytics for Police Department of Public Security Highlight those areas with the highest crime risk. Decision taking support. Predictive Analysis solution for location based Crime Forecasting 13

14 Location Intelligence South Yorkshire Police, England Deployment vs Crime Analysis Visibility Deployment Planning Crime Search POI Deployment Analysis Dangerous area prediction 14 Database Data Mining Spatial MapViewer OBIEE

15 Guardia Civil Spain Customer Background Ministry of Interior and Defence Agency National Security management, coordination and supervision for the different security bodies Challenges/Opportunities National security threats are one of the biggest concerns of the ministry. Look for behavior patterns to anticipate and detect potential threats Matching structured and non-structured information Flight details, passenger data, car plates, Internal notes & Social network information 15 1

16 Intelligent Real Time Systems Protecting Citizens Discovering Protest Activities Background NATO Summit in Chicago, May ,000 Visiting Dignitaries 2,200 Journalists 10 s of thousands of protestor Thousands of Officers Solution Monitor Social Media to identify hot-spots Correlate with police deployment Initiate operational orders Retrospectively identify potential witnesses 16

17 New York State Police New York State Police (NYSP) is the primary law enforcement agency in New York State. The force is comprise of 4,600 offices patrolling 54,500 square miles After Before As part of the New York State Police Network system, NYSP has many intelligence repositories including missing persons, stolen vehicles, stolen plates, wanted persons, lost and stolen property For compliance reason audit records are created each time the system is queried. Beyond ensuring compliance the audit repository contains a wealth of information for investigators. Due to the volume of data and complexity of requests the normal turn around time for a request was hours and manually intensive. 17 Using Big Data the NYSP built a search application which provides access to more then 8 years of audit history 1.5B records. Analysts can now fulfill requests using a familiar search and navigation paradigm that is as easy to use as their favorite website. Request turnaround in minutes not hours. Manual effort reduced from multiple complex steps to a single simple process Estimated annual cost reduction of several hundred thousand dollars in support

18 Big Data Case Study Abu Dhabi Police The General Directorate of Abu Dhabi Police operates with other agencies to achieve a safer society. It seeks to preserve the stability, the reduction of crime and the removal of a sense of fear, as well as contributing to the achievement of justice among the general public. Business Drivers Highest quality, prompt and accurately responsive services to citizens; Technology response was to leverage Social Media sources for data to create high value analytics Improve UAE-wide security Zero deaths on roads, Zero crime in streets ; Technology response was to utilize camera feeds and other such sources and process them using high performance, innovative Data Center environment Improve operational efficiency and control costs; Technology response was to Transform Data Center and optimize performance 18

19 Other Police cases Turkey Police: Dubai Police Improve performance and variety of their current security analysis Enhance performence and flexibility of Hadoop systems to analyze all data from varied social media sources Create and drive timely intelligent actions... Correlate data from a DWH and silos in multiple systems Monitor social media channels, crawl online sources including dark web Create police data dictionary to store, track and search indications of crime and other threats in multiple languages Provide a unified discovery interface suitable for police analysts A foundation platform to store unstructured data that is scalable and easily connect to new data sources. 19

21 Oracle Big Data National Security and Cyber Defence Key Features Based on Big Data Open architecture and scalable platform Low cost enable flexible deployment and expansion Central distributed data management system Innovative analytical tools Open API for external systems and 3rd algorithms 21

22 We Look Forward To Working With You NEXT STEPS Live Demo Meet with key stakeholders to present the Live Demo 22 RECOMMENDATION S / ROADMAP Prepare defined set of Oracle Big Data solution recommendations to address priorities EXECUTIVE PROOF OF CONCEPT Select 1,2 use cases and perform the tests EXECUTIVE READOUT Present findings and recommendations to key stakeholders Oracle Confidential Internal 2