BIG DATA ENABLES ACTIVITY & LOCATION BASED PREDICTIVE ANALYTICS APPLICATIONS

Transcription

1 BIG DATA ENABLES ACTIVITY & LOCATION BASED PREDICTIVE ANALYTICS APPLICATIONS INTELLIGENT, INFORMED DECISION MAKING AND INCREASED EFFICIENCY GOVERNMENT BIG DATA SYMPOSIUM, NOVEMBER 20, 2013 ARLINGTON, VA JOSEPH D. FARGNOLI RITRE CORPORATION

2 It is our objective to show that Big Data enables Predictive Analytics and Activity Based Intelligence (ABI) Big Data Allows Large Data Source Integration (Hyperspectral Imagery) Big Data Makes it to Main Street USA with Integration between GIS and ERP Systems We offer Commercial Examples of the use of BIG DATA to drive better decisions as a GAME CHANGER

3 Responses Challenges Periods of Fundamental Change and BIG DATA opportunity s 1950s 1980s How to prepare for war with Japan? How to prepare for a conflict in Europe? How to develop new military technologies during the Depression /defense budget constraints? How to fight in the time, place, and manner of our choosing in the atomic age? How to leverage new nuclear genie? How to employ trip wires? How to reposture globally for the Cold War? How to roll-back Communist expansion in the 3 rd world? How to impose costs on USSR in terms of technology? How to restore the military balance on the Central European Front? How do we build partnerships to defeat terrorist extremism? How do we defend the homeland in-depth? How do we shape the choices of countries arriving at strategic crossroads? How do we prevent the acquisition or use of WMD by hostile state or non-state actors? New Concepts of Warfare Amphibious warfare Carrier aviation Combined arms / mechanized warfare Strategic bombing New Look Strategy Doctrine of massive retaliation Nuclear Triad (USAF, USN) Pentomic Army Nuclear-capable tactical aircraft (USAF, USN) NRO Reagan Doctrine Competitive strategies SDI Maritime Strategy, horizontal escalation Stealth / precision / technology AirLand Battle SOF capabilities Transformation Capabilities Mix Joint Capability Enablers Roles, Missions & Organizations Business Practices Human Capital Strategy Authorities Employ Predictive Societal Models 3

4 Constrained Budgets Escalating Challenges Non-State Actors Transnational Actors Failed States In the past an analyst worked to solve a puzzle. Today analysts work on mysteries and don t even know if there is a puzzle to be solved Peacekeeping Information Security 4

5 Current Strategic Challenges Traditional challenges - force on force conventional war (PRC, DPRK) Must be prepared, but will likely be the exception Irregular challenges - unconventional methods to counter traditional advantages of stronger opponents Will likely be preferred method to target interests Catastrophic challenges WMD or WMD-like effects Disruptive challenges breakthrough technologies to negate current advantages in key operational domains Predictive Analytics and Big Data allow us to confront challenges earlier and more comprehensively, before they are allowed to mature

6 Current Strategic Challenges Predictive Analytics Application: Cyber What s Predicted: Risk to US Critical Infrastructure What s Done About It: Initial attacks are identified and characterized to determine subsequent stages of attacks and defensive posture is adjusted to optimize defenses. Predictive Analytics Application: Terrorism and Transnational Organized Crime What s Predicted: The Global Jihadist Threat Overseas What s Done About It: Global transactions financial and otherwise are monitored to identify patterns. Predictive Analytics Application: WMD Proliferation What s Predicted: WMD Security in Syria What s Done About It: Relevant materials needed at various stages of deployment yield insights into stages of risk. Predictive Analytics Application: Counterintelligence What s Predicted: Threats to Supply Chains What s Done About It: Business posture of suppliers are characterized for legitimacy practices. 6

7 Current Strategic Challenges Predictive Analytics Application: Counterspace What s Predicted: Threats to commercial and government space based assets What s Done About It: Given limited/no defensive mechanism, awareness allows for diplomatic or cooperative means to bring early awareness of developing threat and repercussions. Predictive Analytics Application: Natural Resources: Insecurity and Competition What s Predicted: Global supply/demand for food, water, energy What s Done About It: Monitor shifts in human geography, climate, disease, and competition for natural resources because they fuel tensions and conflicts - allows for optimal diplomatic posturing. Predictive Analytics Application: Health and Pandemic Threats What s Predicted: Wide scale illness outbreak What s Done About It: Plan response based on seasonal and world patterns of infectious diffusion Predictive Analytics Application: Mass Atrocities What s Predicted: Likelihood of Rwandan type events on any scale What s Done About It: Data correlation from various sources to provide awareness. 7

8 ABI Taxonomy: What is ABI? Not a new phenomenology, rather a paradigm/methodology for analyzing how people, events, actions/activities interact, not occurrence of a single event Transactions = act/change conducted by an entity indicating exchange of information or material Activity = recognized movement/change by an entity indicating specific meaning when viewed with context Inherently Multi-INT, requires Multi-Sourced data processing, timely indications and warning, situation awareness, inference and decision analysis Association and Aggregation of multi-source observations to derive heuristics and intent Traditional intelligence modality (SIGINT, IMINT, OSINT, GEOINT.. ) Non-traditional indicators (social networks, financial transactions, communications. ) Space/air/ground/undersea, cyber, human dimension, open source, kinetic and non-kinetic Geo-spatial and temporal: signatures, events, patterns of life, change detection Actions, Intent and Consequences. It s more than finding a needle in a hay stack, it is about where is the hay stack, which haystack, what kind of needle and which needle. Rationalizing why the needle is there and how is the needle going to be used! 8

9 The ABI Challenges: ABI Tradecraft In environments where there is no visual difference between friend and enemy, it is by their Actions that enemies are visible MOTION is the first indication of Activity; TEMPORAL and VISUAL patterns of CHANGE provides the context for intent Who is the enemy and who is the friend? Determines the level of interest What are they [the Who] doing? Motion and imagery are critical components When and Where are they [the Who] doing [What]? Informs about patterns of nefarious activities Why are they [the Who] doing [What]? Must understand Behavior [Motion] and Intent How are they [the Who] doing the What? - linked to Intent, Behavior and Means The ABI Tradecraft From Heuristics to Truth Are ordinary-looking events being performed for nefarious purposes? What new methods and processes can provide context to ordinary-looking activities to ascertain intent by actors (people, sensors, communications) to predict consequences from observations?

10 ABI Tradecraft Georeference to Discover Enables discovery and integration of activities and transactions across a variety of data types and allows analysts to focus their analysis Data integration and association before data exploitation Analyst uses multiint data at its most discoverable level to begin analysis; the integration and fusion occurs long before a finished product is formed Data Neutrality All data and data sources are potentially equally viable for ABI analytics Sequence Neutrality The importance of data on hand (or being collected) is often not understood until the analyst gets more information - data often holds the answer to a question that has not been asked before - let the data surprise you 10

11 5 Elements of ABI 1. Collect, characterize and locate activities and transactions 2. Identify and locate actors and entities conducting the activities and transactions 3. Identify and locate networks of actors 4. Understand the relationships between networks 5. Develop Patterns of Life Activities and transactions are not necessarily just tied to geo-spatial actions, but also apply across the cyber, social, financial and commercial domains 11

12 Feature Based GEOINT vs. Activity Based GEOINT Feature based GEOINT focuses on: Structural components Specific locations Activity based GEOINT focuses on: Capturing activities as they occur Developing and unraveling networks Persistent Surveillance moves us from being feature focused to being activity focused The focus of collection is activity not the static image Definition of ABI: A discipline of intelligence where the analysis and subsequent collection is focused on the activity and transactions associated within an area of interest 12

13 ABI Goal Moving from Pixel Space to Activity Space 13 Big Data Processing Techniques Enables a new discipline of intelligence focused on the activity and transactions associated within an area of interest

14 Current ABI State of Art Big Data Challenge/Opportunity Significant research advances in Imagery (Stills and Video) and Motion Intelligence (MTI) - Emergence of LIDAR SIGINT, IMINT, OSINT, MASINT, GEOINT, Cyber operates within their own stove pipes Some attempt to integrate within stove pipes No integration cross enterprise to achieve a persistent, synergistic and cohesive whole Cross tip and cue across multiple intelligence modalities is at its infancy No near real time tip/cue and upstream processing at the sensors (forensic and a priori) Enterprise optimization and management of intelligence collection resources across stove pipes does not existent Swimming in sensors, drowning in data Data representation, data duplication, data organization, data security unsolved Activities Based Intelligence integration occurs as a forensic analysis on static information Probabilistic, predictive complex stream analysis Closed-loop dynamic Tasking, Collection, Processing, Exploitation and Dissemination (TCPED) 14

15 Big Data: Four Categories of the Human Domain Biographical Information: Who You Are Activities: What You Do This data category associates specific actions to an individual as well as interactions with other individuals. Relational: Who You Know This is the individual s web of relationships: family, friends, and associates. Social Contextual: Human Geography Information about the context or the environment in which the entity is found. ABI and Human Domain Analytics are not the answer to every intelligence problem; they are merely new arrows in a quiver of capabilities which will form the intellectual underpinning for how we conduct intelligence in the future. 15

16 Activity Based Intelligence and Human Domain Analytics Inbound, on a helicopter over Angola: A combined team of Special Operations, Civil Affairs, and Army Corps of Engineers personnel prepares to enter a village to work with the villagers and to build a well. Based on an assessment of the patterns of life in the village, they understand how the village shifts and moves. They understand the identity of its leaders and the activities and transactions in which they engage and they understand the local culture, beliefs, and history which provide context to the village s activities. Big Data / ABI / Predictive Analytical Techniques have trained them on likely responses based on real time socio-political inputs. Activity Based Intelligence and Human Domain Analytics have combined to give the team the information they need to make the mission a success. 16

17 Objectives and Critical Technologies for ABI Capture, characterize, and geolocate activities or transactions Identify and geolocate actors or entities conducting activities or transactions Identify and geolocate networks between actors and/or entities Understand broader interactions between networks Develop pattern of life Data Discovery Identify, track and automatically tag activity Metadata and standards to enable rapid search and retrieval Data Integration Common analytic environment that supports integration of multiple sources of intelligence Throughput Better/smarter data compression/reduction methods or technologies Improved storage capacities 17

18 Social Network Analysis in ABI Visualizing data so as to uncover patterns in relationships or interactions - follow the paths that information (or basically anything) follows in social networks. Uncover conflicts of interest in hidden connections between government bodies, lobbies and businesses. Studying society from a network perspective is to study individuals as embedded in a network of relations and seek explanations for social behavior in the structure of these networks rather than in the individuals alone SNA has a long history in social science, although much of the work in advancing its methods has also come from mathematicians, physicists, biologists and computer scientists (because they too study networks of different types) The idea that networks of relations are important in social science is not new, but widespread availability of data and advances in computing and methodology have made it much easier now to apply SNA to a range of problems Networks Tie Strength Key Players Cohesion How to represent various social networks How to identify strong/weak ties in the network How to identify key/central nodes in network Measures of overall network structure SNA is also useful to identify the causes for dysfunctional communities or networks, and to promote social cohesion and growth 18

19 BIG DATA Enables a New Paradigm Enacting Evolution PAST: Tasking Collection Processing Exploitation Dissemination (TCPED) FUTURE: Query-Mine-Assemble-Disseminate-Integrate (QMADI) By adapting the way that we Design the Mission.. PAST: Only National Imagery Interpretability Rating Scale (NIIRS) FUTURE: Bring in Big Data/Enabled Predictive Analytical Analysis Rating Scale (PAARS) BIG DATA/ABI Data applied to Predictive Analytics is a New Feature in the System Design Trade Space Driven by a New Metric (PAARS) 19

20 BIG DATA Techniques enable Hyperspectral Spectral Intelligence for Managing and Safeguarding the World s Resources Spectral Intelligence provides for the identification of natural and man made materials on the surface of the earth through their reflective/emissive characteristics; fingerprints. Like fingerprints spectral responses are unique to individual substances. The ability to remotely determine the composition of a target object or area has valuable application across a number of markets (e.g. mining, sustainable agriculture, environmental protection). Traditional images recognize shape and color of an object Spectral Intelligence tells you what something is composed of Proven technology in use today but limited in its applications for two reasons which HySpecIQ remedies: Primary platform is aerial which has limited access and coverage satellite based system rectifies this limitation Presently transforming spectral data into useful information requires highly trained scientists information products (versus current raw data) brings capability to average users 20

21 What is HyperSpectral Information (HSI)? Something New Under the Sun Spectral signature examples Calcite Clay Vegetation Hyperspectral has been around for decades and Big Data techniques will unlock its value bringing this powerful technique into practical use Enabled by Big Trends in Affordable Access to Space, Advances in Space Processors, Accelerating Mobile Processor Technology Developments

22 Spatial Dimension Big Data Why HSI? Bigger Deeper More Information Rich Data Spectra for a Single Pixel Each wavelength band is captured in a separate layer within the same pixel - the finer the range of wavelengths captured within a band, the more layers of information per pixel and the higher the spectral resolution. The HSI Data Cube Big & Deep Image at a Single Wavelength Big Data Techniques are Allowing the Emergence of the Latent Value of HSI

23 Moore s Law In Space: From Big, Deep Data to Immediate Information to the Masses and Specialized Users On board processing (OBP) leverages meteoric acceleration in space based embedded technology driven by the mobile computing engine OBP allows us to take the Big data and reduce it to information products Eliminates the traditional limitation of down linking data and having multiple ground stations Leverage the power trends in embedded processor / GPU technology OBP enables ~ X more Information Product to be distributed and sold than any other designs OBP Converts Big, Deep Data and to Mobile Consumable Information Products

24 Hyperspectral Applications: Improved Geological Maps Oil & Gas Exploration: Mapping Geology and Lithology - mapping of geologic formations and detailed minerals Existing geologic maps lack detail necessary for understanding in this geologic area Hyperspectral Image Original Geological Map Result geologic map shows greater lithologic variation than in the legacy maps for the area Interpretation of the Hyperspectral Image Updated Geological Map HySpecIQ will Deliver a High Volume of Key High Value Fingerprint Information Products

25 Hyperspectral Applications: Precise Vegetation Monitoring Color coded representation defines exclusion zones if there were protected plants or wetlands. Environmental baseline documenting the state of a surface ecosystem before exploration and development operations begin. Red colors indicate stressed vegetation along canyons and waterways impacted by run-off from legacy mining operations. Stressed vegetation also occurs naturally occurring hydrocarbon micro seepage along localized fault systems and fracture networks. Different Spectral Signatures from Several Plant Types Vegetation Intelligence: Using Spectral Libraries Containing the Signatures of Nearly 1,000 Minerals, Floral Species, and Hydrocarbons

26 Raw Data to Actionable Intelligence Space Collection Airborne Collection Client Applications (FMIP) Data Correction Processing FMIP Algorithm Library Hyperspectral Data Repository Signatures Library ( Fingerprints ) Other Imaging Data (i.e., Radar) Target FMIP Algorithm Data Fusion Engine High Validity Inference Key HySpecIQ Intellectual Property Enables Delivery of Customer Value

27 WSI Example of Commercial Clients and Big Data We aim to provide business users answers with high confidence to make more informed decisions that will improve the profitability of their companies and assist governments make better stewards of their natural resources. We will do for Space Hyperspectral Information products what WSI has done for Space Weather Information products... focus on information users simplify ease of distribution WSI WindCast RT WSI WindCast IQ WSI WindCast IQ Big, Deep Data processed and fused into tailored information products. WSI Fusion WSI Pilotbrief Optima Delivering Simple, Specific Information Products

28 Big Data Analytics: Combining the Power of Geographical Information System (GIS) & Business Intelligence (BI) Our Location analytics solution seamlessly integrates Maps into the Target BI application Location Analytics augments mission-critical, enterprise business systems with content, mapping, and geographic capabilities. In today s era of Fiscal Austerity our Cost Effective Location Analytics Solutions offer not only Centralized view of the data / situation but Real- Time Transparent Intelligence for Swifter Decision Making Improved Governance Optimum Utilization of Available Resources through Deliberate Planning Safety of Personnel, Assets & Mission Critical Infrastructure Augmented Logistics capabilities through Predictive Analysis (What If scenarios) Agility in supporting Multiple Mission Areas Balancing Investments, Resources, Risks, and Priorities Leverage Big Data Where it Resides in Civil and Commercial Applications

29 Platform Excellence Model BI & ESRI Product expertise Well Defined Processes BI Platform expertise Global Professional Services BOBJ IBM Cognos Global Services Delivery Resources MicroStrategy SharePoint Service Execution ESRI Product Expertise Esri Product Support Esri Product Consulting Esri Maps for BOBJ Esri Maps for IBM Cognos Implementation Customization Esri Maps for MicroStrategy Low Turnaround Time Optimum Resource Utilization Excellent Quality standards Esri Maps for SAS Enhanced Training Culture Resources trained in multi BI Platforms Skills Incubation Advanced Technology Centre Methods & Procedures Our extensive platform expertise in BI and GIS Products, along with expertise in GIS process standards allows the implementation of Location Analytics products & solutions across the globe

30 An Emergency Management Configuration Logistics Mobile - Damage - Operations Operations Command Tablet/Pad Public Information Emergency Support Function (ESF) Intelligent Maps ICS Alignment Portal for ArcGIS HTML Gallery ArcGIS Online ArcGIS Online Premium ArcGIS Server Desktop & Models Planning & Analysis Risk Assessment Existing Systems Enterprise Database

31 Local Police Department Architectural Overview City Services Police Data AVL Dispatch Gang Data Parolees & Probationers Jurisdictions Maps Imagery Pictometry 311 Calls Parcel data Cloud Services Street View Natural hazards Demographics Social networking feeds ArcGIS ArcGIS for SharePoint Fusion Core SharePoint Imagery Data GeoProcessing Plume modeling Geocoding Visualization Data integration Workflow Intake forms Tracking Data integration Security Dashboards Office integration

32 Implementation Examples Relating Information by Geography Aiding Computer Aided Dispatch School reports missing child Parents notified/ located Identify best quick suspects Identify high priority calls Identify responder to call Provide information on target destination Dispatch officers/case workers to locate & contact nearby/known child offenders Gather name, contact info, address list Make the responding officer safer & more informed

33 Big Data Location Analytics for State & Local Governments Our team provides an extremely advanced, visually intuitive, web-based Geographic Information System (GIS), designed to help organizations drive mission-critical local governance objectives. Offering Quick, Efficient & Cost Effective data collection & management through Automated workflows & Real Time Access to Data of all Corporate Departments within organization. Our product suite offers a hi-tech information infrastructure to facilitate optimum civic planning and development and a unified view into City Services : Maintenance of daily city services such as Public Transport management, Water Management, Solid Waste Management etc. Planning & Development such as Town Planning HealthTracking: Healthcare & wellness planning, Epidemic Control, Situational Awareness etc Property Tax Management: Property mapping, Revenue Analysis, Property Assessment Public Safety: Fire Hazard & Mitigation, Flood management, Disaster Management, Crime Control management. Citizen Portal: Municipal Problem Reporting, Citizen Service Management, Online Services

34 Simplifying Big Data: Enterprise Application - Retail Product Distribution Our team developed 3 custom applications (web based and for the ipad) to help a client evaluate how ESRI s ArcGIS and Business Analyst Server can add value to their critical business operations. The Depletions Mapping Dashboard Application demonstrated the business intelligence capabilities of ArcGIS server by providing a map based sales dashboard for senior executives. The Distributor Partner Manager Route Planner application developed for ipad, demonstrated ArcGIS s routing and mobile capabilities. This application enabled a DPM to plan a route to visit client accounts and take notes on the go.

35 Simplifying Big Data: Dashboards & Planning Tools Sales Data Executive view Point of sale rolled up through zip code, territory, region, business unit Targeted Route Planner Route planner Demographics Segmentation Business Locations Mobile Routing Manager Mobile (ipad) Route Planner Sales data Distributor data Business locations

36 Simplifying Big Data: Asset and Work Management Web Client Notifications Work Orders Spatial Query Temporal Query

37 Simplifying Big Data: Asset and Work Management

38 Simplifying Big Data: Integrating GIS and Business Intelligence Linking of GIS and BI, two highly complementary technologies, in an integrated solution. Plug Esri Maps for SAP Business Objects (BOBJ) or for IBM Cognos directly into the dayto-day Business Intelligence report viewing, authoring and deployment workflow.

39 Simplifying Big Data: Other Application Areas. Transportation Monitor traffic and regularity Enhance operational efficiency and finance control Monitor maintenance and investment actions Track billing and price options Enhance communication Telecom Data Visualization & Reporting Geo-marketing & Trade Areas Analysis Regulatory & Environmental Compliance Resource Allocation Network Infrastructure Monitoring Sales Territory Management

40 BIG DATA FOR GOVERNMENT CONCLUSIONS Opportunities for Government to leverage commercial case studies Look at Commercial Examples that may be ~70% ready for your applications and require only ~30% customization of the solution Look for short implementation cycles with flexibility to customize business processes Real time updates (compared to batch synchronization in other products) with no data duplication required between GIS and ERP system Scalable, easily extends to multiple workflows / business processes

41 RITRE Corporation November 6,