Big Data and Advanced Spatial Analytics

Transcription

1 Big Data and Advanced Spatial Analytics Eve Kleiman Director, APAC Technology Product Releases & Management Programs Oracle 1 Copyright 2012, Oracle and/or its affiliates. All rights

2 Disappearing line between Geospatial Technologies and Information Technologies SOA Mapping Digital data file Geographic Information Systems Spatial Information Technology 2 Copyright 2012, Oracle and/or its affiliates. All rights

3 Location Infused Technology Java, Databases, BI, Applications, Cloud 3 Copyright 2012, Oracle and/or its affiliates. All rights

4 Internet of Things The New Infrastructure 4 Copyright 2012, Oracle and/or its affiliates. All rights

5 Internet of Things Open a World of Possibilities Total economic value add estimated at $1.9 Trillion, Source:Gartner 5 Copyright 2012, Oracle and/or its affiliates. All rights

6 Need for an Enterprise Platform Standardization Streamline how IoT applications are developed, secured & deployed Real-Time Analytics Manage IoT data from collection, storage, processing and analysis Integration Connect intelligent devices to existing enterprise applications Security Manage security and identity of data, devices, enterprise applications 6 Copyright 2011, Oracle and/or its affiliates. All rights

7 Definition: What is Big Data Anyway? The 4 V s of Big Data Volume It s about 100s Terabytes and Petabytes... BUT could be smaller volumes size is in the eye of beholder Velocity Relates to streams of high frequency data arriving e.g. From a sensor, network, Imagery, UAV, or high volume event Variety Highlights the new types of data from the internet of things that can be stored and analysed Value At volume low value data can highlight high value patterns, trends and insights of significant commercial value 7 Copyright 2012, Oracle and/or its affiliates. All rights

8 Spatial Data From Scarcity to Abundance Mobile Sensing & Geocoding Mobile tracking Automated Data Capture / Geocoding Filtering / Geocoding 3D data into mainstream Apple, Google, Microsoft 3D city models, terrains, imagery Enhancing Web and Enterprise apps Business intelligence & operations 8 Copyright 2012, Oracle and/or its affiliates. All rights 8

9 The Business Opportunity 100 million daily Conquering Big Data 1 billion visitors / yr Web Transactions ecommerce 1 million daily Image uploads Claims analysis Insurance If you could have all the data you wanted, what would you do differently? 10 billion daily Device syncs Consumer 10 million meters Hourly uploads Utility 20 million daily Monitoring Healthcare 9 Copyright 2012, Oracle and/or its affiliates. All rights

10 Sample of Big Data Use Cases Today AUTOMOTIVE Auto sensors reporting location, problems COMMUNICATIONS Location-based advertising Retail / CPG Sentiment analysis Hot products Optimized Marketing FINANCIAL SERVICES Risk & portfolio analysis New products EDUCATION & RESEARCH Experiment sensor analysis HIGH TECHNOLOGY / INDUSTRIAL MFG. Mfg quality Warranty analysis OIL & GAS Drilling exploration sensor analysis LIFE SCIENCES Clinical trials Genomics Games Adjust to player behavior In-Game Ads MEDIA/ ENTERTAINMENT Viewers / advertising effectiveness Cross Sell TRAVEL & TRANSPORTATION Sensor analysis for optimal traffic flows Customer sentiment ON-LINE SERVICES / SOCIAL MEDIA People & career matching Web-site optimization UTILITIES Smart Meter analysis for network capacity, HEALTH CARE Patient sensors, monitoring, EHRs Quality of care LAW ENFORCEMENT & DEFENSE Threat analysis - social media monitoring, photo analysis 10 Copyright 2012, Oracle and/or its affiliates. All rights

11 What Does a Big Data World Look Like? Truck / Motor Manufacturer What they collect Monitors the complete car Kilometers Per Litre, Driving techniques Location information How they use it Better tailored servicing plans Better targeted marketing Offer better finance deals or related options More data for R&D Sell on to partners Big Data means The ability to predict when components might fail Create better service plans based on actual usage Drivers can post their data to manufacturers website Black box data can help with accidents 11 Copyright 2012, Oracle and/or its affiliates. All rights

12 Examples Smart City: Parking Management Verizon Telematics provides a full service end-to-end solution Supports over 50 Telematics features (vehicle, driver, social, etc.) Verizon Telematics provides a modular platform Verizon Telematics supports a curated ecosystem The global first implementation of a city wide smart parking management system and technology to manage parking supply and demand more intelligently. Sensors at 8,200 of the city s 27,000 metered parking spaces, to get information from the gate arms at the city s 14 garages, which among them have about 13,000 spaces 12 Copyright 2012, Oracle and/or its affiliates. All rights

13 Examples (cont.) Telematics Provide a composite identity for customers and cars Need for a reliable 24/7 IAM infrastructure Leverage Mobility/Web technologies to provide advanced Infotainment and Safety Services Subsidiary of GM that provides subscription-based communications, invehicle security, hands free calling, turn-byturn navigation, and remote diagnostics systems OnStar relies on Oracle to deliver service to 5.5 million car owners Location Intelligence Company GIS, Mobile, LBS products Spatial Content & Platform Provider InfoMobil vehicle tracking and dispatching system 800 Customers, 2 Operation Centers, 50,000+ Mobile Devices, Users 13 Copyright 2012, Oracle and/or its affiliates. All rights

14 What Does a Big Data World Look Like? Retailer / White Goods What they collect Food monitoring by RFID tags Fridge monitors food usage and sell-by dates Video Surveillance Feedback / comments How they use it Automated food ordering Better targeted marketing due to better data about how/when customer uses products Big Data means More detailed information about customers Better store based demand planning 14 Copyright 2012, Oracle and/or its affiliates. All rights

15 Oil & Gas Industry New ways sit along side existing ways to generate data 3C/4C Seismic Sensors Source Systems Data Warehouse Business Users Micro-seismic monitoring Time-lapse Seismic data FIINANCIALS PROCESS MANUFACTURING REAL-TIME DATA EXTRACTS Enterprise Data Warehouse Monitoring Infrastructure Borehole Data Acquisition PROJECTS CONTRACTS ENTERPRISE PROJECT PORTFOLIO MANAGEMENT TELESERVICE EXTRACTS Data Marts Environmenta l Monitoring Asset Management ENTERPRISE ASSET MANAGEMENT FEEDS 15 Copyright 2011, Oracle and/or its affiliates. All rights

16 Telco Industry Location based marketing Big Data Use Cases Real-time analytics combined with complex event processing to create analysis across the GPS data that a service provider can collect to understand the customers current location or look for patterns or relationships. Network optimization Big Data is used to deliver real-time analytics to detect when a network is down, overloaded or reaching capacity. Sentiment analysis Use of social media (Twitter, Facebook, etc.) - this is not just comms specific but a good example applicable across industry. Collect/stream data from social media sites into CRM and customer service applications to determine importance/ clout of customer and to get a better overall picture of each customer s behavior, likes and dislikes. Weblog or clickstream analysis Applicable across industries; allowing service providers to better understand how their web site or online store is navigated. 16 Copyright 2012, Oracle and/or its affiliates. All rights

17 Healthcare Industry Complete secure solution from data collection to analysis to decision Incoming Data Pharmaceutical Clinical EMR Stream Acquire Organize /Discover Analyze, Visualize Decide Metabolic Activity / Claims Cost Biometrics Lifestyle/History Track outcomes Track treatments, notes Track social media, payments Correlate Omics w/emr data Correlate biometrics data across devices Model protocols Model outcomes Model normal behavior Visualize data Discover unknowns Highlight outliers Determine risk Populate metrics Report Prevent Predict Diagnose Treat Measure Social 17 Copyright 2012, Oracle and/or its affiliates. All rights

18 Nike Healthcare Fuelband monitors active lifestyle of 8 million users on a daily basis Current data grid volume is approximately 150,000 request per minute with about 40 million objects at any given time on the grid 18 Copyright 2012, Oracle and/or its affiliates. All rights

19 Objectives FSI - Bank Real-time Location Based Offers Increase revenue through real-time, location based offers Solution Customer profile enrichment with Big Data Capture credit card POS and merchant data with event processor Determine geo location of POS and nearby bank wholesale customers Leverage real-time decision engine to generate offer to mobile device 19 Copyright 2012, Oracle and/or its affiliates. All rights

20 The Challenge Recognizing a Pattern Predicting Behavior Finding Useful Data Modeling, Statistics Econometrics Decision Science Psychology 20 Copyright 2012, Oracle and/or its affiliates. All rights

21 Big Data in Action DECIDE ANALYZE ACQUIRE ORGANIZE Make Better Decisions Using Big Data 21 Copyright 2012, Oracle and/or its affiliates. All rights

22 Big Data In Action DECIDE ACQUIRE Acquire all available data ANALYZE ORGANIZE 22 Copyright 2012, Oracle and/or its affiliates. All rights

23 Hadoop Architecture Management/Monitoring Distributed file system MapReduce Hadoop Distributed File System (HDFS) Map/Reduce programming paradigm Highly scalable data processing 23 Copyright 2012, Oracle and/or its affiliates. All rights

24 NoSQL Databases A Distributed Key-Value Database Application Application Simple data model NoSQL Driver NoSQL Driver Delete Read Update Read Scalability High Availability Transparent Load Balancing Nodes West Nodes Central Nodes East Simple Administration 24 Copyright 2012, Oracle and/or its affiliates. All rights

25 Real-Time Streaming Data Real-time Data Streams Complex Event Processing Aggregate, Correlate, Filter Immediate Automatic Responses Console Alerts Pattern Detection SMS Message Event capture Workflow Initiation Enrichment Virtual Data Repository Real-time Dashboards 25 Copyright 2012, Oracle and/or its affiliates. All rights

26 Choose the RIGHT tool for the job Hadoop Distributed File System (HDFS) NoSQL Databases Relational Databases File System Key Value Store Relational Database No inherent structure Simple data structure Complex data structures, rich SQL High volume writes Limited functionality, roll-your-own applications Batch Oriented High volume random reads and writes Simple get/put high speed storage, flex configuration Real-Time, web-scale specialized applications High volume OLTP with 2-PC Security, Backup/Restore, Data life cycle mgmt, XML, etc. General purpose SQL platform, multiple applications, ODBC, JDBC 26 Copyright 2012, Oracle and/or its affiliates. All rights

27 Big Data in Action DECIDE ACQUIRE Organize and distill big data using massive parallelism ANALYZE ORGANIZE 27 Copyright 2012, Oracle and/or its affiliates. All rights

28 Big Data Challenge Aligning Data Types Big Data: Decisions based on all your data Semantic Graphs Documents Social Data Machine-Generated Data Geospatial Where is it Happening? In What Context? Video 28 Copyright 2012, Oracle and/or its affiliates. All rights

29 RDF for Enterprise Integration Access & Presentation Layer RDF metadata layer (integrated graph metadata) Index Data Servers Event Server Big Data Appliance Content Mgmt BI Server Data Warehouse Data Sources / Types Machine Generated Data Social Media Human Sourced Information Subscription Services Transaction Systems 29 Copyright 2012, Oracle and/or its affiliates. All rights

30 Graph Analysis Represent your data as a graph, analyze it and discover useful information By considering relationships between data entities Recommendation customer items Influencer Identification Pattern Matching P1 ex P2 friend friend friend P3 P4 Purchase Record Communication Stream (e.g. tweets) Create recommendations for a specific customer from its neighbors in a purchase graph Identify nodes that are critical to connectivity of an information flow graph Find sub-graphs that match a specified pattern 30 Copyright 2012, Oracle and/or its affiliates. All rights

31 Big Data in Action DECIDE ACQUIRE Analyze all your data, at once ANALYZE ORGANIZE 31 Copyright 2012, Oracle and/or its affiliates. All rights

32 Navigation and Visualization of RDF graphs 32 Copyright 2012, Oracle and/or its affiliates. All rights

33 Web Mapping with GeoSPARQL 33 Copyright 2012, Oracle and/or its affiliates. All rights

34 Big Data Tools for Exploring & Analyzing R Statistical Programming Language Open source language and environment Used for statistical computing and graphics Strength in easily producing publication-quality plots Highly extensible with open source community R packages 34 Copyright 2012, Oracle and/or its affiliates. All rights

35 Big Data Tools for Exploring & Analyzing Data Mining Tools for Guided Pattern Discovery & Statistical Analysis Problem Classification Sample Problem Classification Given demographic data about a set of customers, predict customer response to an affinity card program Regression Given demographic and purchasing data about a set of customers, predict customers' age Attribute Importance Given customer response to an affinity card program, find the most significant predictors Copyright 2011, Oracle and/or its affiliates. All rights Copyright 2012, Oracle and/or its affiliates. All rights

36 Big Data Tools for Exploring & Analyzing Automated Discovery & Predictive Analysis Problem Classification Anomaly Detection Association Rules Sample Problem Given demographic data about a set of customers, identify customer purchasing behavior that is significantly different from the norm Find the items that tend to be purchased together and specify their relationship market basket analysis Clustering Segment demographic data into clusters and rank the probability that an individual will belong to a given cluster Feature Extraction Given demographic data about a set of customers, group the attributes into general characteristics of the customers F1 F2 F3 F4 36 Copyright 2012, Oracle and/or its affiliates. All rights

37 Big Data Extends the Depth of Analytics Graph Analytics Statistics Query and Reporting Data Mining 2 miles Spatial Analytics Text Analytics 37 Copyright 2012, Oracle and/or its affiliates. All rights

38 Big Data in Action DECIDE ACQUIRE Decide based on real-time big data ANALYZE ORGANIZE 38 Copyright 2012, Oracle and/or its affiliates. All rights

39 Summary 39 Copyright 2012, Oracle and/or its affiliates. All rights

40 Q&A MORE INFORMATION Copyright 2012, Oracle and/or its affiliates. All rights