Mike Luke National Practice Leader SAS Canada. The Evolution of Data and New Opportunities for Analytics

Transcription

1 Mike Luke National Practice Leader SAS Canada The Evolution of Data and New Opportunities for Analytics

2 Evolution of Data WE VE GROWN OVER THE LAST 60 YEARS

3 Evolution of Data IN THE EARLY DAYS

4 Evolution of Data THE CONNECTIONS STARTED TO GROW

5 Evolution of Data THEN CAME THE NEW PLATFORMS AND CONTENT

6 Evolution of Data THEN THE APPS EMERGED DATA GREW AT AN INCREDIBLE PACE

7 Todays trends RAPID GROWTH OF DATA 1 gigabyte/second 8 terabytes/day 40 terabytes/hour

8 Evolution of Data INTERNET OF THINGS NOW WHAT?

9 Trends today INTERNET OF THINGS

10

11

12

13 Trends today INTERNET OF THINGS

14

15

16 BIG DATA IS EVERYWHERE Internet Social Media Customer Interactions Bank Transactions Financial Feeds Telecommunications Sensors Intelligent Devices.and many more

17 BIG DATA - HOW TO PROCESS IT? BIG DATA FOCUS IS SHIFTING TO STREAMING DATA ANALYSIS FOR LOW LATENCY DECISION MAKING

18 World of Events EVENTs versus DATA EVENT is an occurrence happening at a determinable time and place, with or without the participation of human agents. Let s agree: an eventis an occurrence happening at a determinable time and place that can be recorded at that time as acollection of fields. Data is event-related:group of eventsthat happened is a collection of facts and rightfully might be treated as data. Should we work with events then traditional approaches, which apply analytics after data is stored, may provide insights too late and thus they lead to loss of opportunities. Workingwith events allows you to react faster and therefore not to lose those opportunities!

19 Data processing at High Speed EVENT STREAM PROCESSING ENGINE DATA IN (called Events) Large volume of streaming data flowing at very high rate : Millions of records/sec Data (Events) are processed as soon as they arrive (happen) : Latency: milliseconds DATA OUT (Events)

20 Data Processing ON THE MOVE BATCH ENGINE STREAM ENGINE 1. Prepare data 2. Run Process 3. Get results 4. Go to step 1 1. Run Process 2. Continuous: a. Receive data in b. Process data c. Push results out

21

22 Detect Events of Interest in the stream of incoming data EVENT STREAM PROCESSING ENGINE Detect Events & Patterns of Interest DATA IN (called Events) Filtering Aggregation Pattern detection Computations Correlations Procedural Text analytics Data quality and much more DATA OUT (Events)

23 STREAMING DATA ANALYTICS: BENEFITS BUSINESS USERS IT USERS Descriptive, Predictive, and Optimization Analytics on continuous data streams, right now! Access to streaming data information Reduced time to decision and action Generation of new opportunities Reduced risk and cost Ability to deal with the increasing volume and speed of big data flow Reduced storage (due to data aggregation) and computational needs Faster to market Easier & quicker maintenance of analytical models Focus on business logic rather than production features

24 SOME USE CASES FOR STREAMING ANALYTICS ECOMMERCE OPTIMIZATION Clickstream Analysis Optimize user experience Real time marketing and advertising CONNECTED DEVICES (IoT) Real time sensors survey RT anomalies detection Complement Predictive Analytics RT triggering and decision DECISION MANAGEMENT Complement EDM RT decisions on event streams Deploy additive and incremental models FRAUD DETECTION Real time transaction analysis User behavior detection Customer profile correlation RT alerts and case management TELECOMUNICATIONS CDRs analysis Real Time Marketing Fraud detection IT systems survey CAPITAL MARKETS Complement High Perf Risk Reduce time from trading to reporting Continuous calculations

25 PATTERNS OF INTEREST SAMPLE QUESTIONS CONNECTED DEVICES FRAUD DETECTION REAL TIME MARKETING CAPITAL MARKETS SUPPLY CHAIN Give me the top 3 values every 5 minutes Tell me when an event Ais followed by an event Band not an event Cwithin 3 minutes Detect when a sensor reading is largely above the last hour rolling average value What is the average throughput over a sustained period of time? What is the peak throughput in any rolling short period of time? If the time between in-store credit card transactions in different cities is less than the travel time between those cities, put the account on hold and flag it for investigation. Detect when a customer execute a transaction that does not match to his usual behavior profile How many website visitors are going from the home page to about company, and clicking my profile during a rolling, 10-minute window? Detectwhen a user goes to a competitors web site, comes back to our store and abandons their basket Detect when a broker buys securities for his own account before buying the same securities for his customer, then sells when the price rises; Orisattempting to influence the closing price of a security by executing purchases at the close of normal trading hours. When the stock levels for the book 50 Shades of Grey drops to 10% of the minimum stock level given the last 10 hours of buying behavior; send an event to begin the re-stocking process to the distribution center.

26 CONNECTED TRUCKS - KEEP THEM ROLLING Business Goal Predict maintenance needs of individual trucks before failures occur Proactively service trucks at opportune time Provide new service offering with high SLA Process: Data from 60+ sensors/truck. Integrate the data with product details, warranty claims, and related data sources. Build analytic models to predict the likelihood of specific failures within 30 days. PoC Results Models able to predict likely failures 30 days in advance with 90% accuracy Better root cause insight led to smaller campaign Image credit: Mike,

27 Telco Provider SUB-SECOND OFFER DECISIONS When speed matters, time is money Fast filtering of events of interest Response time faster by more than an order of magnitude! 10x Increase in Offer Acceptance Latency ms Events coming in 20,000 requests/sec REAL TIME DECISION MANAGER ESP Relevant events 15 requests/sec Latency 5-15 ms Copyright 2012, SAS Institute Inc. All rights reserved. Image:

28 CRITICAL COMPONENT FAILURE AVOIDANCE CHALLENGE Monitoring Electronic Submersible Pump (ESP) Efficiency and Well Performance for deep sea drilling in the Gulf of Mexico. ESP s are 1500 horsepower pumps operating some 8,000 feet below sea level. They pump oil 14-15,000 feet from below the ocean floor

29 Adding Multi-phase Analytics: Streaming Analytics front-ending historical statistical analytics ESP RDBMS ESPs store the queries and continuously stream data through the queries Close to edge/sensor Thinnest Data Databases store the data and periodically run queries against the stored data EVENTS INCREMENTAL RESULTS QUERIES RESULTS Reporting can occur throughout lifecycle

30 LOGICAL ARCHITECTURE RESULTS Failure of one of these pumps = $2M/day Platform generates 100K barrels/ day Real-time Reliability/ Performance Management Safety/ Maintenance Asset Efficiency/ Profitability Management One day of lost productivity equates to $20M in deferred revenue Copyright 2012, SAS Institute Inc. All rights reserved.

31 Patient Management CAPTURING CRITICAL EVENTS REAL-TIME Patient Vitals Monitoring Critical Alerts Patient Care Team Lab Results ESP Engine Laboratory Analytics Observations Medical Staff

32 Final thoughts on Analytics for Real-time Decisions 1. Emphasizes a greater requirement for updated and retrained models for the best possible outcomes 2. Requires focus on effective governance to ensure what to capture, cleanse, store, and process 3. Will ultimately provide the ability to gain new insight for operational efficiencies, reduced operations risk, churn avoidance, fraud detection, cyber security, personalized marketing

33 THANK YOU EVERYONE! Copyright 2015, SAS Institute Inc. All rights reserved.