Key Considerations for a Successful Deployment of Real Time Analytics July 23, 2014

Transcription

1 Key Considerations for a Successful Deployment of Real Time Analytics July 23, 2014

2 Brought to you by Vivit Big Data Special Interest Group led by Kate Fontanella, Pramod Singh, Akshar Dave, Abdul B. Rafi, Doug Stone and Sumit Sengupta

3 Today s Presenters Ritin Mathur Sr. Manager: Big Data Analytics HP Global Analytics Ankush Gupta Manager, Digital Marketing Analytics HP Global Analytics

4 Housekeeping This LIVE session is being recorded Recordings are available to all Vivit members Session Q&A: Please type questions in the Questions Pane

5 Agenda Evolution of BI and Analytics Real Time Analytics Application of Real Time Analytics Key considerations for successful deployment

6 Business Analytics Maturity and Evolution Business Intelligence Integrated data across enterprise Transactions, marketing touch, service events, financial data, external data Data preparation, compilation, classification and visualization Advanced Analytics Analysis, modeling optimization and simulation Strategic choices and long term decision making Real Time Analytics Responding to events as they happen Rules based workflows based on advanced analytics Automated decision support systems

7 Business Analytics Maturity and Evolution Business Intelligence Customer Loyalty Who are my most loyal customers? Advanced Analytics What drives customer loyalty Real Time Analytics What in-store offer should I provide? Cross Sell / Upsell What % of product a is sold with product b What are the drivers What product should I present to customer X buying product a?

8 Real Time Analytics in Action Hindsight Insight Foresight Process Descriptive Predictive Prescriptive Data highly structured unstructured; business & social People report writers operations staff data scientists data warehouse, KPI Tech. analytics databases, dashboards predictive systems prediction, optimization & simulation systems Real Time Analytics

9 Real Time Analytics The time element in Real Time is context driven In general, real-time analytics can be defined as the use of, or the capacity to use, available enterprise data and resources when needed. It consists of dynamic analysis and reporting, based on data entered into a system less than 60 seconds ago. - SAP Real-time business intelligence (RTBI) is the process of delivering information about business operations as they occur. Real time means near to zero latency and access to information whenever it is required - IEEE

10 Real Time Analytics Use Cases Marketing Sales Customer Supply chain Risk mgmt. HR, IT Function Real-time location based offers Event based marketing Lead generation Sales optimization Customer experience Customer support Demand-supply planning Procurement analytics Systemic fraud models Compliance and risk Workforce optimization IT performance management Finance Government Telecom Manufacturing Energy Healthcare Industry Fraud detection Anti-money laundering Risk management Law enforcement Counter terrorism Traffic flow optimization Broadcast monitoring Churn prevention Advertising optimization Supply chain optimization Defect tracking Failure Prediction Weather forecasting Natural resource exploration Drug development Scientific research Evidencebased medicine

11 Evolution in data landscape Traditional DW/BI Evolving Current State of DW/BI Converged big data Future Info Apps Infra Can be fully automated rigor is required Restricted on types of data Transaction management (OLTP) Volumes of data (Gbytes Terabytes) Traditional DW/BI In-Database analytics Unstructured data batch processing - Hadoop Advanced Analytics NLP and Artificial Intelligence Latency, compression and speed Requires human intervention Coverage is important rather than rigor Amount of data can be Tbytes Pbytes Improves the system performance by scale-out Statistical data creation, retrieval, and data mining Traditional DW/BI In-Database analytics Converge d Infra. Advanced analytics Hadoop New understanding of all multi-structured data Real-time advanced analytics Superior speed with low latency Process information inmemory, In-time, in-place

12 Big Data landscape Annual Growth ~100% Machine Data 90% of Information Human Information ~10% Business Data 10% of Information

13 Real Time Analytics and Big Data Real Time Analytics Big Data In-Motion Datasets High Velocity Del Intel.. Real Time Actio n Triangulated Datasets Large Datasets High Variety High Volume

14 Real Time Big Data Platform HP HAVEn The Leading Big Data Platform HAVEn Hadoop/ HDFS Autonomy IDOL Vertica Enterprise Security n Apps Catalog massive volumes of distributed data Process and index all information Analyze at extreme scale in real-time Collect & unify machine data Powering HP Software + your apps Transactional Social media Video Audio Texts Mobile data Documents IT/OT Search engine Images hp.com/haven

15 Case Study: Smart Content Delivery in Real Time Engagement Placement Engine Business Constraints Rules Engine Conversion Recommendati on Engine Strategy Designer Demographic Behavioral Attitudinal Data Analysis IC Macro-events data feeds Content Consumption Content Repository Campaign Response Clickstream User Preferences

16 Real Time Analytics Pit Falls Key Requirements Design Deployment Usage Scalability Compatibility Availability Relevance Responsiveness Controllability Common Pit-falls Conventional system design unable to scale up to in-motion data and in database analytics Absence of common interfaces and templates for information sharing across disparate systems In-motion information capture incomplete and off-line intelligence not snapshotted appropriately for triangulated decision making Critical and relevant information trapped in unstructured data, not tapped and unaccounted for intelligence generation Plug-ins designed to deploy static business rules, aren t necessarily scalable to the real time rule deployment thus, becoming very static Lack of sufficient monitoring and triggers resulting in a delayed response to an emergency

17 Key Considerations to avoid pit falls Following are the key considerations for Analytics Managers to successfully deploy Real Time Analytics: 1. Plan for the real time at the start of the design itself and keep Big Data as core of this design 2. Define mutually exclusive but collectively exhaustive process and algorithms aligned to key KPIs 3. Install sufficient checks and balances for business to control the outcome and efficiently deploy the analytics

18 Plan Big to achieve Real Transformation Plan for in-motion data rather than static data sets and identify relevant scenarios Tap in the true value by co-hosting analytics and data in One Platform plan for indatabase algorithms and scoring in real time Big Data at The Core Not a wrapper In-Motion Data Design In-Database Analytics Big Data In-Built Plug-ins In-Real Time Data Warehousing Big Data Platform Key Enabler Define plug and play APIs to deploy output of real time algorithms Scalable system especially to handle high volume unstructured data. Also, define recovery plans for big data as well

19 Achieve Convergence through Alignment Define Optimize Business Process Scenario Designs Analytics consumption process Individual process deployment Business KPI monitoring Realignment Algorithms Business Scenarios Individual Definitions Integrated Training De-dupe (MECE) Validation

20 Deploy Smart Controls to build Smart Systems Intelligent Control System Internal Strategies Marketing and Sales Products Finance 1 3 Business Control objectives and constraints System 2 Test results 5 and corrective feedback Rules Development 4 Compliance Audit 6 Rules and Guidelines Structured and Unstructured Information External Factors Governments and Law Customer Sentiment Industry Players Customers Trust Personalized Information Security Transparent Alignment to company s commitment to customer Tribal Augurs community development of loyal customers Business Tactics Responsive to business strategies Technology Efficiency gains around key KPIs Touch Optimized for best next action

21 Case Study: E-Commerce Platform Transformation Plan Align Control Big Database Architecture Hadoop and Vertica as core platforms for data aggregation and management In-Motion Data Design Live streaming of data through different APIs to Hadoop 3 layered data architecture client, webserver, data server In-Database Analytics Using Vertica-SQL and Distributed R In-built plugins Personalization and Recommendation APIs Catalogue process Real time product listing, pricing and stock/shipping Flash event support User experience process Personalized navigation and filter options Cross-sell and Up-sell recommendation Demand conversion process Save the sale interception Banner and discount deployment Synchronized Communication Real time KPI monitoring Traffic tracking User experience Inventory management Recommendation control Restrict incompatible bundling Rule malfunction alerts Min and Max thresholds for critical indicators Alerts and corrective workflows Fraud alerts Check on fraudulent use of deal codes Payment option approvals

22 Summary

23 Thank you

24