HP Big Data & Analytics for CSPs and Customers. Fouad Bendris / Big Data Lead Enterprise Group PreSales & Strategic Pursuits - EMEA

Transcription

1 HP Big Data & Analytics for CSPs and Customers Fouad Bendris / Big Data Lead Enterprise Group PreSales & Strategic Pursuits - EMEA

2 Agenda Big Data Trends - Challenges & Opportunities Haven Platform : The Big Data foundation HP SPS - Haven for CSPs Use Cases Predictive & Prescriptive Analytics 2

3 Big Data Trends! Challenges & Opportunities

4 IoT - Internet of things The new Data tsunami! Big Data Big Volume! We have gone beyond the decimal system In the near future, Brontobyte will be the measurement to describe the type of sensor data that will be generated from the IoT (Internet of Things) 4 This is our digital universe today = 250 trillion of DVDs Exabyte 1 EB of data is created on the internet each day = 250 million DVDs worth of information. The proposed Square Kilometer Array telescope will generated an EB of data per day Brontobyte This will be our digital universe tomorrow Zettabyte Yottabyte ZB of network traffic Terabyte 500TB of new data per day are ingested in Facebook databases Megabyte Petabyte Gigabyte by 2016 The CERN Large Hadron Collider generates 1PB per second

5 Expansion of the Digital Universe Big Volume = Big opportunity? ZB 1.2ZB ZB ZB Growing by a Factor of 33 in 10 years! More than doubling every 2 years 70 % created by individuals Mostly Unstructured Data ZB IDC projects that the Digital Universe will reach 40 ZB by 2020, an amount that exceeds previous forecasts by 5 ZBs. *Digital Universe = Digital information created or captured in a year 5

6 Accelerating Innovation & Change AOL took 9 years to get to 1 million users Draw Something took 9 days... Facebook took 9 months 6

7 The Age of Unicorns The billion-dollar tech startup was supposed to be the stuff of myth Now they seem to be everywhere! 7

8 The changing Big Data landscape Insight from 100% of Data ~100% Machine Data 90% of Information Human Information Annual Growth ~10% Business Data 8 10% of Information

9 Analytics by numbers: external challenges Required new forms of processing to enable decision making Traditional Enterprise Data 75% of currently deployed data warehouses will not scale sufficiently to meet demand by Big Data 56% Aren t sure how to get value from Big Data 2 Dark Data CRM ERP Data Warehouse Web Social Log Files Machine Data Semi-structured Unstructured Note: Dark data is a type of unstructured, untagged and untapped data that is found in data repositories and has not been analyzed or processed 9

10 Business drivers Adopting Big Data is not a matter of If, but When! Solutions to problems not solvable by traditional approach When existing traditional Data processing solutions can t scale anymore When it is too expensive to extend existing Data processing solutions 10 Opportunities To handle new types of data unstructured and semi-structured To value data that are not used dark data To reduce costs for data storage and processing To speed up processes become more competitive, more profitable To react in real time interact in real time with customers, partners, suppliers, employees, To analyse and forsee the customer behavior (predictive analysis) To develop new product and service offerings based on data

11 How Telco s can leverage Big Data

12 How Telcos can use big data New revenues stream Process optimization Cloud CEM Advertising CDN & IPTV Network Analytics Actionable Experience OTT Co-petition B2B M2M Personalize d Mobile Experience B2C Churn Mgt Ecosystem Indoor Positioning Services Mobility E- Commerce Revenue Intelligence Mobility Next Best Offer 12 In all these initiatives - Big Data & Analytics play a significant role!

13 Low Complexity High Complexity CSPs Big Data : Market Evolution and Economical Impact Customer Experience Management Challenging Today Strategic Next 3 years 13 User Data Consolidatio n Subscription Management & control Advanced Churn and Loyalty Management Archive Optimization Customer experience Assurance Customer Self-care Subscriber Network usage analytics Automatied promotions Revenue Assurance & Protection Hanging Fruits Subscriber preference & Information brokering Subscriber digital identity Intelligent IP routing Customer Care optimization (call reduction) Mobile Advertising Content Recommendation Next best Action engine Automatic problem diagnostic Adoption Complexity Customer Care optimization (Automatic resolution) Next Best Offer (Marketing automation) Intelligent Firewalls Analytics as a Services M2M Analytics WiFi Indoor analytics Actionable Networks (PCRF 2.0) Store of the future B2B Value Added Services Tactical Automatic cyber attack prevention Augmented reality Customer Experience Management Area Actionable Networks ( NFV) Analytics will drive network optimization Time to adopt Use case evolution path Analytics will support new revenue generation Analytics will improve security and revenue protection Analytics will drive Marketing personalization

14 The new Analytics Paradigm 1.0 Descriptive Analytics what did happen and why it did happen? KPIs, KQIs, KBIs, 2.0 Predictive Analytics what is likely to happen now? Net Promoter Scores, Churn Scores, 3.0 Prescriptive Analytics What should I do now? Promote, Recommend, Upsell, 14

15 From Batch To Actionable Information Actionable Real time Streaming and Pattern Detection Predictive Prescriptive Data Value Required Market Focus Interactive Parameterized reports Drilldown visualization Exploration Non-interactive Data preparation Incremental batch processing Dashboards, scorecards Batch Current Market Focus: Data Warehousing and Analytics Operational batch processing Enterprise reports Data mining Sub-second Response Time Hours 15 Transaction Support Data Integrity Real-time Performance Operational Query Optimization Workload Management

16 Introduction to Haven

17 HP Haven Big Data platform HP applications Customer applications Developer applications Gain insight from 100% of your data Analyze machine, business, human data Connect to any existing data source system Scale x faster than legacy systems Develop modern data-driven applications & web services Haven Defined programming interfaces Analytics, context and categorization Premise & OnDemand Scalable data stores & data connectors Social media Video Audio Texts Mobile Transactional Documents IT/OT Search data engine Images Records Compliance archives 17

18 Haven Big Data Platform Haven Hadoop/ HDFS Autonomy IDOL Vertica Enterprise Security n Apps Catalog massive volumes of distributed data Process and index all information Analyze at extreme scale in real-time Collect & unify machine data Powering HP Software + your apps 18 Transactional Social media Video Audio Texts Mobile data Documents IT/OT Search engine Images

19 HP Haven Hadoop 19 HP Reference Architectures and Services for Apache Hadoop Helps design and implement a high-performance, integrated Big Data platform Hadoop Map Reduce Powerful distributed Data processing framework (Java, batch oriented) Hadoop Distributed FileSystem (HDFS) PetaByte scale clustered storage solution with inherent High Availability and Performance features HP Reference Architectures for Hadoop Comprehensive, pre-tested, reference architectures (Cloudera, MapR, Hortonworks) HP Enterprise Design Service for Hadoop Detailed specification design for Hadoop distributions as the guideline for a planned implementation HP Implementation Service for Hadoop Provides risk reduction based on skill sets, project acceleration as a result of applied experience. HP consultants install, configure, deploy the client s architecture, and test it Strategy, design, implementation and infrastructure support

20 HP Haven Hadoop 5 Reasons Why 1 Open Uses Commodity Hardware - no expensive or proprietary hardware required 2 Flexible Supports and processes all data Structured and Unstructured 3 Open Source Developed by hundreds, commercially supported by many 4 ROI Scales horizontally using commodity infrastructure only buy what you need 5 Safe Bet Supported and backed by the largest IT companies in the World Who Uses Hadoop? Large-scale web companies : Yahoo, Facebook, Twitter, Linked-in, ebay, Amazon, Banks, Telco's, Energy providers, Retails, Healthcare, Governments, and HP! HP provides tuned, ProLiant based, reference architectures for Hadoop

21 HP Haven Autonomy IDOL HP Autonomy IDOL platform High-performance human information processing 400+ connectors Seamlessly access virtually any enterprise content repository, including file systems, , or knowledge bases 1,000+ file types Process virtually any file type such as text ( , tweet, document), audio, video, and even people profiles & behavior Over 500 functions Leverage the power of functions like sentiment, categorization, and clustering to deliver intelligence and insight. 15 years and over $280M in R&D, >170 Patents Distributable architecture Achieve big data scalability and high performance with distributable ingest and query architecture All data types, all content repositories unmatched understanding 21

22 HP Haven - Vertica HP Vertica Analytics platform High-performance data analytics platform purpose-built for big data Blazing fast analytics Gain insight into your data in near-real time by running queries 50x-1,000x faster than legacy products Massive scalability Infinitely scale your solution by adding an unlimited number of industry-standard servers Open architecture Protect and embrace your investment in hardware and software with built-in support for Hadoop, R, and a range of ETL and BI tools Optimized data storage Store 10x-30x more data per server than row databases with patented columnar compression Speed, scalability, and openness at lower TCO 22

23 HP Haven ESP ArcSight HP ArcSight Universal log management platform High-performance universal log management to consolidate machine data across IT 315+ connectors Collect, normalize, and categorize machine data such as logs, events, and flows from any device, any time, anywhere from any vendor Search over 1,000,000 events per second The unified machine data through filtering and parsing is enriched with rich metadata, which allows you to search machine data through simple text-based keywords without the need of domain expertise Store years worth of data The unified data is stored through high compression ratio in any of your existing storage formats, eliminating the need for expensive databases and DBAs Analytics & intelligence Built-in content packs, algorithms, rules, and the unified machine data help you deploy IT security, IT operations, IT GRC, and log analytics Collect, store, and analyze any machine data across IT 23

24 n Apps, the n in Haven HP Operations Analytics HP Service Anywhere HP Digital Marketing Hub Analytical security Commercial insurance risk assessment Ecommerce predictive analytics 24 Healthcare analytics Insider threat analysis Service analytics for manufacturing

25 COLLECTION MANAGEMENT CONSUMPTION Analytics platform High level functional Architecture BUSINESS ANALYSTS MANAGEMENT DATA SCIENTISTS APPLICATIONS! Structured Analysis Multi format Analysis Unstructured Analysis Adhoc Reports OLAP Canned Reports Alerts KPI Dashboards Real time Scoring Interaction Optimization Analytic DW Unstructured Human Data Business Data System Data Relational DW Unstructured Human Interaction Data Custom Semi-structured/System Data Enterprise Application Data Chat Call Center Web & Mobile Social Media Audio Video News Media Business Owned 3rd Party Logs Config Status Sensor Data Finance Sales & Marketing Services Supply Chain 25

26 HP & Partner Services ALM BSM ESM Haven Framework Data Flows Apps HP Ops Analytics HP Promote Community & Partner Visual Reporting & Visualization (Tableau, MicroStrategy, SaS, SAP BO, Etc.) Process Integrated Analytics ( SAS, Revolution Analytics, INFR, etc.) Big Data Storage Hadoop Unstructured Storage In Database Analytics (SQL, R & KXEN) Vertica Interactive Analytic Accelerator Source Data Autonomy Unstructured Categorization Arcsight Logger Sensor & System Generated Vertica Structured Data Develop, Deploy, Secure & Manage 26 Transactional Social media Video Audio Texts Mobile data Documents IT/OT Search engine Images

27 Example: Analyzing Twitter Feeds with Haven Extracted tweets from Hadoop Structured tweets in Vertica Sentiments with Autonomy Tableau dashboard HP Anywhere mobile platform 27

28 HP SPS (Smart Profile Server) Haven for CSPs

29 HP Smart Profile Server: Big Data & Analytics Platform Applications (specific analytic use cases) Targeted products and marketing offers IT and Network Experience Optimization New business Models enablement Proactive Data Strategies Software ( BigData architecture) VERTICA - Structured Analytic Database IDOL - Unstructured Analytic Engine HADOOP - Distributed Big Data file systems HAVEn + eium Real Time data collection SPS Analytics run-time engine and designer DRAGON Blue DPI network probes DEL Data Exposure Layer Data Sources Market data Subscriber data Network data Application data 29 Transactional Social media Video Audio Texts Mobile data Documents IT/OT Search engine Images

30 HP & Partner Services HP Smart Profile Server: Haven for Telco s Apps Visual Process HP Ops Analytics HP Promote Community & Partners Reporting & Visualization (Tableau, MicroStrategy, SaS, SAP BO, Etc.) Integrated Analytics ( SAS, Revolution Analytics, etc) SPS Value Packs Campain Mgr 3 rd party Businesses RT Reports/Notifications/APIs RT Value Packs Big Data Storage Hadoop Unstructured Storage SPS / Vertica Interactive Analytics SQL, R Industry Data Models SPS CEP Real-time Analytics Source Data Autonomy Unstructured Categorization Arcsight Logger Security Logs SPS / Vertica Structured Data CEP Streaming Data 30 Transactional Social media Video Audio Texts Mobile data Documents IT/OT Search engine Images

31 Complex Event Processing Requirements for CSPs A massively scalable, real time Complex Event Processing Ability to process millions of events/streams per seconds onfew nodes of commodity hardware Ability to Combine of rules based pattern detection and online based calculations in a box (fraud management, customer experience management, social networks, campaigns feedbacks, clickstreams) A Simplified usage across the event processing lifecycle (development, configuration, deployment, Monitoring) Ability to develop and to deploy rules, kpi logic in minutes Ability to Combine pre-built operators (filters, aggregator, correlators ) without specific skills Ability to Update pattern detection logic based at runtime Transformation of streams into insights with low latency Ability to Collect, detect and notify in milliseconds to update business processes in real-time (Next best Offer, Marketing Campaign optimization) 31

32 Complex Event Processing Terminology Streaming Data : Data coming as a continuous flow from one or multiple sources, synchronous or asynchronous. Streams/events: are tuplets of data regrouping multiple fields such as, session events (start, duration etc ) and measurements (volume, latency measurements) and context/content information such as location, MSISDN, tweets.. Real-time: SPS CEP is designed to process streaming data in real-time e-g between a few milliseconds to a second processing time Event Processing Graphs: SPS CEP Engine executes Event Processing Graphs (EPG) a-k-a topologies. An EPG is a combination of collectors, event processing agents (EPA) and notifiers. A EPG is materially a Storm Topology embedding, some HP IP, a streaming SQL language runtime. They are deployed and run with SPS CEP real-time engine. Their lifecycle (start, stop, ) is done through SPS operation console Event Processing Agents (EPA): are implementing the online calculation/pattern detection logic SPS CEP Designers: are used to build Specify and build automatically EPGs as part of Value Packs SPS Analysis: a set of customizable CEP EPG to allow rapid specification of business rules by business analysts. The SPS BRE relies on SPS CEP technology. 32

33 Use Cases with SPS Predictive & Prescriptive Analytics

34 HP SPS Value Packs - Specific Analytics Packages for CSPs Advertising Next Best Offer Analyzing Subscriber s traffic and application usage to discovery the best tariff plan and best offer to propose for them Ad Experience Personalization Enabling customers to partner with Ad networks, search engines and commerce sites Churn Management Analyzing All Subscriber's inputs (data, traffic, call center requests, application usage, social information, trouble tickets', ) to identify churn probability and propose mitigate actions (new plans, discounts, free services, etc.) Massive Network Traffic Storage Helping Customers to optimize data archives size Content Delivery Network Analytics Real Time analysis of IPTV and Content Delivery Network helps recognize how media contents are consumed by subscriber. Actionable Experience Management This solution helps Customerss Operators to immediately resolve issues of customer experience. Subscriber Network Usage Analytics Real-time analysis of Network usage helps recognize situations of service cannibalization between different services or by OTT services. Customer Experience Management Analyze e2e networks and services to proactively determine subscriber impact and manage subscriber relationship. 34 Revenue Assurance Helping CSPs can reduce or eliminate subscription fraud and proactively recognize Churn. Mobile Experience Personalization Customers can better connect and engage with their users using a mobile app installed on their device.

35 What is NBO? Next-best offer makes use of the predictive analytics and complex event processing technologies to identify the products or services that customers are most likely to be interested in for their next purchase NBO Trigger based marketing Target a customer with his specific need Collaborative Filtering Product preference Computer Based on trouble ticket system and KPI Engine Content Based Recommender Search customers to inform about the offer Find and suggest the best offers to him SPS Recommendation Engine Weighted Average Computer CURRENT FUTURE 35

36 Next best offer Relevant, Contextual and Personalized promotions Tier One Russia: CPR Increase by 5%* Call Center Network Events RT Analytics and Recommendation Engines (HP Smart Profile Server) Internet Usage Data Audio Audio Texts CRM Marketing Data Campaign and promotions engine (HP Campaign Manager) Offers Catalogue Self-Care Mobile interaction (HP SPS MEP) Optional module SMS Smart App Shop 3 rd party Business Self-Care Customer profile management Customer Social Media CDR/XDR Network Documents $ Images Transactional data Customer traffic & service usage Real 36 value to customer + Revenue uptake + Relationship interactions + Churn Mitigation

37 Where NBO Is Used? Harvest customer leads from sales, marketing & other sources Insights into customer acquisitions Target offers to boost customer lifetime value Insights to tailor product creation Improved customer experience Achieving Better ROI Campaign efficiency Incentive management Minimize the net present value of cost for acquiring, retaining, and growing relationship with customer 37

38 Subscriber network usage analytics Enabling CSPs to identify customer behavior inside mobile Internet traffic 38

39 Personalized Advertising Benefits for subscriber and the CSP Get relevant promotions or advertising Get discounts or special offers for CSPs services Enable new revenue streams (3rd party advertising) Improve customer loyalty 84% of Mobile subscribers are Willing to sign up for a plan in which they accept advertising in exchange for free features. Subscriber CSP ~$20B annual revenue by % CAGR. 39 Bell Canada