HP BI Modernization - BI meets unstructured data
Path to a data-driven and agile enterprise Market drivers and trends Example Use Cases Processing of unstructured data Aggregation with classic sources HP BI Modernization strategy 2
What does Big mean? Today data scientist uses Yottabytes to describe how much government data the NSA or FBI have on people altogether. 10 24 10 27 This will be our digital A Geopbyte is 10 30 Bytes In the near future, Brontobyte will be the measurement to describe the type of sensor data that will be generated from the IoT (Internet of Things) Brontobyte universe tomorrow Yottabyte 10 21 Zettabyte 1.3 ZB of network traffic There are~10 19 This is our digital universe today grains of sand on = 250 trillion of DVDs by 2016 the earth Exabyte 1 EB of data is created on the internet each day = 250 million DVDs worth of information. The proposed Square Kilometer Array telescope will generated an EB of data per day Terabyte 500TB of new data per day are ingested in Facebook databases Megabyte 10 18 10 12 10 6 10 15 Petabyte The CERN Large Hadron Collider generates 1PB per second 10 9 Gigabyte
So What? How do you benefit from the 2.5 exabytes humanity creates every day? We don t have better algorithms, we just have more data Peter Norvig, Director of Research, Google
Evolution of business intelligence and analytics The next wave of innovation requires better management of traditional and new forms of data, and will be captured by data-driven and agile enterprises Traditional BI Focus on corporate performance reporting Analytics based on transactional data, after the fact Big Data Analytics Emergence of data companies (Google, Facebook, LinkedIn) Big Data (Social, IoT, Industry 4.0) becomes relevant Predictive analytics emerges Data Science goes mainstream Data-driven, agile enterprise Democratization of data and embedded analytics Data lakes and advanced analytics/ visualization New data sets/types (transactional, human, machine)
A changing world demands a new style of IT Develop Old style of IT Desktop, months New style of IT Mobile, days Operate Secure Govern Monetize Local, procedural, bureaucratic Harden the edge, trust the core Structured data, process Physical, one-to-many Virtual, predictive self-service Borderless, proactive, dynamic Real-time analytics, human information Digital, one-to-one
To remain relevant you must ask different questions What used to be enough What customer? What need? What product? What price? What problem? Why is what really matters Why they buy? Why they care? Why they stay? Why did it happen?
Completing Analytical Vision Traditional Enterprise Data Big Data Dark Data CRM ERP Data Warehouse Web Social Log Files Semi structured Human Information Machine Data
A day in the life of Big Data An intelligent end-to-end approach delivers the right information to the right person at the right time Executive Dashboards Enterprise Search Customer Interaction Predictive Analytics Web Engagement Intelligent End-to-end Approach Variety Velocity Volume Social Media Video Audio Email MGD Texts Transactional Data CRM (Sales) Transactional Operational Strategic Web ERP (Procurement) Supply Chain (Ops) Word, Excel Logs Clickstream Data HR Images Machine Generated Data
Path to a data-driven and agile enterprise Market drivers and trends Example Use Cases Processing of unstructured data Aggregation with classic sources HP BI Modernization strategy 10
Demo
HP Warranty Analytics: Early Defect Detection Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Impact of unexpected defects in the Auto Industry Brand Damage Increased Warranty Cost Litigation 13 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Customer Satisfaction
Defect Detection Connected Sensor Data HP Predictive Maintenance Offering Insights Unstructured Text Non-connected Social & Call Center Conversations Structured Data Text Encoding & Categorization Analytics Reporting Warranty Service Records Unstructured Text HP Early Defect Detection Offering 14 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Early Defect Detection in Warranty Early Detection Late Detection Structured Data Service Records Known Defects Unstructured Text Customer and Service comments go unanalyzed! Text Analytics Unknown Defects found in text Textual Correlations linking Known & Unknown defects Early identification of Unknown defects Reporting Known defects By conducting textual analytics on customer and service commentary, previously unknown defects can be discovered and unknown relationships between known defects can reveal emerging issues faster than traditional methods 15 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
16 Risk Management Use Cases
Trial by Fire: Story about Crisis Response Importance of effectively sensing & responding to disasters Two companies; same crisis event yet strikingly different end outcomes. Nokia: Expanded market share by 30% and profits by 42% Ericsson: Suffered cumulative loss of $ 2.8B, merged with Sony to save the handset business Source: The Impact of Catastrophes on Shareholder Value by Rory F. Knight & Deborah J. Pretty, Oxford University 17
HP Example: Impact due to Floods Multiple crises in the past couple of years have impacted HP significantly I've been on the phone with the heads of all four of our disk drive partners and I'm not even sure they have a complete picture about when they're going to be back up and running. Meg Whitman CEO - HP Notes: Japan earthquake: HP expected loss (link) Thailand floods: HP s revenue dropped (link) 18
Path to a data-driven and agile enterprise Market drivers and trends Example Use Cases Processing of unstructured data Aggregation with classic sources HP BI Modernization strategy 19
7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB 7.2K SATA 2.0 TB UID ProLiant DL380e Gen8 7.2K SATA 500 GB 7.2K SATA 500 GB 7.2K SATA 500 GB 7.2K SATA 500 GB UID UID UID UID UID 1 4 7 10 13 1 4 7 10 13 2 5 8 11 14 2 5 8 11 14 3 6 9 12 15 3 6 9 12 15 7.2K SATA 500 GB 7.2K SATA 500 GB UID 1 4 7 10 13 2 5 8 11 14 3 6 9 12 15 ProLiant SL4540 Gen8 HP Big Data platform Hadoop centric view an HP company Analytics Data Intelligence Security SQL DBMS HP Vertica HP IDOL HP SecureData Trafodion Open Source Hadoop Ecosystem Open Source HP ProLiant / Converged Infrastructure DL380, Apollo 4200, Apollo 4530, Moonshot 1500, Network Cluster Operation HP BSM / HP DSM / HP CMU 20
Vertica, an industry leading analytics database A core component of HP HAVEn Maximize value The Vertica analytics platform is used by more than 2,000 customers worldwide. Here s what it can do for you: Maximize value: a performance leader for concurrent load and query over large data sets. It runs queries 50x to 1000x faster on average than legacy products. Protect and leverage existing IT investment: supports industry standards like SQL, ODBC, JDBC, and R for data analysis. These standards protect existing investments including ETL and BI tools. Massively parallel processing Column orientation Application integration High availability Reduce total cost of ownership: designed for ease of use with features like automated DB design, tuning, recovery, high availability, and backup built in as standard. The Vertica platform s patented columnar compression stores10x to 30x more data per server than row databases. Protect & leverage existing IT investment Advanced compression Automatic DB design Reduce total cost of ownership A 2013 Forrester study showed that HP customers who deployed the HP Vertica Analytics Platform and HP Converged AppSystem for Hadoop saw an ROI of 155 percent and a payback period of less than a month. Performance at scale 21
HP Intelligent Data Operating Layer (IDOL) The OS for human information Single processing layer to handle the continuum of human information Connect Understand Act & Automate Access virtually any source of information Form an understanding of information, including docs, emails, databases, social media, rich media, etc. Over 500 functions to derive actionable insights aka: HP Autonomy IDOL 22
HP Autonomy IDOL HP Autonomy IDOL platform High-performance human information processing 400+ connectors Seamlessly access virtually any enterprise content repository, including file systems, email, or knowledge bases Over 500 functions Leverage the power of functions like sentiment, categorization, and clustering to deliver intelligence and insight 1,000+ file types Process virtually any file type such as text (email, tweet, document), audio, video, and even people profiles & behavior Distributable architecture Achieve big data scalability and high performance with distributable ingest and query architecture All data types, all content repositories unmatched understanding Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP Automomy IDOL: Why is processing human data different? Human Information is made up of ideas, is diverse and has context. Ideas don t exactly match like data does; they have distance. Human Information is not static it s dynamic and lives everywhere. Legacy techniques have all fallen short. Social Media Video Audio Email Texts Mobile Transactional Data Documents Search Engine Images IT/OT Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Unstructured Data Processing Extraction Enrichment/ Filtering Connector Connector Ingestion Image Audio Video Text Document (DOC, XLS, TXT ) Container (ZIP, PST ) Eduction Sentiment Clustering/ Classification (Custom Logic) IDOL Meaning Based Index Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Eduction <Places> Moscow St. Petersburg Washington Syria Russia <Organization> National Security Agency <Names> President Obama Vladimir Putin Edward Snowden Names Places IP addresses Companies Events Relationships Medicines Airports Cars Social Security numbers Phone numbers Credit cards Dates Holidays Job titles Currencies Multi-byte language support including Chinese, Japanese, Thai and Russian and more Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Personalize your data Agents Expertise Communities Expertise Explicit profiling (Agent): user-defined Define your interest using: - Natural language descriptions - Keyword/ boolean rules - Refine by example Automatically monitor information Customizable Share interests with knowledge community Dynamic communities of interest Expert identification Define business rules to guide relationships Automatically form and manage community Collaboration Networks Document rating Consumer groups Profiles Expertise Implicit profiling: capturing behavior data Fully automatic Ongoing monitoring of data consumption and contribution Multi-faceted profiles Always up-to-date Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Search video as easily as text Transform rich media into intelligent assets Automate Automatically create metadata, keyframes, transcriptions Understand Understand video footage and audio streams in real time Act Apply advanced analytics such as clustering and categorization, and link with other file types Live video or playback from archived footage Automatically generated transcript using speech recognition On-screen text recognition Face identification Timecode synchronization Speaker identification Automatic keyframe generation Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP IDOL explore & explore cloud Analyze multichannel data regardless of type, source or location Plug and Play with built-in social media connections Audience-centric reporting widgets Integrate with existing structured data repositories Easy to use no PhD required Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Path to a data-driven and agile enterprise Market drivers and trends Example Use Cases Processing of unstructured data Aggregation with classic sources HP BI Modernization strategy 30
Simplistic architecture Social Media News feeds Video Audio Broadcast Machine Data Data Connectors HP Vertica Analytic DB HP IDOL Indexing and Meaning of Video, Audio, Text Hadoop Data lake & Data Management Visualization & Reporting R Advanced Analytics KPIs Reports Insights Interactions Company Data
Path to a data-driven and agile enterprise Market drivers and trends Example Use Cases Processing of unstructured data Aggregation with classic sources HP BI Modernization strategy 32
Modernized BI: What is the desired state? Hybrid Data Management People Applications Machines Traditional analytics BI/DW Focus on corporate performance and core processes Analytics based on past data Big Data analytics Analytics at the core of business agility Unstructured data and data outside of the enterprise became relevant Emergence of Big Data tools and Data Scientists Embedded analytics Blending traditional and Big Data analytics Embedded prescriptive analytics provides new business value
Catch the next wave of business innovation Business innovation Data-driven products and services Customer experience Operations & IT optimization Managing company risks New business models Corporate and financial performance Discovery environments Analytics solutions New/enhanced applications BI/DW Hybrid Data Management EXPLORE * TEST * SHARE * LEARN ANALYZE * UNDERSTAND * ACT MANAGE* REPORT* COMPLY INTEGRATE *OPTIMIZE * INDUSTRIALIZE Always with appropriate: Security & Information Governance Consumption options: As-a-Service On-premise Managed Internal sources External sources All forms of data $ CRM, SCM, ERP Transactional data Email Texts Documents Images Audio Video Social Mobile Machine Weather GPS Open Media Data Data data Sensor data
Analyzing data is a multi step process Requires Easy To Use tools to meet wide range of skills Types Descriptive Diagnostic Predictive Prescriptive Process Acquisition Preparation Visualization Analysis Presentation Collaboration Data type Structured tables Semi-structured Unstructured Documents Images Audio Video Skill set Business users Programmer Database expert Statistician Mathematician Subject Matter Expert Speed Batch Interactive Real-time
Anatomy of a BI Modernization Journey Example Inability to harness relevant data Advisory Services Need for business agility Data Discovery Services Discovery environments Inability to industrialize analytics Large Investment into new technologies Skilled data scientists Advanced analytics, business accelerators Center of expertise Analytics Solutions Complete stack solution Insufficient talent Security environments Flexible consumption models Open, integrated Big Data platform Hybrid Data Management Services Managed Services Data-driven and agile enterprise 36 Inadequate Technology
Thank You! An expert is anyone who is one chapter ahead of you in whatever book you happen to be reading.