Solving Big Data Issues with Big Data Based Solutions

Transcription

7 Webinar Abstract More and more firms are trying to determine how they can best-use the mountains and mountains of information in their possession. Banks need faster data processing and sophisticated analytics to achieve a 360-degree view of the customer, developing true relationship-based pricing, and achieve critical objectives for banking transformations such as a customer-focused enterprise, optimized enterprise risk management, and more flexible and streamlined operations At one of the biggest retailers in N. America, customer analyses and market trends taking 30days, were reduced to 3hours using Big Data. Not only that, but now they are able to launch marketing campaigns tailored to buying patterns, customer demographics and event. Healthcare institutions have used Big Data based solution to execute healthcare data analytics and provide workflow information that helps the entity to customize business services for individual physicians, and also helps the company develop educational materials to help train physicians on the most effective Big data projects that are making city government smarter and more efficient make it easier to spot patterns and pull meaning from information such as sensors and video feeds that can t be neatly organized in a spreadsheet. Federal entities are exploring how big data can democratize scientific research and better spot waste, fraud and abuse. How do new Big Data capabilities turn into tangible value? Is the business value of Big Data being able to do something faster today than yesterday? Solving Big Data issues, actually requires a Big Data solution. What is common to all of these examples apart from Big Data, is that IT should innovate to provide agile means to support the business, whatever means the business prefers to choose to facilitate delivery to their customers. In this session we describe some of the above examples, and tell you how HP Software is providing leading, innovative Big Data based solutions to help IT in this innovation. 7 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

8 Agenda Big Data trends today Adoption and application in the industry Is important to manage Big Data Resources How HO can help Applying Big Data to IT Operations Call to Action Q&A 8 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

9 Accelerating innovation & time to value Every 60 seconds Burroughs IBM NEC Unisys Hitachi Product Configurator Mainframe Kilobytes Fijitsu Bull Claim Processing Payroll Sales tracking & Marketing Commissions Bills of Material SCM Costing ERP Database Manufacturing Projects CRM Quality Control HCM SAP Engineering HP MRM Inventory EMC Cost Management Cash Management Time and Expense Accounts Receivable Billing Activity Management Training Time & Attendance Data Warehousing Service Order Entry Joyent Client/Server Megabytes Rostering HCM Fixed Assets PLM Intacct DCC Saba CCC IntraLinks Adobe Microsoft Cornerstone ondemand Softscape Plex Systems ebay Quickbooks NetSuite OpSource Hosting.com Tata Communications Datapipe PPM NetDocuments Microsoft Corel Saba Softscape Volusion Google Ariba Alterian ADP VirtualEdge Kenexa OpenText Workscape Yahoo Quadrem Xerox SugarCRM FinancialForce.com Avid NetReach Zoho Serif Yahoo! CyberShift Sage Sonar6 Hyland Music Qvidian kaggle SuperCam SLI Systems Elemica SCM Kinaxis Xactly CyberShift SmugMug ihandy The Internet Gigabytes NetSuite Exact Online PaperHost Sonar6 Facebook Fring Rackspace SolidFire Snapfish GoGrid Atlassian Paint.NET Amazon DocuSign Dragon Diction buzzd Cookie Doodle Ah! Fasion Girl Hootsuite Renren Education Flickr UPS Mobile salesforce.com Bromium Scanner Pro Foursquare nebula Zynga ischedule Pandora Khan Academy BrainPOP MobileFrame.com myhomework Toggl Xing MailChimp Amazon Web Services LimeLight News LinkedIn Workday Navigation SuccessFactors Associatedcontent MobilieIron Twitter AppFog PingMe cloudability CloudSigma HP eprint RightScale Fed Ex Mobile SmugMug YouTube Business Amazon Utilities Scribd. Zillabyte Reference Games Productivity TripIt Twitter Atlassian Baidu Atlassian Tumblr. New Relic Urban Parse Yandex Finance Workbrain Yandex Mozy Zynga Entertainment Jive Software Qzone Travel box.net Mixi Heroku CYworld BeyondCore Taleo Lifestyle Splunk ScaleXtreme Pinterest Sport Mobile, Social, Big Data & The Cloud Zettabytes Photo & Video Yammer Answers.com Viber Social Networking dotcloud PingMe 98,000+ tweets 695,000 status updates 11million instant messages 698,445 Google searches 168 million+ s sent 1,820TB of data created 217 new mobile web users Yottabytes 9 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

10 Big Data Market Trends Big data market trends IDC FY13 Market Spending: $10 Billion, CAGR 40%, 30% Software Big data pure play vendors : $0.5 Billion FY12 Vertica (NO.1), Opera, Aster Data, Splunk, Greenplum Traditional players : Teradata, Oracle, IBM, EMC, SAP etc. IDC Hadoop-Map reduce software ecosystem FY13 $378 Million, CAGR : 60% HADOOP - Open Source for Big Data VERTICA - Next Generation Data Warehouse 10 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

11 There is a huge opportunity 100% 10x 100x! Growth Real time info processing $100B SAP, Oracle, Siebel, PeopleSoft... n-apps 10% Relational Technology ( ) 10x 100% 11 10% Information monetized Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

12 Innovative companies are changing rules of the game All driven by the power of big data Develop disruptive business models Create better products and services Enhance customer experience Drive sustained competitive advantage Leverage your data, make it matter 12 Copyright Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

13 Big Data opportunities across industries and use cases Innovative analytic use cases are cutting across structured, unstructured and semi structured data Finance Government Telecom Manufacturing Energy Healthcare Fraud detection Anti-money laundering Risk management 13 Sentiment analysis Social CRM / network analysis Churn mitigation Brand monitoring Cross and Up sell Law enforcement Counter terrorism Traffic flow optimization Loyalty & promotion analysis Web application optimization Broadcast monitoring Churn prevention Advertising optimization Marketing campaign optimization Brand management Social media analytics Pricing optimization Internal risk assessment Customer behavior analysis Revenue assurance Supply chain optimization Defect tracking RFID Correlation Warranty management Horizontal use cases Weather forecasting Natural resource exploration Logistics optimization Clickstream analysis Influencer analysis IT infrastructure analysis Legal discovery Equipment monitoring Enterprise search Drug development Scientific research Evidence based medicine Healthcare outcomes analysis Sources: Copyright IDC: Hewlett-Packard Worldwide Development Big Data Technology Company, L.P. and The Services information Forecast: contained , herein is subject Gartner: to change 2012 without Big Data notice. Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending Through 2016

14 Large retailer using BigData Large USA appliance reseller 2600 stores BEFORE BigData Customer Analysis Marketing Campaign Campaign1 # Of 30 10% Data Window Examined Before Campaign2 Campaign 3 AFTER BigData Days Before 3 After 100% Data Window Examined After Benefits - Reports in 3 days instead of 6-12 weeks - Costs 33% of legacy RDBMS systems Benefits - Mobile commerce analyses done daily not weekly - 1 min difference between 100M and 2B records 14 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Source: Information Week

15 Big Data needs a unified approach One platform for structured, semi, and unstructured to profit from 100% of data. Enable you to: Capture Store Manage Analyze Optimize on 100% of data Structured warehouses CRM, transactions, sales, marketing Universal log management IT logs, security logs, social, tweets, JSOn s Unstructured Audio, Video, s, sentiments, threat 15 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

16 HAVEn Big Data platform HAVEn Hadoop/ HDFS Autonomy IDOL Vertica Enterprise Security n Apps Catalog massive volumes of distributed data Process and index all information Analyze at extreme scale in real-time Collect & unify machine data Powering HP Software + your apps 17 Transactional Social media Video Audio Texts Mobile data Documents IT/OT Search engine Images Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. hp.com/haven

18 On-premise + cloud Mobility Today s IT Bring your own device Composite apps = complex IT Point tool proliferation for management Optimized fabric + networking Software-defined datacenter 19 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

19 April $100 million No single cause Perfect storm: Cause of the problem Not immediately clear 20 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

20 21 Problem: How do you defend against the perfect storm? Amazon.com offline 30 to 40 minutes, estimated $3 million in losses 1 Outlook.com users locked out up to 7 hours 1 Google Search, Gmail,YouTube stopped working worldwide 2 American Express out of service for hours 3 32k users incapacitated at South Korea's banks and television networks 3 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential. 1 Evolven.com, 2 Minyanville, 3 - UsaToday

21 Solution: Prepare Gather and retain all the data Store it in a system that understands what it means System Events Business metrics Log files Structured Machine Data LOB Telemetry Unstructured 22 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

22 Solution: Predict Harness the power of Predictive Analytics Uncover trends before they become catastrophes 23 Analytics Extract signal from the noise Spot relationships Machine learning Subtle effects Forecasting Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

23 Solution: Pinpoint Replay history to recreate the events Pinpoint root causes with advanced analytics Haystack Needle On a rocket In a storm 24 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

24 Solution: HP Operations Analytics Big Data Analytics for your IT Operations Prepare Predict Pinpoint Big Data store for IT Connects to all your operations data Stores, indexes and understands Automated Analytics Self-calibrates via machine learning Predicts future behavior Instant replay Recreates history with beautiful visualizations Isolates trouble in time in context 25 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

25 HP Operations Analytics Big Data Analytics for your IT Operations 26 80% safer Reduced severe outage incidents 80% faster To diagnose complex trouble 80% lower cost Than 3 rd party solutions Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential. ROI: we hope you ll never find out

27 HP Business Service Management Ensure optimal IT performance and availability in a dynamic world Business Service Management Operations Bridge Consolidated management, accelerated root cause analysis Application Management End to end composite application performance management Forrester & Gartner Leader Featuring the Run Time Service Model Ops Analytics the next big thing! Operations Intelligence Actionable analytics from events, logs, metrics & topology data Infrastructure Management Comprehensive visibility and monitoring of hybrid IT Unique RTSM engine Patented Analytics: IT s crystal ball Huge Installed Base Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

28 HP Operations Analytics Comprehensive, actionable insight into all aspects of operations for the New Style of IT IT Search Guided Troubleshooting Visual Analytics Standalone, Scalable Operations Analytics platform New standalone solution Performance Metrics Operational Events Topology Logs Application Mobile app Storage Network 3 rd Party tools e.g. Splunk System Analyze end to end all your management data Cloud 29 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

29 HP Operations Analytics Provides Advanced, Big Data Analytics for the New Style of IT IT Search Guided Troubleshooting Visual Analytics 30 Analytics Query Language (AQL) Phrase Query Language (PQL) Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential. Analytics Query Language (AQL)

30 For whom and why? Persona Subject Matter Experts - Application Support Infra/Network Support Problem Manager Service Manager NOC/Ops Bridge Mgr Primary use cases Application & Infra problem isolation using Guided Troubleshooting with application specific content & visualization IT Search Ad hoc use cases for problem isolation Problem analysis against event data, logs and metrics to understand repeat issues, or service delivery improvements Data Analyst Data specialist The ability to create ad hoc queries against metrics, events, topology and logs to gain insight with analytics tools such as trending, volume and custom algorithms 31 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

31 Why HP for HP Operations Analytics? Why care about HP Operations Analytics With HP Operations Analytics Prepare, Predict, Pinpoint 80% Faster - practice analytics otherwise humanly not feasible 80% Safer less downtime, reduce triage time, reduce outages 80% cheaper costs not related to data volume collected Embeds Vertica, state of the art Big Data technologies Embeds Arcsight Logger, leader in Gartner MQ Quadrant HP Operations Analytics Is Enterprise ready and scalable to virtually any store size Protects your investment Exploits existing data from HP data collectors incl EUM Integrates but is independent of HP BSM Integrates with 3 rd Party solutions e.g. Splunk voted Strong Value by EMA As both a platform and a vision, HP Operations Analytics offers a highly modular and adaptive approach to tackling new and even unforeseen problem sets. HP s introduction of Operations Analytics has all the makings of a visionary industry event.. Dennis Drogseth, Vice President Enterprise Management Associates, Inc. 32 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

32 HP Operations Analytics How to get started Explore 1 st Generation operations analytics - Download the freemium integration components - (Optional) Download freemium Arcsight logger and/or OMi - Integrate your OMi or OM or NNMi - Exploit Award winning search to find issues fast Adopt the New Style of IT Management - Ask to enter the Early Release Program for HP Operations Analytics - Install the Virtual appliance 33 - Get all the advantages of BigData Analytics for Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

33 At Vodafone Ireland, we see our successful implementation of HP BSM software for consolidated operations, being extremely well complemented by the potential we see in HP Operations Analytics. The capability to rapidly search through massive amounts of data, events, topology, metrics, machine data and log files, will complement our monitoring solution, and helps us master the dynamics of our Hybrid IT environment Luke Bradley, Service Management Tools Manager, Technology Operations, Vodafone Ireland Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

35 Predict Enterprise IT environments are more diverse, complex and constantly changing No early warning of service degradation Static thresholds are problematic to maintain And sometimes impossible to set No easy way to correlate if current issue is similar to those seen in the past 36 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

36 HP BSM: Service Health Analyzer Foresight of potential issues with behavioural learning and predictive analytics Predictive alerting based on learned pattern detection Unique topology-based performance analytics Automated root-cause and similarity analysis HP patented algorithms / IP SHA Detected an issue 36 minutes before regular monitoring detected it Healthcare Provider SHA detected an issue 1 hour before it was reported by end user PlayTech Gaming Services provider (business metrics) 37 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

37 Product Strength SHA Value Summary Reduce negative business Impact Reduce Labor Costs Get notified earlier Prevent and reduce business impact of problems No need to set and maintain thresholds Triage multiple events at the same time Avoid reinventing the wheel - automatically detect similar issues Low Risk Deployment Leverage BSM investment and RTSM Topology Analyze any time series data Easy to deploy Cost Efficiency 38 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

38 Predictive Analytics with HP SHA How to get started Attend the HP Virtual Events September 10, 2013 Americas October 1, 2013 EMEA Follow the Webinar Get the power of predictive analytics featuring Forrester Research Read the Solution Brief You can anticipate IT issues before they impact your business. Predict and preemptively solve more IT problems with HP Service Health Analyzer Watch the HP Service Health Analyzer overview and demo Learn how customers have gained success 39 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

39 The management cockpit for Big Data platforms such as Hadoop and Vertica As a Hadoop Administrator, I would like to know the availability of the Hadoop Services at any given point of time Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

40 HP Big Data Monitoring Cockpit HP Operations Manager i + Management Packs Management Cockpit Provides Real time Visualization of your end to end big data environment Monitor availability & performance of big data applications & infrastructure components Single version of the truth for all Operations Benefits to Y Centralise management information, faster triage 41 Isolate, diagnose & Remediate problems impacting your complex big data environment Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential. Lowered costs, increaed operational efficiencies

41 Visualize your big data infrastructure from the top Obtain the health & performance summary of specific Big Data technology element Hadoop Dashboard Real time reflection of topology & health Instant information on availability of services e.g., HDFS, MapReduce Notifications on performance & workload Management Cockpit on IOS and Android too 42 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

42 Big Data monitoring use cases Ensure Hadoop jobs are completed on optimal time Monitor available M/R slots, slots used, completed and waiting jobs Alert on high wait time or failure of jobs Visualize map and reduce slot usage trends Understand the impact of failed jobs Remediate: Realign the Hadoop cluster for optimal performance Ensure analytics reports are executed on time on Vertica Monitor the resource queues & rejection, lock and query execution time Alert on lock, execution time, long queue or resource rejection Analyze real time & historical data together, Check for performance trends Remediate: Create resource pool for high priority requests Avoid any data loss on Vertica cluster Monitor and alert on node down, not K-Safe and critical node state Visualize the cluster availably and impact on your service topology Report on capacity planning & forecasting Remediate: Immediately restore any failed /Shutdown node. 43 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

43 Maintain the health of your dynamic infrastructure End to end Automated Integrated visibility for all stakeholders to keep up with demand in no time for seamless support Ensure Performance and high availability for big-data middleware across network, storage, compute resources Fast, dynamic, and costeffective monitoring Monitoring scale-up and scale-out with the big-data platform. Quick and consistent trouble response integrated in existing 24x7 support operations. HP Solution: Big Data Monitoring Cockpit 44 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

44 OMi: Big Data Monitoring Cockpit Out of the box Available today More to come Discovery Data Collection Availability monitoring Performance monitoring Event Correlation Real time graphs Trend Reporting Infrastructure System, OS, Virtual OS, Cluster Hadoop Distributed storage & application processing Vertica Interactive real-time analytics Variety of supported operating systems / virtual operating systems - CPU - Memory - Disk - Network - Resources Monitor 1000s of Hadoop nodes & clusters through a single dashboard - MapReduce - Job Statistics - RW ops Throughput - HDFS availability Maintain the infrastructure analyzing your big data - Query Performance - Resource rejection - Process status - Node status 45 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

45 HP Management Cockpit helps customers Reduce cost, Reduce risk, Increase confidence Increase service availability Vodafone reduced downtime by 77%, increased availability to 99.9% Minimize business impact when issues happen Florida Blue lowered the time to resolve to problems by 80% Reduce cost Boeing reduced their costs by $1M per year with HP BSM 46 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.

46 Call to action Download HP Operations Manager I (OMi) and OMi Management Packs Today! Plus! Download the new OMi White Paper: Intelligent IT Operations. Making IT smarter with advanced event correlation and management HP.COM/GO/OpsAnalytics Free downloadable software White Papers Videos Recorded webinars Customer use cases Industry experts opinions 47 Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Confidential.