Deploying Big Data to the Cloud: Roadmap for Success

Transcription

1 Deploying Big Data to the Cloud: Roadmap for Success James Kobielus Chair, CSCC Big Data in the Cloud Working Group IBM Big Data Evangelist. IBM Data Magazine, Editor-in- Chief. IBM Senior Program Director, Product Marketing, Big Data Analytics

2 Clouds with big data are a foundation of the smarter planet Cloud Mobile Social Internet of Things

3 Key Takeaways Big data is integral many cloud services applications. Big data platforms ensure scalability, flexibility, and cost-effectiveness for cloud analytics: massively parallel processing, in-database execution, storage optimization, data virtualization, and mixed-workload management Cloud services realize big data objectives: ubiquitous, convenient, on-demand network access to data analytics shared pool of configurable data analytic computing resources dynamic provisioning and release of data analytic resources CSCC has recently published guidance for organizations to realize value from cloud-based big data initiatives: Deploying Big Data Analytics Applications to The Cloud: Roadmap for Success" (

4 Cloud Ensures Scalability for Big Data Application Workloads Big Data = realizing differentiated value from advanced analytics and trustworthy data at cloud scales. Volume Velocity Variety 12 terabytes of Tweets create daily s million trade events per second video feeds from surveillance cameras Analyze sentiment Identify potential fraud Monitor events of interest 350 billion 500 million 80% data meter readings per annum growth call detail records per day are images, video, documents Predict power consumption Prevent churn Improve constituent satisfaction

5 Why is Big Data in the Cloud Important Now? The power of all data coming together to deliver improved outcomes Top 5 Use Cases We ve Observed 1. Enrich your information base with Big Data Exploration 2. Improve customer interaction with Enhanced 360º View of the Customer with the power of cloud services 3. Optimize operations with Operations Analysis 4. Gain IT efficiency and scale with Data Warehouse Augmentation 5. Prevent crime with Security and Intelligence Extension 5

6 Cloud is the Latest Major Wave of Technology Confluence of Social Mobile Cloud Big Data / Analytics Back Office Computing Client Server PC World Wide Web and ebusiness 60 s 80 s 90 s We are here

7 Industry Movement from Traditional Environments to Clouds Many users are already on the way to cloud with consolidation and virtualization efforts CLOUD Dynamic provisioning for workloads VIRTUALIZE Increase Utilization CONSOLIDATE Physical Infrastructure STANDARDIZE Operational Efficiency SHARED RESOURCES Common workload profiles AUTOMATE Flexible delivery & Self Service Traditional IT Leon Katsnelson 7

8 Industry Puts Cloud At the Forefront of Their Business Strategies 1 Factors impacting organizations: 1. Technology factors 2. People skills 3. Market factors 4. Macro-economic factors 5. Regulatory concerns 6. Globalization Source: IBM CEO Study 2012 Speed Value 90% view cloud as critical to their plans Extended Reach 1Billion Smartphones and 1.2 billion mobile employees by 2014 Responsiveness 20B+ Intelligent business assets New Insights 2.7ZB of digital content in 2012, up 50% from

9 What Industries Are Doing with Big Data in the Cloud Financial Services Fraud detection 360 View of the Customer Utilities Weather analysis Smart grid management Transportation Logistics optimization Traffic congestion Health & Life Sciences Epidemic early warning ICU monitoring 9 Telecommunications Geomapping / marketing Network monitoring Multiple Industries Customer Retention Customer Acquisition Manufacturing Manufacturing Efficiency IT System Log Analysis Cybersecurity Outage prevention Resource Prediction Warehouse Integration Retail Marketing Campaign Efficiency Targeted Marketing MicroSegmentation Law Enforcement Multimodal surveillance Cyber security detection

10 Cloud-based Big Data Spans Systems of Record & Engagement Established Approach Structured, analytical, logical Systems of Record Transaction Data Internal App Data Mainframe Data OLTP System Data ERP Data Data Warehouse Structured Repeatable Linear Hadoop and Streams Unstructured Exploratory Dynamic Emerging Approach Creative, holistic thought, intuition Systems of Engagement Multimedia Web Logs Social Data Text Data: s Sensor data: images RFID Traditional Sources New Sources

11 How to get there? Step 1: Build your business case for big data in the cloud

12 How to get there? Step 2: Assess which big-data functions are best deployed in cloud Enterprise apps already hosted in cloud? High-volume data requiring extensive preprocessing? Tactical apps beyond capabilities of legacy platforms? Elastic provisioning of very data-intensive but shortlived analytic and data management apps?

13 How to get there? Step 3: Develop your cloud big-data technical approach Public vs. private vs. hybrid vs. MPP RDBMS vs. Hadoop vs. NoSQL vs Consolidated vs. multitier vs. federated vs Homogeneous vs. hybrid

14 How to get there? Step 4: Maintain tight controls over your big data in the cloud Governance Security Privacy Risk Accountability Compliance

15 How to get there? Step 5: Deploy, integrate, & operationalize your bigdata cloud Converge Big Data cloud operational siloes Administer Big Data cloud through consolidated system management tools Provide Big Data cloud users with a single throat to choke on Big Data cloud support Automate Big Data cloud support functions to maximum extent feasible Deliver consulting support to users considering implementing new Big Data cloud initiatives Don t do big data in the cloud unless you can make it production-ready from Day One!

16