Mission-Driven Big Data

Transcription

1 Mission-Driven Big Data Tim Brooks Jamie Milne Principal Engagement Manager Copyright 2014 World Wide Technology, Inc. All rights reserved.

2 Experience Across Big Data Deliverables PUBLIC SECTOR COMMERCIAL ENTERPRISE SERVICE PROVIDER ADVANCED TECHNOLOGY CENTER SUPPLY CHAIN NETWORK COLLABORATION DATA CENTER BIG DATA SECURITY BUSINESS IMPACT Extract value from data to drive multiple Use Cases WWT BIG DATA Vision TECHNOLOGY OPTIMIZATION Accomplish data tasks, faster, cheaper, better

3 IT Budget ($Tn) Data (EB) Expanding Demands, Economic Constraints Increase in appetite for analytics... & data outpacing IT budget User-defined BI Reporting Big Data Initiatives 30% % both require an agile cost-effective Big Data solution

4 data access probability data access probability Mission-Driven Big Data Expectation Data Efficacy 92% of data becomes COLD immediately e.g. Unstructured data (social, text) time Reality Data may be critical for: Real-time decision making Aggregated and holistic views time

5 Data Preparation (ETL) Data Preparation (ELT) Current Data Warehouse Environment Source Systems Operational Systems Data Preparation (e.g. Informatica) Data Warehouse (e.g. Teradata, Netezza) $17K/TB Hot Data Access Layer Reporting & Analytics ERP, CRM Cold Data Third Party Data Unstructured Data Large amounts of unstructured data do not make it into DW due to rigid schema Preparation of data for warehouse discards potentially valuable data Additional preparation runs on DW, increasing storage and decreasing performance Cold data dominates DW storage, rarely accessed by end users

6 Data Preparation Data Warehouse Optimization (DWO) Source Systems Operational Systems Hadoop ~$2K/TB Data Warehouse (e.g. Teradata, Netezza) ~$17K/TB Hot Data Access Layer Reporting & Analytics ERP, CRM All Data 3 Third Party Data Unstructured Data Unstructured data can now be loaded into Hadoop in native format Low-cost Hadoop environment enables retention of all source data for analysis Data warehouse performance increases and storage cost decreases Users can access Hadoop directly for some analytics and reports, further decreasing DW storage and processing requirements

7 DWO Value Proposition Same user experience The end user experience is seamless since existing reports and analytics can remain on data warehouse, or be run on Hadoop using same software tools Vastly improved data ecosystem for supporting analytics Significantly reduced cost Moving cold data, data preparation and batch processing from a data warehouse and into Hadoop reduces DW licensing and hardware storage costs Up to 60% savings on traditional DW solution

8 DWO Software Stack Optional data analytics tools & apps ETL tools Native Hadoop tools for exploratory analytics and reporting Native Hadoop tools for data ingest Choice of any of the major Hadoop distributions RedHat Enterprise Linux OS

9 DWO Engagement Methodology Assess 1. Assess and prioritize or stratify 2. Collect and monitor over time 3. Analyze metadata from ETL tools 4. Include people and workflow 5. Map Data Warehouse cold/hot zones and workflows Pilot and Prove 6. Configure IT stack for Big Data 7. Prioritize ETL/ELT workflow and data 8. Migrate representative sample and prove ROI 9. Optimize UX Scale and Optimize 10. Scale workflow to Hadoop 11. Expand Hadoop and DW to match workflow 12. Document maintenance and support requirements 13. Train end-users and transfer knowledge

10 FlexpodSelect White Box Case Study: Global Info. Measurement Company Customer Big Data Needs Upgrade to enterprise storage solution Sustain needs and performance of current architecture Using WWT Advanced Technology Center VS Comparing Storage Solutions Equal or better performance for enterprise-grade storage over a range of tasks Test capabilities

11