Big Data System and Architecture

Transcription

1 CHANGE, a 2012 DAC workshop 2nd International Workshop on Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments Moscone Center, San Francisco, California, June 3, 2012 Big Data System and Architecture Jian Li IBM Research in Austin jianli@us.ibm.com 2011 IBM Corporation

2 Agenda Big Data requirements Industry view Big Data application scenarios Case study: Watson Big Data platform Case study: A real world deployment Big Data research items Case studies: Performance 2

3 IBM Disclaimer Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. More info at:

4 Big Data Big/Deep Insights to enterprise and society 44x as much Data and Content Over Coming Decade zettabytes 1 in 3 don t have Business leaders frequently make decisions based on information they don t trust, or Kilobyte (kb) 1,000 Bytes Megabyte (MB) Gigabyte (GB) 1,000 Kilobytes 1,000 Megabytes Business leaders say they don t have access to the information 1 in 2 they need to do their jobs Terabyte (TB) 1,000 Gigabytes Petabyte (PB) 1,000 Terabytes Exabyte (EB) Zettabyte (ZB) 1,000 Petabytes 1,000 Exabytes ,000 petabytes 80% Of world s data is unstructured 83% of CIOs cited Business intelligence and analytics as part of their visionary plans to enhance competitiveness of CEOs need to do a better job capturing and understanding information rapidly in order to 60% make swift business decisions The resulting explosion of information (plus intermediate data) creates a need for a new kind of intelligence

5 Big Data Presents Big Opportunities Extract insight from a high volume, variety, velocity and veracity of data in a timely and cost-effective manner Veracity Variety: Velocity: Volume: Veracity: Manage and benefit from diverse data types and data structures Analyze streaming data (data in motion) and large volumes of persistent data (data at rest) Scale from terabytes to zettabytes Establish confidence in data, information and solutions, e.g. Watson 5

6 Categories of Analytics Degree of Complexity / Competitive Advantage Stochastic Optimization Optimization Predictive modeling Forecasting Simulation Alerts Query/drill down Ad hoc reporting Standard Reporting What will happen next if? What if these trends continue? What could happen.? What actions are needed? What exactly is the problem? How many, how often, where? What happened? (Self?) learning system, expert system e.g. Watson How can we achieve the best outcome including the effects of variability? How can we achieve the best outcome? Prescriptive (E.g., ILOG) Predictive (E.g., SPSS, WebSphere Business Modeler) Descriptive (E.g., Cognos) Based on: TLE 2010 in CA. 6

7 Case Study: IBM Watson IBM Watson is a breakthrough in analytic innovation, but it is only successful because of the quality of the information from which it is working.

8 Big Data and Watson Big Data technology is used to build Watsonʼs knowledge base! Watson technology offers great potential for advanced business analytics! Watson used the Apache Hadoop open framework to distribute the workload for loading information into memory." Approx. 200M pages of text (To compete on Jeopardy!) POS Data CRM Data InfoSphere BigInsights Social Media Distilled Insight - Spending habits - Social relationships - Buying trends 10 racks of P750s, 2870 processor cores Watson s Memory (15TB) Advanced search and analysis

9 IBM Big Data Platform Vision IBM Big Data Solutions Big Data Operators and Accelerators Client and Partner Solutions Text Statistics Financial Geospatial Acoustic Image/Video Mining Times Series Mathematical Connectors Graph Analysis Accelerators Big Data Enterprise Engines InfoSphere Streams InfoSphere BigInsights Productivity Tools and Optimization Workload Management and Optimization Consumability and Management Tools Open Source Foundation Components Eclipse Oozie Hadoop HBase Pig Lucene Jaql Linux POWER (GA June 2012) x86 9

10 BigInsights: Analytics for Data at Rest BigInsights Enterprise Edition Adaptive MapReduce SystemML Unstructured Analytics (SystemT) Metatracker GPFS SNC BigInsights Core Install & Configuration Monitoring Jaql Management console Streams, DB and Warehouse integration Pig,Hive,Flume,Sqoop, etc Applications & Solutions Enabling Infrastructure BigSheets (included in BigInsights) Applications / Solutions / Partners / Community Cognos Consumer Insights SPSS and R Next Generation Credit Risk Analytics Custom applications IBM Distribution of Apache Hadoop Passed IBM legal and IP review, safe to use 10

11 Streams: Analytics for Data in Motion Real time delivery Volume Variety Terabytes per second Petabytes per day All kinds of data All kinds of analytics ICU Monitoring Algo Trading Powerful Analytics Cyber Security Government / Law enforcement Environment Monitoring Smart Grid Telco churn predict Velocity Insights in microseconds Millions of events per second Microsecond Latency Agility Dynamically responsive Rapid application development Traditional / Non-traditional data sources 11

12 Streams and BigInsights Hybrid and Integrated Analytics on Data in Motion & Data at Rest Visualization of realtime and historical insights Data Data ingest, preparation, online analysis, model validation InfoSphere Streams 1. Data Ingest 2. Bootstrap/Enrich Control flow 3. Adaptive Analytics Model InfoSphere BigInsights, Database & Warehouse Data Integration, data mining, machine learning, statistical modeling

13 Case Study: A Real World Hybrid Big Data Deployment RDMS Client/Internet Presentation/Web Application/Object Message Queues 12 x 10Gbe network Message Center Sample Services: Sales & Marketing Recommendation Systems Credit management and Fraud Detection Offline Analytics Hadoop + Mahout + Hive + Pig + DFS 500+ nodes Online Analytics HBase + Solr (Lucene) + Yahoo! S4 + DFS 80+ nodes

14 Example Research Issues Data generation, benchmarks, metrics and workload characterization for analytics and dataintensive computing, e.g. how accurate/fast/efficient is necessary and tradeoffs Accelerators in heterogeneous and hybrid systems for analytics and data-intensive computing Scalable system and network designs for capturing large numbers of concurrent data streams or high bandwidth data streaming Data management for vast amounts of unstructured data OS, distributed systems and system management support for very large-scale analytics Debugging and performance analysis tools for analytics and data-intensive computing Programming systems and language support for deep analytics Mapreduce and other processing paradigms for analytics Processor, memory and system architectures for data analytics Implications of data analytics to cloud computing Implications of data analytics to mobile, embedded and autonomous systems Energy-efficiency and energy-efficient designs for analytics Availability, fault tolerance and data recovery in large-scale data-oriented environments Self learning and cognitive system? 14

15 Performance Case Study: Sort 1 TB of Data on PowerLinux Lab Results Initial results demonstrate 1 Terabyte of data sorted in 14 minutes on 10 nodes running PowerLinux (April 2012) Fastest industry results demonstrated to-date is 24 minutes one result used 10 x86 Linux nodes and another used 11 x86 Linux nodes* Both with newer/faster Hadoop versions and patches Lab continuing tuning to fully exploit PowerLinux benefits Continuing hardware-software co-optimization and architecture innovation of IBM Big Data systems Test Hardware 10 Power 730 express servers 120 cores (2-socket, 6-cores per socket, 3.72 GHz) 640GB DRAM (64GB each) 144 TB SAS drives (24x600GB DAS each) Nodes inter-connected with 10GbE switch Test Software Early code (pre-ga) - BigInsights v1.3 on PowerLinux PowerLinux - SLES11sp1 15 *MapR used 1 master node and 10 slave nodes for its Terasort results (@Hadoop Summit 2011). Cloudera used 10 nodes (@Hadoop World 2011).

16 13 minutes 48 seconds Terasort by exploring PowerLinux 16 Results as of April 2012; further Integrated Optimization underway

17 Technology Example: GPFS-SNC for better Performance, Availability, Integrity and Manageability Query languages like Pig and JAQL need good random I/O performance Sort requires better sequential throughput GPFS is twice HDFS for both of the above GPFS-SNC Key technology Locality awareness Write Affinity Metablocks Pipelined replication Distributed recovery For document index lookups, client side caching is a big win 17x throughput speedup Hadoop Indexing (HDFS) Copy Web Service Layer Database Upload (ext3) Fetch HDFS: Extra copy overhead and network fetch, separate clusters for analytics and database Workload Isolation Hadoop Indexing + Database Upload (GPFS) Cache Web Service Layer GPFS: Single cluster for analytics and database, no copying required, caching for web layer Proven data integrity Replicated metadata services Yahoo keeps 3 copies of 3 versions of HDFS because of unknown data integrity [1] Quantcast deletes files once HDFS is 50% full [2] [1] Care and Feeding of Hadoop Clusters, Marc Nicosia, Usenix 2009 [2] The Komos Distributed File System, Sriram Rao, Quantcast Inc. 17

18 Technology Example: OpenCL OpenCL (Open Computing Language) is an open standard for cross-platform, parallel programming of modern processors found in personal computers, servers and handheld/ embedded devices. Highly flexible: supports computation on CPUs, GPUs, accelerators (SIMD, FPGAs, DSPs) Research contributions 2.8 X acceleration factor for the sparse coding phase (considering the best timings for each implementation) for NMF 2.0 X overall algorithm improvement factor, including the preprocessing costs of NMF Java Cluster CPU Usage JOCL Cluster CPU Usage 18

19 Technology Example: Reconfigurable FPGA Acceleration FPGA FPGA CPU CPU N E T W O R K Host Server (POWER) Bandwidth reduction (& capacity increase) Through (De)Compression World s fastest gzip (Research Contributions) Bandwidth reduction Through Filtering Big Data: Dictionary & Regexp based filtering Net result: Significant increase in capacity and throughput in place (computing where data is at)

20 Recap: Example Research Issues Data generation, benchmarks, metrics and workload characterization for analytics and dataintensive computing, e.g. how accurate/fast/efficient is necessary and tradeoffs Accelerators in heterogeneous and hybrid systems for analytics and data-intensive computing Scalable system and network designs for capturing large numbers of concurrent data streams or high bandwidth data streaming Data management for vast amounts of unstructured data OS, distributed systems and system management support for very large-scale analytics Debugging and performance analysis tools for analytics and data-intensive computing Programming systems and language support for deep analytics Mapreduce and other processing paradigms for analytics Processor, memory and system architectures for data analytics Implications of data analytics to cloud computing Implications of data analytics to mobile, embedded and autonomous systems Energy-efficiency and energy-efficient designs for analytics Availability, fault tolerance and data recovery in large-scale data-oriented environments Self learning and cognitive system? 20

21 Conclusions Big Data, Big Data, Big Data!! Open to research collaboration At your convenience, welcome to stop by 2 nd ASBD (Architecture and Systems for Big Data) workshop at ISCA next Saturday, 06/09 in Portland!

22 CHANGE, a 2012 DAC workshop 2nd International Workshop on Computing in Heterogeneous, Autonomous 'N' Goal-oriented Environments Moscone Center, San Francisco, California, June 3, 2012 Big Data System and Architecture Jian Li IBM Research in Austin jianli@us.ibm.com 2011 IBM Corporation