Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012 1
Market Trends Big Data Growing technology deployments are creating an exponential increase in the volume of data available Existing analytical techniques not adequate for business decision-making processes A successful approach of big data analytics will be a critical core competency Delivering significant competitive advantage to organizations Nationwide Electricity Grid Analysis 2
Big Data Not just a Volume Play Facebook statistics 2009 350 Million Named users 175 Million Active users in one day 35 Million Users updating status each day 2.5 Billion Photos / Month 1.6 Million Active pages Growth: 12 TB /day, 2 PB /year Global data volume : 8.7 PB Source: Gartner 3
Market Trends Serial to Parallel Computing Big Data Analytics is Parallel Processing Single Core Multi cores Single Computer Clustered Computing Traditional Data-Processing Pipeline Big Data parallel processing over Hadoop 4
Hadoop Cluster over High Performance Networking Mapper Shuffle IO Intensive Phases Bursty characteristics HDFS Replication Data Split Merge Sort Reducer Out Split Mapper Data Split Merge Sort Reducer Mapper Out Split Data Split 5
Avg Cap (GB) ASP ($), ASP/GB (Cent/GB) Peak Bendwidth (MB/s) I/O Bottlenecks: from Disk to Network Trends Use multiple SATA drives SSD is around the corner Matching network speed 10Gigabit Ethernet and beyond 7000 6000 5000 4000 3000 2000 1000 0 Disk and Network Performance SATA 4 SATA 12 SATA SSD (SAS) 12 SSD (SAS) SSD (PCIe) Quad SSD (PCIe) Bandwidth (MB/s) 40GE 10GE 1GE 500.00 450.00 400.00 350.00 Enterprise Enterprise Server Server SSD SSD Trends 1,400.0 1,200.0 1,000.0 300.00 250.00 200.00 150.00 100.00 50.00 - Source: "SSD Summary", Gartner, May 2011 2009A 2010A 2011 2012 2013 2014 2015 Avg Capacity (GB) ASP ($) ASP/GB (Cent/GB) 800.0 600.0 400.0 200.0-6
Efficient Fabric Services Scalable and non-blocking Matches extensive data exchange at peak rates Non blocking east-west traffic High Capacity Match all I/O bandwidth of the server Losslessness Efficiency and avoidance of retransmission Offload Full transport offload reduces CPU utilization RDMA zero copy operations 7
UDA Plug In Architecture Plug-in architecture Hadoop applications are unmodified Plug-in to Apache Hadoop Enabled via xml configuration Efficient Map Reduce Data communication over RDMA Using RDMA for In-Memory processing Enables to start the Reduce operation in parallel to the Shuffle operation Reduce disk IO operation Supports InfiniBand and Ethernet Zero copy, transport offload, kernel bypass 8
Software Architecture Hadoop (Java) UDA Plugin (C++) TaskTracker MapTask JobTracker TaskTracker ReduceTask Plug-in Benefits: Zcopy datapath Transport offload Improved merge algorithm MOFSupplier NetMerger Data Engine RDMA Server RDMA Client Merging Merging Thread Merging Thread Thread RDMA NIC / HCA 9
New Pipelined Data Flow Map Stage Map Map Map Map Map Map Map Map Map Map Map Map Shuffle Merge Vanilla Algorithm shuffle shuffle merge merge Reduce Reduce shuffle merge Header fetch Shuffle Merge New Algorithm shuffle merge Reduce Header fetch Reduce start Time 10
UDA MapReduce for Hadoop 1.X Shuffle portion executed in-memory Eliminating time consuming HDD read/writes UDA reads Map Output Files (MOF) from mappers Predefined portions of the MOFs, default 1KB Reduce tasks begins only with all mapper tasks completion 11
Execution Time (sec) UDA Enables Higher Performance 1000 Terasort Benchmark* (20GB file size, 16GB data per node, 8 Mappers, 4 reducers, 4 Disks) 900 800 700 600 500 45% Lower is better 1 GE 10 GE 400 UDA 10GE 300 200 ~2X Acceleration 100 0 8 Nodes 10 Nodes 12 Nodes *TeraSort is a popular benchmark used to measure the performance of Hadoop cluster 12
Accelerating Big Data Analytics EMC 1000-Node Analytic Platform Accelerates Industry's Hadoop Development 24 PetaByte of physical storage Half of every written word since inception of mankind Mellanox VPI Solutions Hadoop Acceleration 2X Faster Hadoop Job Run-Time High Throughput, Low Latency, RDMA Critical for ROI 13
Thank You Email: motti@mellanox.com 14