Accelera'ng Big Data Processing with Hadoop, Spark and Memcached

Size: px
Start display at page:

Download "Accelera'ng Big Data Processing with Hadoop, Spark and Memcached"

Transcription

1 Accelera'ng Big Data Processing with Hadoop, Spark and Memcached Talk at HPC Advisory Council Switzerland Conference (Mar 15) by Dhabaleswar K. (DK) Panda The Ohio State University E- mail: state.edu h<p:// state.edu/~panda

2 Introduc'on to Big Data Applica'ons and Analy'cs Big Data has become the one of the most important elements of business analyfcs Provides groundbreaking opportunifes for enterprise informafon management and decision making The amount of data is exploding; companies are capturing and digifzing more informafon than ever The rate of informafon growth appears to be exceeding Moore s Law Commonly accepted 3V s of Big Data Volume, Velocity, Variety Michael Stonebraker: Big Data Means at Least Three Different Things, hsp:// stonebraker.pdf 5V s of Big Data 3V + Value, Veracity HPCAC Switzerland Conference (Mar '15) 2

3 Data Management and Processing on Modern Clusters SubstanFal impact on designing and uflizing modern data management and processing systems in mulfple Fers Front- end data accessing and serving (Online) Memcached + DB (e.g. MySQL), HBase Back- end data analyfcs (Offline) HDFS, MapReduce, Spark Front-end Tier Back-end Tier Internet Data Accessing and Serving Web Server Web Server Web Server Memcached + DB Memcached (MySQL) + DB Memcached (MySQL) + DB (MySQL) NoSQL DB (HBase) NoSQL DB (HBase) NoSQL DB (HBase) Data Analytics Apps/Jobs MapReduce Spark HDFS 3

4 Overview of Apache Hadoop Architecture Open- source implementafon of Google MapReduce, GFS, and BigTable for Big Data AnalyFcs Hadoop Common UFliFes (RPC, etc.), HDFS, MapReduce, YARN h<p://hadoop.apache.org Hadoop 1.x Hadoop 2.x MapReduce (Data Processing) Other Models (Data Processing) MapReduce (Cluster Resource Management & Data Processing) YARN (Cluster Resource Management & Job Scheduling) Hadoop Distributed File System (HDFS) Hadoop Distributed File System (HDFS) Hadoop Common/Core (RPC,..) Hadoop Common/Core (RPC,..) 4

5 Spark Architecture Overview An in- memory data- processing framework Worker IteraFve machine learning jobs InteracFve data analyfcs Scala based ImplementaFon Zookeeper Worker Standalone, YARN, Mesos Scalable and communicafon intensive Driver Worker HDFS Wide dependencies between Resilient Distributed Datasets (RDDs) SparkContext Master Worker Worker MapReduce- like shuffle operafons to reparffon RDDs Sockets based communicafon hsp://spark.apache.org 5

6 Memcached Architecture Main memory CPUs Main memory CPUs SSD HDD... SSD HDD Internet!"#$%"$#& High Performance Networks High Performance Networks Main memory SSD CPUs HDD High Performance Networks Main memory SSD CPUs HDD Main memory CPUs (Database Servers) SSD HDD Web Frontend Servers (Memcached Clients) (Memcached Servers) Three- layer architecture of Web 2.0 Web Servers, Memcached Servers, Database Servers Distributed Caching Layer Allows to aggregate spare memory from mulfple nodes General purpose Typically used to cache database queries, results of API calls Scalable model, but typical usage very network intensive 6

7 Presenta'on Outline Challenges for AcceleraFng Big Data Processing The High- Performance Big Data (HiBD) Project RDMA- based designs for Apache Hadoop and Spark Case studies with HDFS, MapReduce, and Spark Sample Performance Numbers for RDMA- Hadoop 2.x Release RDMA- based designs for Memcached and HBase RDMA- based Memcached with SSD- assisted Hybrid Memory RDMA- based HBase Challenges in Designing Benchmarks for Big Data Processing OSU HiBD Benchmarks Conclusion and Q&A 7

8 Drivers for Modern HPC Clusters High End CompuFng (HEC) is growing dramafcally High Performance CompuFng Big Data CompuFng Technology Advancement MulF- core/many- core technologies and accelerators Remote Direct Memory Access (RDMA)- enabled networking (InfiniBand and RoCE) Solid State Drives (SSDs) and Non- VolaFle Random- Access Memory (NVRAM) Accelerators (NVIDIA GPGPUs and Intel Xeon Phi) Tianhe 2 Titan Stampede Tianhe 1A 8

9 Interconnects and Protocols in the Open Fabrics Stack Applica'on / Middleware Interface Applica'on / Middleware Sockets Verbs Protocol Kernel Space TCP/IP TCP/IP RSockets SDP TCP/IP RDMA RDMA Ethernet Driver IPoIB Hardware Offload User Space RDMA User Space User Space User Space Adapter Ethernet Adapter InfiniBand Adapter Ethernet Adapter InfiniBand Adapter InfiniBand Adapter iwarp Adapter RoCE Adapter InfiniBand Adapter Switch Ethernet Switch InfiniBand Switch Ethernet Switch InfiniBand Switch InfiniBand Switch Ethernet Switch Ethernet Switch InfiniBand Switch 1/10/40 GigE IPoIB 10/40 GigE- TOE RSockets SDP iwarp RoCE IB Na've 9

10 Wide Adop'on of RDMA Technology Message Passing Interface (MPI) for HPC Parallel File Systems Lustre GPFS Delivering excellent performance: < 1.0 microsec latency 100 Gbps bandwidth 5-10% CPU uflizafon Delivering excellent scalability 10

11 Challenges in Designing Communica'on and I/O Libraries for Big Data Systems Applica'ons Benchmarks Big Data Middleware (HDFS, MapReduce, HBase, Spark and Memcached) Upper level Changes? Programming Models (Sockets) Other RDMA Protocols? Point- to- Point Communica'on Communica'on and I/O Library Threaded Models and Synchroniza'on Virtualiza'on I/O and File Systems QoS Fault- Tolerance Networking Technologies (InfiniBand, 1/10/40GigE and Intelligent NICs) Commodity Compu'ng System Architectures (Mul'- and Many- core architectures and accelerators) Storage Technologies (HDD and SSD) 11

12 Can Big Data Processing Systems be Designed with High- Performance Networks and Protocols? Current Design Applica'on Our Approach Applica'on Sockets OSU Design Verbs Interface 1/10 GigE Network 10 GigE or InfiniBand Sockets not designed for high- performance Stream semanfcs onen mismatch for upper layers Zero- copy not available for non- blocking sockets 12

13 Presenta'on Outline Challenges for AcceleraFng Big Data Processing The High- Performance Big Data (HiBD) Project RDMA- based designs for Apache Hadoop and Spark Case studies with HDFS, MapReduce, and Spark Sample Performance Numbers for RDMA- Hadoop 2.x Release RDMA- based designs for Memcached and HBase RDMA- based Memcached with SSD- assisted Hybrid Memory RDMA- based HBase Challenges in Designing Benchmarks for Big Data Processing OSU HiBD Benchmarks Conclusion and Q&A 13

14 Overview of the HiBD Project and Releases RDMA for Apache Hadoop 2.x (RDMA- Hadoop- 2.x) RDMA for Apache Hadoop 1.x (RDMA- Hadoop) RDMA for Memcached (RDMA- Memcached) OSU HiBD- Benchmarks (OHB) hsp://hibd.cse.ohio- state.edu Users Base: 95 organizafons from 18 countries More than 2,900 downloads RDMA for Apache HBase and Spark will be available in near future 14

15 RDMA for Apache Hadoop 2.x Distribu'on High- Performance Design of Hadoop over RDMA- enabled Interconnects High performance design with nafve InfiniBand and RoCE support at the verbs- level for HDFS, MapReduce, and RPC components Enhanced HDFS with in- memory and heterogeneous storage High performance design of MapReduce over Lustre Easily configurable for different running modes (HHH, HHH- M, HHH- L, and MapReduce over Lustre) and different protocols (nafve InfiniBand, RoCE, and IPoIB) Current release: Based on Apache Hadoop Compliant with Apache Hadoop APIs and applicafons Tested with Mellanox InfiniBand adapters (DDR, QDR and FDR) RoCE support with Mellanox adapters Various mulf- core plarorms Different file systems with disks and SSDs and Lustre hsp://hibd.cse.ohio- state.edu 15

16 RDMA for Memcached Distribu'on High- Performance Design of Memcached over RDMA- enabled Interconnects High performance design with nafve InfiniBand and RoCE support at the verbs- level for Memcached and libmemcached components Easily configurable for nafve InfiniBand, RoCE and the tradifonal sockets- based support (Ethernet and InfiniBand with IPoIB) High performance design of SSD- Assisted Hybrid Memory Current release: Based on Memcached and libmemcached Compliant with libmemcached APIs and applicafons Tested with Mellanox InfiniBand adapters (DDR, QDR and FDR) RoCE support with Mellanox adapters Various mulf- core plarorms SSD hsp://hibd.cse.ohio- state.edu 16

17 OSU HiBD Micro- Benchmark (OHB) Suite - Memcached Released in OHB (ohb_memlat) Evaluates the performance of stand- alone Memcached Three different micro- benchmarks SET Micro- benchmark: Micro- benchmark for memcached set operafons GET Micro- benchmark: Micro- benchmark for memcached get operafons MIX Micro- benchmark: Micro- benchmark for a mix of memcached set/get operafons (Read:Write rafo is 90:10) Calculates average latency of Memcached operafons Can measure throughput in TransacFons Per Second 17

18 Presenta'on Outline Challenges for AcceleraFng Big Data Processing The High- Performance Big Data (HiBD) Project RDMA- based designs for Apache Hadoop and Spark Case studies with HDFS, MapReduce, and Spark Sample Performance Numbers for Hadoop 2.x Release RDMA- based designs for Memcached and HBase RDMA- based Memcached with SSD- assisted Hybrid Memory RDMA- based HBase Challenges in Designing Benchmarks for Big Data Processing OSU HiBD Benchmarks Conclusion and Q&A 18

19 Accelera'on Case Studies and In- Depth Performance Evalua'on RDMA- based Designs and Performance EvaluaFon HDFS MapReduce Spark 19

20 Design Overview of HDFS with RDMA Others Java Socket Interface 1/10 GigE, IPoIB Network Applica'ons HDFS Write Java Na've Interface (JNI) OSU Design Verbs RDMA Capable Networks (IB, 10GE/ iwarp, RoCE..) Design Features RDMA- based HDFS write RDMA- based HDFS replicafon Parallel replicafon support On- demand connecfon setup InfiniBand/RoCE support Enables high performance RDMA communicafon, while supporfng tradifonal socket interface JNI Layer bridges Java based HDFS with communicafon library wri<en in nafve code 20

21 Communica'on Times in HDFS Communica'on Time (s) GigE IPoIB (QDR) OSU- IB (QDR) Reduced by 30% 0 Cluster with HDD DataNodes 30% improvement in communicafon Fme over IPoIB (QDR) 56% improvement in communicafon Fme over 10GigE Similar improvements are obtained for SSD DataNodes 2GB 4GB 6GB 8GB 10GB File Size (GB) N. S. Islam, M. W. Rahman, J. Jose, R. Rajachandrasekar, H. Wang, H. Subramoni, C. Murthy and D. K. Panda, High Performance RDMA- Based Design of HDFS over InfiniBand, Supercompu'ng (SC), Nov 2012 N. Islam, X. Lu, W. Rahman, and D. K. Panda, SOR- HDFS: A SEDA- based Approach to Maximize Overlapping in RDMA- Enhanced HDFS, HPDC '14, June

22 Enhanced HDFS with In- memory and Heterogeneous Storage Design Features Hybrid Replica'on Applica'ons Triple- H Data Placement Policies Evic'on/ Promo'on Heterogeneous Storage RAM Disk SSD HDD Lustre Three modes Default (HHH) In- Memory (HHH- M) Lustre- Integrated (HHH- L) Policies to efficiently uflize the heterogeneous storage devices RAM, SSD, HDD, Lustre EvicFon/PromoFon based on data usage pa<ern Hybrid ReplicaFon Lustre- Integrated mode: Lustre- based fault- tolerance N. Islam, X. Lu, M. W. Rahman, D. Shankar, and D. K. Panda, Triple- H: A Hybrid Approach to Accelerate HDFS on HPC Clusters with Heterogeneous Storage Architecture, CCGrid 15, May

23 Enhanced HDFS with In- memory and Heterogeneous Storage Three Modes HHH (default): Heterogeneous storage devices with hybrid replicafon schemes I/O operafons over RAM disk, SSD, and HDD Hybrid replicafon (in- memory and persistent storage) Be<er fault- tolerance as well as performance HHH- M: High- performance in- memory I/O operafons Memory replicafon (in- memory only with lazy persistence) As much performance benefit as possible HHH- L: Lustre integrated Take advantage of the Lustre available in HPC clusters Lustre- based fault- tolerance (No HDFS replicafon) Reduced local storage space usage 23

24 Performance Improvement on TACC Stampede (HHH) Total Throughput (MBps) Increased by 2x IPoIB (FDR) OSU- IB (FDR) Increased by 7x Execu'on Time (s) IPoIB (FDR) OSU- IB (FDR) Reduced by 3x 0 write TestDFSIO For 160GB TestDFSIO in 32 nodes For 120GB RandomWriter in 32 Write Throughput: 7x improvement nodes over IPoIB (FDR) read Read Throughput: 2x improvement over IPoIB (FDR) 0 8:30 16:60 32:120 Cluster Size:Data Size RandomWriter 3x improvement over IPoIB (QDR) 24

25 Performance Improvement on SDSC Gordon (HHH- L) Execu'on Time (s) HDFS- IPoIB (QDR) Lustre- IPoIB (QDR) OSU- IB (QDR) Data Size (GB) Sort For 60GB Sort in 8 nodes Reduced by 54% 24% improvement over default HDFS 54% improvement over Lustre 33% storage space saving compared to default HDFS Storage Used (GB) HDFS- IPoIB (QDR) Lustre- IPoIB (QDR) OSU- IB (QDR) 240 Storage space for 60GB Sort 25

26 Evalua'on with PUMA and CloudBurst (HHH- L/HHH) Execu'on Time (s) Reduced by 17% HDFS- IPoIB (QDR) Lustre- IPoIB (QDR) OSU- IB (QDR) Reduced by 29.5% HDFS- IPoIB (FDR) OSU- IB (FDR) s 48.3 s 0 SequenceCount PUMA PUMA on OSU RI Grep SequenceCount with HHH- L: 17% benefit over Lustre, 8% over HDFS Grep with HHH: 29.5% benefit over Lustre, 13.2% over HDFS CloudBurst CloudBurst on TACC Stampede With HHH: 19% improvement over HDFS 26

27 Accelera'on Case Studies and In- Depth Performance Evalua'on RDMA- based Designs and Performance EvaluaFon HDFS MapReduce Spark 27

28 Design Overview of MapReduce with RDMA MapReduce Java Socket Interface Job Tracker 1/10 GigE, IPoIB Network Applica'ons Task Tracker Map Reduce Java Na've Interface (JNI) OSU Design Verbs RDMA Capable Networks (IB, 10GE/ iwarp, RoCE..) Design Features RDMA- based shuffle Prefetching and caching map output Efficient Shuffle Algorithms In- memory merge On- demand Shuffle Adjustment Advanced overlapping map, shuffle, and merge shuffle, merge, and reduce On- demand connecfon setup InfiniBand/RoCE support Enables high performance RDMA communicafon, while supporfng tradifonal socket interface JNI Layer bridges Java based MapReduce with communicafon library wri<en in nafve code 28

29 Advanced Overlapping among different phases Default Architecture A hybrid approach to achieve maximum possible overlapping in MapReduce across all phases compared to other approaches Efficient Shuffle Algorithms Dynamic and Efficient Switching On- demand Shuffle Adjustment Enhanced Overlapping Advanced Overlapping M. W. Rahman, X. Lu, N. S. Islam, and D. K. Panda, HOMR: A Hybrid Approach to Exploit Maximum Overlapping in MapReduce over High Performance Interconnects, ICS, June

30 Performance Evalua'on of Sort and TeraSort IPoIB (QDR) UDA- IB (QDR) OSU- IB (QDR) Reduced by 40% 700 IPoIB (FDR) UDA- IB (FDR) OSU- IB (FDR) Job Execu'on Time (sec) Job Execu'on Time (sec) Reduced by 38% 0 Data Size: 60 GB Data Size: 120 GB Data Size: 240 GB 0 Data Size: 80 GB Data Size: 160 GB Data Size: 320 GB Cluster Size: 16 Cluster Size: 32 Cluster Size: 64 Cluster Size: 16 Cluster Size: 32 Cluster Size: 64 Sort in OSU Cluster For 240GB Sort in 64 nodes (512 cores) 40% improvement over IPoIB (QDR) with HDD used for HDFS TeraSort in TACC Stampede For 320GB TeraSort in 64 nodes (1K cores) 38% improvement over IPoIB (FDR) with HDD used for HDFS 30

31 Evalua'ons using PUMA Workload Normalized Execu'on Time GigE IPoIB (QDR) OSU- IB (QDR) Reduced by 50% Reduced by 49% AdjList (30GB) SelfJoin (80GB) SeqCount (30GB) WordCount (30GB) InvertIndex (30GB) Benchmarks 50% improvement in Self Join over IPoIB (QDR) for 80 GB data size 49% improvement in Sequence Count over IPoIB (QDR) for 30 GB data size 31

32 Op'mize Hadoop YARN MapReduce over Parallel File Systems Compute Nodes App Master Map Reduce Lustre Client MetaData Servers Object Storage Servers Lustre Setup HPC Cluster Deployment Hybrid topological solufon of Beowulf architecture with separate I/O nodes Lean compute nodes with light OS; more memory space; small local storage Sub- cluster of dedicated I/O nodes with parallel file systems, such as Lustre MapReduce over Lustre Local disk is used as the intermediate data directory Lustre is used as the intermediate data directory 32

33 Design Overview of Shuffle Strategies for MapReduce over Lustre Design Features Map 1 Map 2 Map 3 Intermediate Data Directory Reduce 1 In- memory merge/sort Lustre Lustre Read / RDMA reduce Reduce 2 In- memory merge/sort reduce Two shuffle approaches Lustre read based shuffle RDMA based shuffle Hybrid shuffle algorithm to take benefit from both shuffle approaches Dynamically adapts to the be<er shuffle approach for each shuffle request based on profiling values for each Lustre read operafon In- memory merge and overlapping of different phases are kept similar to RDMA- enhanced MapReduce design M. W. Rahman, X. Lu, N. S. Islam, R. Rajachandrasekar, and D. K. Panda, High Performance Design of YARN MapReduce on Modern HPC Clusters with Lustre and RDMA, IPDPS, May

34 Job Execu'on Time (sec) Performance Improvement of MapReduce over Lustre on TACC- Stampede Local disk is used as the intermediate data directory IPoIB (FDR) OSU- IB (FDR) Reduced by 44% Job Execu'on Time (sec) IPoIB (FDR) OSU- IB (FDR) Reduced by 48% Data Size (GB) 0 20 GB 40 GB 80 GB 160 GB 320 GB 640 GB Cluster: 4 Cluster: 8 Cluster: 16 Cluster: 32 Cluster: 64 Cluster: 128 For 500GB Sort in 64 nodes For 640GB Sort in 128 nodes 44% improvement over IPoIB (FDR) 48% improvement over IPoIB (FDR) M. W. Rahman, X. Lu, N. S. Islam, R. Rajachandrasekar, and D. K. Panda, MapReduce over Lustre: Can RDMA- based Approach Benefit?, Euro- Par, August

35 Case Study - Performance Improvement of MapReduce over Lustre on SDSC- Gordon Lustre is used as the intermediate data directory Job Execu'on Time (sec) IPoIB (QDR) OSU- Lustre- Read (QDR) OSU- RDMA- IB (QDR) OSU- Hybrid- IB (QDR) Reduced by 34% Job Execu'on Time (sec) IPoIB (QDR) OSU- Lustre- Read (QDR) OSU- RDMA- IB (QDR) OSU- Hybrid- IB (QDR) Reduced by 25% Data Size (GB) Data Size (GB) For 80GB Sort in 8 nodes For 120GB TeraSort in 16 nodes 34% improvement over IPoIB (QDR) 25% improvement over IPoIB (QDR) 35

36 Accelera'on Case Studies and In- Depth Performance Evalua'on RDMA- based Designs and Performance EvaluaFon HDFS MapReduce Spark 36

37 Design Overview of Spark with RDMA Task Java NIO Shuffle Server (default) BlockManager Netty Shuffle Server (optional) Java Socket Task RDMA Shuffle Server (plug-in) 1/10 Gig Ethernet/IPoIB (QDR/FDR) Network Spark Applications (Scala/Java/Python) Spark (Scala/Java) Task Task BlockFetcherIterator Java NIO Shuffle Fetcher (default) Netty Shuffle Fetcher (optional) RDMA Shuffle Fetcher (plug-in) RDMA-based Shuffle Engine (Java/JNI) Native InfiniBand (QDR/FDR) Design Features RDMA based shuffle SEDA- based plugins Dynamic connecfon management and sharing Non- blocking and out- of- order data transfer Off- JVM- heap buffer management InfiniBand/RoCE support Enables high performance RDMA communicafon, while supporfng tradifonal socket interface JNI Layer bridges Scala based Spark with communicafon library wri<en in nafve code X. Lu, M. W. Rahman, N. Islam, D. Shankar, and D. K. Panda, Accelera'ng Spark with RDMA for Big Data Processing: Early Experiences, Int'l Symposium on High Performance Interconnects (HotI'14), August

38 Preliminary Results of Spark- RDMA Design - GroupBy GroupBy Time (sec) Reduced by 18% GroupBy Time (sec) GigE IPoIB RDMA Reduced by 20% Data Size (GB) Data Size (GB) Cluster with 4 HDD Nodes, GroupBy with 32 cores Cluster with 8 HDD Nodes, GroupBy with 64 cores Cluster with 4 HDD Nodes, single disk per node, 32 concurrent tasks 18% improvement over IPoIB (QDR) for 10GB data size Cluster with 8 HDD Nodes, single disk per node, 64 concurrent tasks 20% improvement over IPoIB (QDR) for 20GB data size 38

39 Presenta'on Outline Challenges for AcceleraFng Big Data Processing The High- Performance Big Data (HiBD) Project RDMA- based designs for Apache Hadoop and Spark Case studies with HDFS, MapReduce, and Spark Sample Performance Numbers for Hadoop 2.x Release RDMA- based designs for Memcached and HBase RDMA- based Memcached with SSD- assisted Hybrid Memory RDMA- based HBase Challenges in Designing Benchmarks for Big Data Processing OSU HiBD Benchmarks Conclusion and Q&A 39

40 Performance Benefits TestDFSIO in TACC Stampede (HHH) Total Throughput (MBps) Increased by 7.8x IPoIB (FDR) OSU- IB (FDR) Execu'on Time (s) IPoIB (FDR) OSU- IB (FDR) Reduced by 2.5x Data Size (GB) Data Size (GB) Cluster with 32 Nodes with HDD, TestDFSIO with a total 128 maps For Throughput, 6-7.8x improvement over IPoIB for GB file size For Latency, 2.5-3x improvement over IPoIB for GB file size 40

41 Performance Benefits RandomWriter & TeraGen in TACC- Stampede (HHH) Execu'on Time (s) IPoIB (FDR) OSU- IB (FDR) Reduced by 3x Execu'on Time (s) IPoIB (FDR) OSU- IB (FDR) Reduced by 4x Data Size (GB) Data Size (GB) RandomWriter Cluster with 32 Nodes with a total of 128 maps TeraGen RandomWriter 3-4x improvement over IPoIB for GB file size TeraGen 4-5x improvement over IPoIB for GB file size 41

42 Execu'on Time (s) Performance Benefits Sort & TeraSort in TACC- Stampede (HHH) IPoIB (FDR) OSU- IB (FDR) Reduced by 52% Execu'on Time (s) IPoIB (FDR) Reduced by 44% OSU- IB (FDR) Data Size (GB) Data Size (GB) Cluster with 32 Nodes with a total of 128 maps and 57 reduces Cluster with 32 Nodes with a total of 128 maps and 64 reduces Sort with single HDD per node 40-52% improvement over IPoIB for GB data TeraSort with single HDD per node 42-44% improvement over IPoIB for GB data 42

43 Performance Benefits TestDFSIO on TACC Stampede and SDSC Gordon (HHH- M) Total Throughput (MBps) IPoIB (FDR) OSU- IB (FDR) Increased by 28x Data Size (GB) Total Throughput (MBps) IPoIB (QDR) OSU- IB (QDR) Increased by 6x Data Size (GB) TACC Stampede Cluster with 32 Nodes with HDD, TestDFSIO with a total 128 maps SDSC Gordon Cluster with 16 Nodes with SSD, TestDFSIO with a total 64 maps TestDFSIO Write on TACC Stampede 28x improvement over IPoIB for GB file size TestDFSIO Write on SDSC Gordon 6x improvement over IPoIB for GB file size 43

44 Performance Benefits TestDFSIO and Sort in SDSC- Gordon (HHH- L) Total Throughput (MBps) HDFS Lustre OSU- IB Increased by 9x Increased by 29% Write TestDFSIO Read Execu'on Time (s) Cluster with 16 Nodes HDFS Lustre Reduced by 50% OSU- IB Data Size (GB) 80 Sort TestDFSIO for 80GB data size Write: 9x improvement over HDFS Read: 29% improvement over Lustre Sort up to 28% improvement over HDFS up to 50% improvement over Lustre 44

45 Presenta'on Outline Challenges for AcceleraFng Big Data Processing The High- Performance Big Data (HiBD) Project RDMA- based designs for Apache Hadoop and Spark Case studies with HDFS, MapReduce, and Spark Sample Performance Numbers for Hadoop 2.x Release RDMA- based designs for Memcached and HBase RDMA- based Memcached with SSD- assisted Hybrid Memory RDMA- based HBase Challenges in Designing Benchmarks for Big Data Processing OSU HiBD Benchmarks Conclusion and Q&A 45

46 Memcached- RDMA Design Sockets Client RDMA Client Master Thread Sockets Worker Thread Verbs Worker Thread Sockets Worker Thread Verbs Worker Thread Shared Data Memory Slabs Items Server and client perform a negofafon protocol Master thread assigns clients to appropriate worker thread Once a client is assigned a verbs worker thread, it can communicate directly and is bound to that thread All other Memcached data structures are shared among RDMA and Sockets worker threads NaFve IB- verbs- level Design and evaluafon with Server : Memcached (h<p://memcached.org) Client : libmemcached (h<p://libmemcached.org) Different networks and protocols: 10GigE, IPoIB, nafve IB (RC, UD) 46

47 Time (us) 1000 Memcached Performance (FDR Interconnect) Memcached GET Latency OSU- IB (FDR) IPoIB (FDR) Latency Reduced by nearly 20X K 2K 4K Message Size Thousands of Transac'ons per Second (TPS) Memcached Get latency 4 bytes OSU- IB: 2.84 us; IPoIB: us 2K bytes OSU- IB: 4.49 us; IPoIB: us Memcached Throughput (4bytes) 4080 clients OSU- IB: 556 Kops/sec, IPoIB: 233 Kops/s Nearly 2X improvement in throughput Experiments on TACC Stampede (Intel SandyBridge Cluster, IB: FDR) 0 Memcached Throughput No. of Clients 2X 47

48 Latency (sec) Micro- benchmark Evalua'on for OLDP workloads Memcached- IPoIB (32Gbps) Reduced by 66% 3500 Memcached- RDMA (32Gbps) No. of Clients IllustraFon with Read- Cache- Read access pa<ern using modified mysqlslap load tesfng tool Memcached- RDMA can - improve query latency by up to 66% over IPoIB (32Gbps) - throughput by up to 69% over IPoIB (32Gbps) Throughput (Kq/s) Memcached- IPoIB (32Gbps) Memcached- RDMA (32Gbps) No. of Clients D. Shankar, X. Lu, J. Jose, M. W. Rahman, N. Islam, and D. K. Panda, Can RDMA Benefit On- Line Data Processing Workloads with Memcached and MySQL, ISPASS 15 48

49 Performance Benefits on SDSC- Gordon OHB Latency & Throughput Micro- Benchmarks Average latency (us) IPoIB (32Gbps) RDMA- Mem (32Gbps) RDMA- Hyb (32Gbps) Throughput (million trans/sec) IPoIB (32Gbps) RDMA- Mem (32Gbps) RDMA- Hybrid (32Gbps) 2X Message Size (Bytes) No. of Clients ohb_memlat & ohb_memthr latency & throughput micro- benchmarks Memcached- RDMA can - improve query latency by up to 70% over IPoIB (32Gbps) - improve throughput by up to 2X over IPoIB (32Gbps) - No overhead in using hybrid mode when all data can fit in memory 49

50 Latency with penalty (us) Performance Benefits on OSU- RI- SSD OHB Micro- benchmark for Hybrid Memcached RDMA- Mem (32Gbps) RDMA- Hybrid (32Gbps) 53% 512 2K 8K 32K 128K Message Size (Bytes) ohb_memhybrid Uniform Access Pa<ern, single client and single server with 64MB Success Rate of In- Memory Vs. Hybrid SSD- Memory for different spill factors 100% success rate for Hybrid design while that of pure In- memory degrades Average Latency with penalty for In- Memory Vs. Hybrid SSD- Assisted mode for spill factor 1.5. up to 53% improvement over In- memory with server miss penalty as low as 1.5 ms Success Rate RDMA- Mem- Uniform RDMA- Hybrid Spill Factor 50

51 Presenta'on Outline Challenges for AcceleraFng Big Data Processing The High- Performance Big Data (HiBD) Project RDMA- based designs for Apache Hadoop and Spark Case studies with HDFS, MapReduce, and Spark Sample Performance Numbers for Hadoop 2.x Release RDMA- based designs for Memcached and HBase RDMA- based Memcached with SSD- assisted Hybrid Memory RDMA- based HBase Challenges in Designing Benchmarks for Big Data Processing OSU HiBD Benchmarks Conclusion and Q&A 51

52 HBase- RDMA Design Overview Applica'ons HBase Java Socket Interface Java Na've Interface (JNI) OSU- IB Design IB Verbs 1/10 GigE, IPoIB Networks RDMA Capable Networks (IB, 10GE/ iwarp, RoCE..) JNI Layer bridges Java based HBase with communicafon library wri<en in nafve code Enables high performance RDMA communicafon, while supporfng tradifonal socket interface 52

53 HBase YCSB Read- Write Workload Time (us) Time (us) IPoIB (QDR) 10GigE OSU- IB (QDR) No. of Clients No. of Clients Read Latency HBase Get latency (Yahoo! Cloud Service Benchmark) 64 clients: 2.0 ms; 128 Clients: 3.5 ms 42% improvement over IPoIB for 128 clients HBase Put latency 64 clients: 1.9 ms; 128 Clients: 3.5 ms 40% improvement over IPoIB for 128 clients Write Latency J. Huang, X. Ouyang, J. Jose, M. W. Rahman, H. Wang, M. Luo, H. Subramoni, Chet Murthy, and D. K. Panda, High- Performance Design of HBase with RDMA over InfiniBand, IPDPS 12 53

54 Presenta'on Outline Challenges for AcceleraFng Big Data Processing The High- Performance Big Data (HiBD) Project RDMA- based designs for Apache Hadoop and Spark Case studies with HDFS, MapReduce, and Spark Sample Performance Numbers for Hadoop 2.x Release RDMA- based designs for Memcached and HBase RDMA- based Memcached with SSD- assisted Hybrid Memory RDMA- based HBase Challenges in Designing Benchmarks for Big Data Processing OSU HiBD Benchmarks Conclusion and Q&A 54

55 Designing Communica'on and I/O Libraries for Big Data Systems: Solved a Few Ini'al Challenges Applica'ons Benchmarks Big Data Middleware (HDFS, MapReduce, HBase, Spark and Memcached) Upper level Changes? Programming Models (Sockets) Other RDMA Protocols? Point- to- Point Communica'on Communica'on and I/O Library Threaded Models and Synchroniza'on Virtualiza'on I/O and File Systems QoS Fault- Tolerance Networking Technologies (InfiniBand, 1/10/40GigE and Intelligent NICs) Commodity Compu'ng System Architectures (Mul'- and Many- core architectures and accelerators) Storage Technologies (HDD and SSD) 55

56 Are the Current Benchmarks Sufficient for Big Data Management and Processing? The current benchmarks provide some performance behavior However, do not provide any informafon to the designer/ developer on: What is happening at the lower- layer? Where the benefits are coming from? Which design is leading to benefits or bo<lenecks? Which component in the design needs to be changed and what will be its impact? Can performance gain/loss at the lower- layer be correlated to the performance gain/loss observed at the upper layer? 56

57 OSU MPI Micro- Benchmarks (OMB) Suite A comprehensive suite of benchmarks to Compare performance of different MPI libraries on various networks and systems Validate low- level funcfonalifes Provide insights to the underlying MPI- level designs Started with basic send- recv (MPI- 1) micro- benchmarks for latency, bandwidth and bi- direcfonal bandwidth Extended later to MPI- 2 one- sided CollecFves GPU- aware data movement OpenSHMEM (point- to- point and collecfves) UPC Has become an industry standard Extensively used for design/development of MPI libraries, performance comparison of MPI libraries and even in procurement of large- scale systems Available from h<p://mvapich.cse.ohio- state.edu/benchmarks Available in an integrated manner with MVAPICH2 stack 57

58 Challenges in Benchmarking of RDMA- based Designs Applica'ons Big Data Middleware (HDFS, MapReduce, HBase, Spark and Memcached) Programming Models (Sockets) Benchmarks Other RDMA Protocols? Current Benchmarks Correla'on? Communica'on and I/O Library Point- to- Point Communica'on Threaded Models and Synchroniza'on Virtualiza'on No Benchmarks I/O and File Systems QoS Fault- Tolerance Networking Technologies (InfiniBand, 1/10/40GigE and Intelligent NICs) Commodity Compu'ng System Architectures (Mul'- and Many- core architectures and accelerators) Storage Technologies (HDD and SSD) 58

59 Itera've Process Requires Deeper Inves'ga'on and Design for Benchmarking Next Genera'on Big Data Systems and Applica'ons Applica'ons Benchmarks Big Data Middleware (HDFS, MapReduce, HBase, Spark and Memcached) Applica'ons- Level Benchmarks Programming Models (Sockets) Other RDMA Protocols? Communica'on and I/O Library Point- to- Point Communica'on I/O and File Systems Threaded Models and Synchroniza'on QoS Virtualiza'on Fault- Tolerance Micro- Benchmarks Networking Technologies (InfiniBand, 1/10/40GigE and Intelligent NICs) Commodity Compu'ng System Architectures (Mul'- and Many- core architectures and accelerators) Storage Technologies (HDD and SSD) 59

60 OSU HiBD Micro- Benchmark (OHB) Suite - HDFS Evaluate the performance of standalone HDFS Five different benchmarks SequenFal Write Latency (SWL) SequenFal or Random Read Latency (SRL or RRL) SequenFal Write Throughput (SWT) SequenFal Read Throughput (SRT) SequenFal Read- Write Throughput (SRWT) N. S. Islam, X. Lu, M. W. Rahman, J. Jose, and D. K. Panda, A Micro- benchmark Suite for Evalua'ng HDFS Opera'ons on Modern Clusters, Int'l Workshop on Big Data Benchmarking (WBDB '12), December Benchmark File Name File Size HDFS Parameter Readers Writers Random/ SequenFal Read Seek Interval SWL SRL/RRL (RRL) SWT SRT SRWT 60

61 OSU HiBD Micro- Benchmark (OHB) Suite - RPC Two different micro- benchmarks to evaluate the performance of standalone Hadoop RPC Latency: Single Server, Single Client Throughput: Single Server, MulFple Clients A simple script framework for job launching and resource monitoring Calculates stafsfcs like Min, Max, Average Network configurafon, Tunable parameters, DataType, CPU UFlizaFon Component Network Address Port Data Type Min Msg Size Max Msg Size No. of IteraFons Handlers lat_client lat_server Verbose Component Network Address Port Data Type Min Msg Size Max Msg Size No. of IteraFons No. of Clients Handlers Verbose thr_client thr_server X. Lu, M. W. Rahman, N. Islam, and D. K. Panda, A Micro- Benchmark Suite for Evalua'ng Hadoop RPC on High- Performance Networks, Int'l Workshop on Big Data Benchmarking (WBDB '13), July

62 OSU HiBD Micro- Benchmark (OHB) Suite - MapReduce Evaluate the performance of stand- alone MapReduce Does not require or involve HDFS or any other distributed file system Considers various factors that influence the data shuffling phase underlying network configurafon, number of map and reduce tasks, intermediate shuffle data pa<ern, shuffle data size etc. Three different micro- benchmarks based on intermediate shuffle data pa<erns MR- AVG micro- benchmark: intermediate data is evenly distributed among reduce tasks. MR- RAND micro- benchmark: intermediate data is pseudo- randomly distributed among reduce tasks. MR- SKEW micro- benchmark: intermediate data is unevenly distributed among reduce tasks. D. Shankar, X. Lu, M. W. Rahman, N. Islam, and D. K. Panda, A Micro- Benchmark Suite for Evalua'ng Hadoop MapReduce on High- Performance Networks, BPOE- 5 (2014). 62

63 Future Plans of OSU High Performance Big Data Project Upcoming Releases of RDMA- enhanced Packages will support Spark HBase Plugin- based designs Upcoming Releases of OSU HiBD Micro- Benchmarks (OHB) will support HDFS MapReduce RPC ExploraFon of other components (Threading models, QoS, VirtualizaFon, Accelerators, etc.) Advanced designs with upper- level changes and opfmizafons 63

64 Concluding Remarks Presented an overview of Big Data processing middleware Discussed challenges in accelerafng Big Data middleware Presented inifal designs to take advantage of InfiniBand/RDMA for Hadoop, Spark, Memcached, and HBase Presented challenges in designing benchmarks Results are promising Many other open issues need to be solved Will enable Big processing community to take advantage of modern HPC technologies to carry out their analyfcs in a fast and scalable manner 64

65 Personnel Acknowledgments Current Students A. Awan (Ph.D.) M. Li (Ph.D.) A. Bhat (M.S.) M. Rahman (Ph.D.) S. Chakraborthy (Ph.D.) D. Shankar (Ph.D.) C.- H. Chu (Ph.D.) A. Venkatesh (Ph.D.) N. Islam (Ph.D.) J. Zhang (Ph.D.) Past Students P. Balaji (Ph.D.) W. Huang (Ph.D.) D. BunFnas (Ph.D.) W. Jiang (M.S.) S. Bhagvat (M.S.) J. Jose (Ph.D.) L. Chai (Ph.D.) S. Kini (M.S.) B. Chandrasekharan (M.S.) M. Koop (Ph.D.) N. Dandapanthula (M.S.) R. Kumar (M.S.) V. Dhanraj (M.S.) S. Krishnamoorthy (M.S.) T. Gangadharappa (M.S.) K. Kandalla (Ph.D.) K. Gopalakrishnan (M.S.) P. Lai (M.S.) Past Post- Docs J. Liu (Ph.D.) H. Wang E. Mancini X. Besseron S. Marcarelli H.- W. Jin J. Vienne M. Luo Current Senior Research Associates K. Hamidouche H. Subramoni X. Lu Current Post- Doc Current Programmer J. Lin J. Perkins Current Research Specialist M. Arnold M. Luo (Ph.D.) G. Santhanaraman (Ph.D.) A. Mamidala (Ph.D.) A. Singh (Ph.D.) G. Marsh (M.S.) J. Sridhar (M.S.) V. Meshram (M.S.) S. Sur (Ph.D.) S. Naravula (Ph.D.) H. Subramoni (Ph.D.) R. Noronha (Ph.D.) K. Vaidyanathan (Ph.D.) X. Ouyang (Ph.D.) A. Vishnu (Ph.D.) S. Pai (M.S.) J. Wu (Ph.D.) S. Potluri (Ph.D.) W. Yu (Ph.D.) R. Rajachandrasekar (Ph.D.) Past Research Scien8st Past Programmers S. Sur D. Bureddy 65

66 Thank You! state.edu Network- Based CompuFng Laboratory h<p://nowlab.cse.ohio- state.edu/ The MVAPICH2/MVAPICH2- X Project h<p://mvapich.cse.ohio- state.edu/ The High- Performance Big Data Project h<p://hibd.cse.ohio- state.edu/ 66

67 Call For Par'cipa'on Interna'onal Workshop on High- Performance Big Data Compu'ng (HPBDC 2015) In conjuncfon with InternaFonal Conference on Distributed CompuFng Systems (ICDCS 2015) In Hilton Downtown, Columbus, Ohio, USA, Monday, June 29th, 2015 h<p://web.cse.ohio- state.edu/~luxi/hpbdc

68 Mul'ple Posi'ons Available in My Group Looking for Bright and EnthusiasFc Personnel to join as Post- Doctoral Researchers PhD Students Hadoop/Big Data Programmer/Sonware Engineer MPI Programmer/Sonware Engineer If interested, please contact me at this conference and/or send an e- mail to state.edu HPCAC China, Nov. '14 68

Accelerating Big Data Processing with Hadoop, Spark and Memcached

Accelerating Big Data Processing with Hadoop, Spark and Memcached Accelerating Big Data Processing with Hadoop, Spark and Memcached Talk at HPC Advisory Council Stanford Conference (Feb 15) by Dhabaleswar K. (DK) Panda The Ohio State University E-mail: panda@cse.ohio-state.edu

More information

Accelerating Big Data Processing with Hadoop, Spark and Memcached

Accelerating Big Data Processing with Hadoop, Spark and Memcached Accelerating Big Data Processing with Hadoop, Spark and Memcached Talk at HPC Advisory Council Switzerland Conference (Mar 15) by Dhabaleswar K. (DK) Panda The Ohio State University E-mail: panda@cse.ohio-state.edu

More information

Accelerating Spark with RDMA for Big Data Processing: Early Experiences

Accelerating Spark with RDMA for Big Data Processing: Early Experiences Accelerating Spark with RDMA for Big Data Processing: Early Experiences Xiaoyi Lu, Md. Wasi- ur- Rahman, Nusrat Islam, Dip7 Shankar, and Dhabaleswar K. (DK) Panda Network- Based Compu2ng Laboratory Department

More information

Big Data: Hadoop and Memcached

Big Data: Hadoop and Memcached Big Data: Hadoop and Memcached Talk at HPC Advisory Council Stanford Conference and Exascale Workshop (Feb 214) by Dhabaleswar K. (DK) Panda The Ohio State University E-mail: panda@cse.ohio-state.edu http://www.cse.ohio-state.edu/~panda

More information

A Micro-benchmark Suite for Evaluating Hadoop RPC on High-Performance Networks

A Micro-benchmark Suite for Evaluating Hadoop RPC on High-Performance Networks A Micro-benchmark Suite for Evaluating Hadoop RPC on High-Performance Networks Xiaoyi Lu, Md. Wasi- ur- Rahman, Nusrat Islam, and Dhabaleswar K. (DK) Panda Network- Based Compu2ng Laboratory Department

More information

Can High-Performance Interconnects Benefit Memcached and Hadoop?

Can High-Performance Interconnects Benefit Memcached and Hadoop? Can High-Performance Interconnects Benefit Memcached and Hadoop? D. K. Panda and Sayantan Sur Network-Based Computing Laboratory Department of Computer Science and Engineering The Ohio State University,

More information

Enabling High performance Big Data platform with RDMA

Enabling High performance Big Data platform with RDMA Enabling High performance Big Data platform with RDMA Tong Liu HPC Advisory Council Oct 7 th, 2014 Shortcomings of Hadoop Administration tooling Performance Reliability SQL support Backup and recovery

More information

Accelerating Big Data Processing with Hadoop, Spark, and Memcached over High-Performance Interconnects

Accelerating Big Data Processing with Hadoop, Spark, and Memcached over High-Performance Interconnects Slides are available from http://www.cse.ohio-state.edu/~panda/hoti15-bigdata-tut.pdf Accelerating Big Data Processing with Hadoop, Spark, and Memcached over High-Performance Interconnects Tutorial at

More information

RDMA over Ethernet - A Preliminary Study

RDMA over Ethernet - A Preliminary Study RDMA over Ethernet - A Preliminary Study Hari Subramoni, Miao Luo, Ping Lai and Dhabaleswar. K. Panda Computer Science & Engineering Department The Ohio State University Outline Introduction Problem Statement

More information

Hadoop on the Gordon Data Intensive Cluster

Hadoop on the Gordon Data Intensive Cluster Hadoop on the Gordon Data Intensive Cluster Amit Majumdar, Scientific Computing Applications Mahidhar Tatineni, HPC User Services San Diego Supercomputer Center University of California San Diego Dec 18,

More information

Hari Subramoni. Education: Employment: Research Interests: Projects:

Hari Subramoni. Education: Employment: Research Interests: Projects: Hari Subramoni Senior Research Associate, Dept. of Computer Science and Engineering The Ohio State University, Columbus, OH 43210 1277 Tel: (614) 961 2383, Fax: (614) 292 2911, E-mail: subramoni.1@osu.edu

More information

RDMA-based Plugin Design and Profiler for Apache and Enterprise Hadoop Distributed File system THESIS

RDMA-based Plugin Design and Profiler for Apache and Enterprise Hadoop Distributed File system THESIS RDMA-based Plugin Design and Profiler for Apache and Enterprise Hadoop Distributed File system THESIS Presented in Partial Fulfillment of the Requirements for the Degree Master of Science in the Graduate

More information

Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012

Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012 Unstructured Data Accelerator (UDA) Author: Motti Beck, Mellanox Technologies Date: March 27, 2012 1 Market Trends Big Data Growing technology deployments are creating an exponential increase in the volume

More information

HiBench Introduction. Carson Wang (carson.wang@intel.com) Software & Services Group

HiBench Introduction. Carson Wang (carson.wang@intel.com) Software & Services Group HiBench Introduction Carson Wang (carson.wang@intel.com) Agenda Background Workloads Configurations Benchmark Report Tuning Guide Background WHY Why we need big data benchmarking systems? WHAT What is

More information

High Performance Data-Transfers in Grid Environment using GridFTP over InfiniBand

High Performance Data-Transfers in Grid Environment using GridFTP over InfiniBand High Performance Data-Transfers in Grid Environment using GridFTP over InfiniBand Hari Subramoni *, Ping Lai *, Raj Kettimuthu **, Dhabaleswar. K. (DK) Panda * * Computer Science and Engineering Department

More information

Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA

Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA WHITE PAPER April 2014 Driving IBM BigInsights Performance Over GPFS Using InfiniBand+RDMA Executive Summary...1 Background...2 File Systems Architecture...2 Network Architecture...3 IBM BigInsights...5

More information

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand

Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand Exploiting Remote Memory Operations to Design Efficient Reconfiguration for Shared Data-Centers over InfiniBand P. Balaji, K. Vaidyanathan, S. Narravula, K. Savitha, H. W. Jin D. K. Panda Network Based

More information

MapReduce Evaluator: User Guide

MapReduce Evaluator: User Guide University of A Coruña Computer Architecture Group MapReduce Evaluator: User Guide Authors: Jorge Veiga, Roberto R. Expósito, Guillermo L. Taboada and Juan Touriño December 9, 2014 Contents 1 Overview

More information

GraySort on Apache Spark by Databricks

GraySort on Apache Spark by Databricks GraySort on Apache Spark by Databricks Reynold Xin, Parviz Deyhim, Ali Ghodsi, Xiangrui Meng, Matei Zaharia Databricks Inc. Apache Spark Sorting in Spark Overview Sorting Within a Partition Range Partitioner

More information

Big Fast Data Hadoop acceleration with Flash. June 2013

Big Fast Data Hadoop acceleration with Flash. June 2013 Big Fast Data Hadoop acceleration with Flash June 2013 Agenda The Big Data Problem What is Hadoop Hadoop and Flash The Nytro Solution Test Results The Big Data Problem Big Data Output Facebook Traditional

More information

A Micro-benchmark Suite for Evaluating Hadoop MapReduce on High-Performance Networks

A Micro-benchmark Suite for Evaluating Hadoop MapReduce on High-Performance Networks A Micro-benchmark Suite for Evaluating Hadoop MapReduce on High-Performance Networks Dipti Shankar (B), Xiaoyi Lu, Md. Wasi-ur-Rahman, Nusrat Islam, and Dhabaleswar K. (DK) Panda Department of Computer

More information

Hadoop Optimizations for BigData Analytics

Hadoop Optimizations for BigData Analytics Hadoop Optimizations for BigData Analytics Weikuan Yu Auburn University Outline WBDB, Oct 2012 S-2 Background Network Levitated Merge JVM-Bypass Shuffling Fast Completion Scheduler WBDB, Oct 2012 S-3 Emerging

More information

SMB Direct for SQL Server and Private Cloud

SMB Direct for SQL Server and Private Cloud SMB Direct for SQL Server and Private Cloud Increased Performance, Higher Scalability and Extreme Resiliency June, 2014 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server

More information

Performance Comparison of Intel Enterprise Edition for Lustre* software and HDFS for MapReduce Applications

Performance Comparison of Intel Enterprise Edition for Lustre* software and HDFS for MapReduce Applications Performance Comparison of Intel Enterprise Edition for Lustre software and HDFS for MapReduce Applications Rekha Singhal, Gabriele Pacciucci and Mukesh Gangadhar 2 Hadoop Introduc-on Open source MapReduce

More information

Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014

Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet. September 2014 Comparing SMB Direct 3.0 performance over RoCE, InfiniBand and Ethernet Anand Rangaswamy September 2014 Storage Developer Conference Mellanox Overview Ticker: MLNX Leading provider of high-throughput,

More information

Hadoop: Embracing future hardware

Hadoop: Embracing future hardware Hadoop: Embracing future hardware Suresh Srinivas @suresh_m_s Page 1 About Me Architect & Founder at Hortonworks Long time Apache Hadoop committer and PMC member Designed and developed many key Hadoop

More information

Outline. High Performance Computing (HPC) Big Data meets HPC. Case Studies: Some facts about Big Data Technologies HPC and Big Data converging

Outline. High Performance Computing (HPC) Big Data meets HPC. Case Studies: Some facts about Big Data Technologies HPC and Big Data converging Outline High Performance Computing (HPC) Towards exascale computing: a brief history Challenges in the exascale era Big Data meets HPC Some facts about Big Data Technologies HPC and Big Data converging

More information

RDMA for Apache Hadoop 0.9.9 User Guide

RDMA for Apache Hadoop 0.9.9 User Guide 0.9.9 User Guide HIGH-PERFORMANCE BIG DATA TEAM http://hibd.cse.ohio-state.edu NETWORK-BASED COMPUTING LABORATORY DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING THE OHIO STATE UNIVERSITY Copyright (c)

More information

Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software

Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software WHITEPAPER Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software SanDisk ZetaScale software unlocks the full benefits of flash for In-Memory Compute and NoSQL applications

More information

Maximizing Hadoop Performance and Storage Capacity with AltraHD TM

Maximizing Hadoop Performance and Storage Capacity with AltraHD TM Maximizing Hadoop Performance and Storage Capacity with AltraHD TM Executive Summary The explosion of internet data, driven in large part by the growth of more and more powerful mobile devices, has created

More information

Accelerating and Simplifying Apache

Accelerating and Simplifying Apache Accelerating and Simplifying Apache Hadoop with Panasas ActiveStor White paper NOvember 2012 1.888.PANASAS www.panasas.com Executive Overview The technology requirements for big data vary significantly

More information

Hadoop Deployment and Performance on Gordon Data Intensive Supercomputer!

Hadoop Deployment and Performance on Gordon Data Intensive Supercomputer! Hadoop Deployment and Performance on Gordon Data Intensive Supercomputer! Mahidhar Tatineni, Rick Wagner, Eva Hocks, Christopher Irving, and Jerry Greenberg! SDSC! XSEDE13, July 22-25, 2013! Overview!!

More information

Oracle s Big Data solutions. Roger Wullschleger. <Insert Picture Here>

Oracle s Big Data solutions. Roger Wullschleger. <Insert Picture Here> s Big Data solutions Roger Wullschleger DBTA Workshop on Big Data, Cloud Data Management and NoSQL 10. October 2012, Stade de Suisse, Berne 1 The following is intended to outline

More information

Advancing Applications Performance With InfiniBand

Advancing Applications Performance With InfiniBand Advancing Applications Performance With InfiniBand Pak Lui, Application Performance Manager September 12, 2013 Mellanox Overview Ticker: MLNX Leading provider of high-throughput, low-latency server and

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

Sockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck

Sockets vs. RDMA Interface over 10-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Sockets vs. RDMA Interface over 1-Gigabit Networks: An In-depth Analysis of the Memory Traffic Bottleneck Pavan Balaji Hemal V. Shah D. K. Panda Network Based Computing Lab Computer Science and Engineering

More information

FLOW-3D Performance Benchmark and Profiling. September 2012

FLOW-3D Performance Benchmark and Profiling. September 2012 FLOW-3D Performance Benchmark and Profiling September 2012 Note The following research was performed under the HPC Advisory Council activities Participating vendors: FLOW-3D, Dell, Intel, Mellanox Compute

More information

Introduction to Hadoop on the SDSC Gordon Data Intensive Cluster"

Introduction to Hadoop on the SDSC Gordon Data Intensive Cluster Introduction to Hadoop on the SDSC Gordon Data Intensive Cluster" Mahidhar Tatineni SDSC Summer Institute August 06, 2013 Overview "" Hadoop framework extensively used for scalable distributed processing

More information

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms

Distributed File System. MCSN N. Tonellotto Complements of Distributed Enabling Platforms Distributed File System 1 How do we get data to the workers? NAS Compute Nodes SAN 2 Distributed File System Don t move data to workers move workers to the data! Store data on the local disks of nodes

More information

Mellanox Cloud and Database Acceleration Solution over Windows Server 2012 SMB Direct

Mellanox Cloud and Database Acceleration Solution over Windows Server 2012 SMB Direct Mellanox Cloud and Database Acceleration Solution over Windows Server 2012 Direct Increased Performance, Scaling and Resiliency July 2012 Motti Beck, Director, Enterprise Market Development Motti@mellanox.com

More information

GraySort and MinuteSort at Yahoo on Hadoop 0.23

GraySort and MinuteSort at Yahoo on Hadoop 0.23 GraySort and at Yahoo on Hadoop.23 Thomas Graves Yahoo! May, 213 The Apache Hadoop[1] software library is an open source framework that allows for the distributed processing of large data sets across clusters

More information

Lustre * Filesystem for Cloud and Hadoop *

Lustre * Filesystem for Cloud and Hadoop * OpenFabrics Software User Group Workshop Lustre * Filesystem for Cloud and Hadoop * Robert Read, Intel Lustre * for Cloud and Hadoop * Brief Lustre History and Overview Using Lustre with Hadoop Intel Cloud

More information

Overview on Modern Accelerators and Programming Paradigms Ivan Giro7o igiro7o@ictp.it

Overview on Modern Accelerators and Programming Paradigms Ivan Giro7o igiro7o@ictp.it Overview on Modern Accelerators and Programming Paradigms Ivan Giro7o igiro7o@ictp.it Informa(on & Communica(on Technology Sec(on (ICTS) Interna(onal Centre for Theore(cal Physics (ICTP) Mul(ple Socket

More information

New Storage System Solutions

New Storage System Solutions New Storage System Solutions Craig Prescott Research Computing May 2, 2013 Outline } Existing storage systems } Requirements and Solutions } Lustre } /scratch/lfs } Questions? Existing Storage Systems

More information

Big Data Analytics - Accelerated. stream-horizon.com

Big Data Analytics - Accelerated. stream-horizon.com Big Data Analytics - Accelerated stream-horizon.com Legacy ETL platforms & conventional Data Integration approach Unable to meet latency & data throughput demands of Big Data integration challenges Based

More information

How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014

How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014 How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014 2014 年 云 计 算 效 率 与 能 耗 暨 第 一 届 国 际 云 计 算 咨 询 委 员 会 中 国 高 峰 论 坛 Contents Background

More information

Hadoop IST 734 SS CHUNG

Hadoop IST 734 SS CHUNG Hadoop IST 734 SS CHUNG Introduction What is Big Data?? Bulk Amount Unstructured Lots of Applications which need to handle huge amount of data (in terms of 500+ TB per day) If a regular machine need to

More information

Installing Hadoop over Ceph, Using High Performance Networking

Installing Hadoop over Ceph, Using High Performance Networking WHITE PAPER March 2014 Installing Hadoop over Ceph, Using High Performance Networking Contents Background...2 Hadoop...2 Hadoop Distributed File System (HDFS)...2 Ceph...2 Ceph File System (CephFS)...3

More information

Hadoop MapReduce over Lustre* High Performance Data Division Omkar Kulkarni April 16, 2013

Hadoop MapReduce over Lustre* High Performance Data Division Omkar Kulkarni April 16, 2013 Hadoop MapReduce over Lustre* High Performance Data Division Omkar Kulkarni April 16, 2013 * Other names and brands may be claimed as the property of others. Agenda Hadoop Intro Why run Hadoop on Lustre?

More information

CSE-E5430 Scalable Cloud Computing Lecture 2

CSE-E5430 Scalable Cloud Computing Lecture 2 CSE-E5430 Scalable Cloud Computing Lecture 2 Keijo Heljanko Department of Computer Science School of Science Aalto University keijo.heljanko@aalto.fi 14.9-2015 1/36 Google MapReduce A scalable batch processing

More information

Self service for software development tools

Self service for software development tools Self service for software development tools Michal Husejko, behalf of colleagues in CERN IT/PES CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/it Self service for software development tools

More information

Hadoop Ecosystem B Y R A H I M A.

Hadoop Ecosystem B Y R A H I M A. Hadoop Ecosystem B Y R A H I M A. History of Hadoop Hadoop was created by Doug Cutting, the creator of Apache Lucene, the widely used text search library. Hadoop has its origins in Apache Nutch, an open

More information

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance

LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance 11 th International LS-DYNA Users Conference Session # LS-DYNA Best-Practices: Networking, MPI and Parallel File System Effect on LS-DYNA Performance Gilad Shainer 1, Tong Liu 2, Jeff Layton 3, Onur Celebioglu

More information

Introduction to Infiniband. Hussein N. Harake, Performance U! Winter School

Introduction to Infiniband. Hussein N. Harake, Performance U! Winter School Introduction to Infiniband Hussein N. Harake, Performance U! Winter School Agenda Definition of Infiniband Features Hardware Facts Layers OFED Stack OpenSM Tools and Utilities Topologies Infiniband Roadmap

More information

Architecting for the next generation of Big Data Hortonworks HDP 2.0 on Red Hat Enterprise Linux 6 with OpenJDK 7

Architecting for the next generation of Big Data Hortonworks HDP 2.0 on Red Hat Enterprise Linux 6 with OpenJDK 7 Architecting for the next generation of Big Data Hortonworks HDP 2.0 on Red Hat Enterprise Linux 6 with OpenJDK 7 Yan Fisher Senior Principal Product Marketing Manager, Red Hat Rohit Bakhshi Product Manager,

More information

Map- reduce, Hadoop and The communica3on bo5leneck. Yoav Freund UCSD / Computer Science and Engineering

Map- reduce, Hadoop and The communica3on bo5leneck. Yoav Freund UCSD / Computer Science and Engineering Map- reduce, Hadoop and The communica3on bo5leneck Yoav Freund UCSD / Computer Science and Engineering Plan of the talk Why is Hadoop so popular? HDFS Map Reduce Word Count example using Hadoop streaming

More information

Benchmarking Hadoop & HBase on Violin

Benchmarking Hadoop & HBase on Violin Technical White Paper Report Technical Report Benchmarking Hadoop & HBase on Violin Harnessing Big Data Analytics at the Speed of Memory Version 1.0 Abstract The purpose of benchmarking is to show advantages

More information

High-Performance Networking for Optimized Hadoop Deployments

High-Performance Networking for Optimized Hadoop Deployments High-Performance Networking for Optimized Hadoop Deployments Chelsio Terminator 4 (T4) Unified Wire adapters deliver a range of performance gains for Hadoop by bringing the Hadoop cluster networking into

More information

Apache Hadoop. Alexandru Costan

Apache Hadoop. Alexandru Costan 1 Apache Hadoop Alexandru Costan Big Data Landscape No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard, except Hadoop 2 Outline What is Hadoop? Who uses it? Architecture HDFS MapReduce Open

More information

Application and Micro-benchmark Performance using MVAPICH2-X on SDSC Gordon Cluster

Application and Micro-benchmark Performance using MVAPICH2-X on SDSC Gordon Cluster Application and Micro-benchmark Performance using MVAPICH2-X on SDSC Gordon Cluster Mahidhar Tatineni (mahidhar@sdsc.edu) MVAPICH User Group Meeting August 27, 2014 NSF grants: OCI #0910847 Gordon: A Data

More information

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray ware 2 Agenda The Hadoop Journey Why Virtualize Hadoop? Elasticity and Scalability Performance Tests Storage Reference

More information

Windows 8 SMB 2.2 File Sharing Performance

Windows 8 SMB 2.2 File Sharing Performance Windows 8 SMB 2.2 File Sharing Performance Abstract This paper provides a preliminary analysis of the performance capabilities of the Server Message Block (SMB) 2.2 file sharing protocol with 10 gigabit

More information

InfiniBand Software and Protocols Enable Seamless Off-the-shelf Applications Deployment

InfiniBand Software and Protocols Enable Seamless Off-the-shelf Applications Deployment December 2007 InfiniBand Software and Protocols Enable Seamless Off-the-shelf Deployment 1.0 Introduction InfiniBand architecture defines a high-bandwidth, low-latency clustering interconnect that is used

More information

Simplifying Big Data Deployments in Cloud Environments with Mellanox Interconnects and QualiSystems Orchestration Solutions

Simplifying Big Data Deployments in Cloud Environments with Mellanox Interconnects and QualiSystems Orchestration Solutions Simplifying Big Data Deployments in Cloud Environments with Mellanox Interconnects and QualiSystems Orchestration Solutions 64% of organizations were investing or planning to invest on Big Data technology

More information

Understanding Hadoop Performance on Lustre

Understanding Hadoop Performance on Lustre Understanding Hadoop Performance on Lustre Stephen Skory, PhD Seagate Technology Collaborators Kelsie Betsch, Daniel Kaslovsky, Daniel Lingenfelter, Dimitar Vlassarev, and Zhenzhen Yan LUG Conference 15

More information

Data Management in the Cloud: Limitations and Opportunities. Annies Ductan

Data Management in the Cloud: Limitations and Opportunities. Annies Ductan Data Management in the Cloud: Limitations and Opportunities Annies Ductan Discussion Outline: Introduc)on Overview Vision of Cloud Compu8ng Managing Data in The Cloud Cloud Characteris8cs Data Management

More information

Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control

Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control Developing Scalable Smart Grid Infrastructure to Enable Secure Transmission System Control EP/K006487/1 UK PI: Prof Gareth Taylor (BU) China PI: Prof Yong-Hua Song (THU) Consortium UK Members: Brunel University

More information

Network-Aware Scheduling of MapReduce Framework on Distributed Clusters over High Speed Networks

Network-Aware Scheduling of MapReduce Framework on Distributed Clusters over High Speed Networks Network-Aware Scheduling of MapReduce Framework on Distributed Clusters over High Speed Networks Praveenkumar Kondikoppa, Chui-Hui Chiu, Cheng Cui, Lin Xue and Seung-Jong Park Department of Computer Science,

More information

Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems

Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems Rekha Singhal and Gabriele Pacciucci * Other names and brands may be claimed as the property of others. Lustre File

More information

Analysis and Optimization of Massive Data Processing on High Performance Computing Architecture

Analysis and Optimization of Massive Data Processing on High Performance Computing Architecture Analysis and Optimization of Massive Data Processing on High Performance Computing Architecture He Huang, Shanshan Li, Xiaodong Yi, Feng Zhang, Xiangke Liao and Pan Dong School of Computer Science National

More information

Accelerating Hadoop MapReduce Using an In-Memory Data Grid

Accelerating Hadoop MapReduce Using an In-Memory Data Grid Accelerating Hadoop MapReduce Using an In-Memory Data Grid By David L. Brinker and William L. Bain, ScaleOut Software, Inc. 2013 ScaleOut Software, Inc. 12/27/2012 H adoop has been widely embraced for

More information

THE HADOOP DISTRIBUTED FILE SYSTEM

THE HADOOP DISTRIBUTED FILE SYSTEM THE HADOOP DISTRIBUTED FILE SYSTEM Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler Presented by Alexander Pokluda October 7, 2013 Outline Motivation and Overview of Hadoop Architecture,

More information

Near Real Time Indexing Kafka Message to Apache Blur using Spark Streaming. by Dibyendu Bhattacharya

Near Real Time Indexing Kafka Message to Apache Blur using Spark Streaming. by Dibyendu Bhattacharya Near Real Time Indexing Kafka Message to Apache Blur using Spark Streaming by Dibyendu Bhattacharya Pearson : What We Do? We are building a scalable, reliable cloud-based learning platform providing services

More information

Spark in Action. Fast Big Data Analytics using Scala. Matei Zaharia. www.spark- project.org. University of California, Berkeley UC BERKELEY

Spark in Action. Fast Big Data Analytics using Scala. Matei Zaharia. www.spark- project.org. University of California, Berkeley UC BERKELEY Spark in Action Fast Big Data Analytics using Scala Matei Zaharia University of California, Berkeley www.spark- project.org UC BERKELEY My Background Grad student in the AMP Lab at UC Berkeley» 50- person

More information

HPC ABDS: The Case for an Integrating Apache Big Data Stack

HPC ABDS: The Case for an Integrating Apache Big Data Stack HPC ABDS: The Case for an Integrating Apache Big Data Stack with HPC 1st JTC 1 SGBD Meeting SDSC San Diego March 19 2014 Judy Qiu Shantenu Jha (Rutgers) Geoffrey Fox gcf@indiana.edu http://www.infomall.org

More information

Hadoop Architecture. Part 1

Hadoop Architecture. Part 1 Hadoop Architecture Part 1 Node, Rack and Cluster: A node is simply a computer, typically non-enterprise, commodity hardware for nodes that contain data. Consider we have Node 1.Then we can add more nodes,

More information

Wrangler: A New Generation of Data-intensive Supercomputing. Christopher Jordan, Siva Kulasekaran, Niall Gaffney

Wrangler: A New Generation of Data-intensive Supercomputing. Christopher Jordan, Siva Kulasekaran, Niall Gaffney Wrangler: A New Generation of Data-intensive Supercomputing Christopher Jordan, Siva Kulasekaran, Niall Gaffney Project Partners Academic partners: TACC Primary system design, deployment, and operations

More information

Chapter 7. Using Hadoop Cluster and MapReduce

Chapter 7. Using Hadoop Cluster and MapReduce Chapter 7 Using Hadoop Cluster and MapReduce Modeling and Prototyping of RMS for QoS Oriented Grid Page 152 7. Using Hadoop Cluster and MapReduce for Big Data Problems The size of the databases used in

More information

High Throughput File Servers with SMB Direct, Using the 3 Flavors of RDMA network adapters

High Throughput File Servers with SMB Direct, Using the 3 Flavors of RDMA network adapters High Throughput File Servers with SMB Direct, Using the 3 Flavors of network adapters Jose Barreto Principal Program Manager Microsoft Corporation Abstract In Windows Server 2012, we introduce the SMB

More information

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat ESS event: Big Data in Official Statistics Antonino Virgillito, Istat v erbi v is 1 About me Head of Unit Web and BI Technologies, IT Directorate of Istat Project manager and technical coordinator of Web

More information

Mambo Running Analytics on Enterprise Storage

Mambo Running Analytics on Enterprise Storage Mambo Running Analytics on Enterprise Storage Jingxin Feng, Xing Lin 1, Gokul Soundararajan Advanced Technology Group 1 University of Utah Motivation No easy way to analyze data stored in enterprise storage

More information

Cray XC30 Hadoop Platform Jonathan (Bill) Sparks Howard Pritchard Martha Dumler

Cray XC30 Hadoop Platform Jonathan (Bill) Sparks Howard Pritchard Martha Dumler Cray XC30 Hadoop Platform Jonathan (Bill) Sparks Howard Pritchard Martha Dumler Safe Harbor Statement This presentation may contain forward-looking statements that are based on our current expectations.

More information

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks

Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks WHITE PAPER July 2014 Achieving Real-Time Business Solutions Using Graph Database Technology and High Performance Networks Contents Executive Summary...2 Background...3 InfiniteGraph...3 High Performance

More information

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency

Solving I/O Bottlenecks to Enable Superior Cloud Efficiency WHITE PAPER Solving I/O Bottlenecks to Enable Superior Cloud Efficiency Overview...1 Mellanox I/O Virtualization Features and Benefits...2 Summary...6 Overview We already have 8 or even 16 cores on one

More information

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh

Hadoop: A Framework for Data- Intensive Distributed Computing. CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 1 Hadoop: A Framework for Data- Intensive Distributed Computing CS561-Spring 2012 WPI, Mohamed Y. Eltabakh 2 What is Hadoop? Hadoop is a software framework for distributed processing of large datasets

More information

Storage Architectures for Big Data in the Cloud

Storage Architectures for Big Data in the Cloud Storage Architectures for Big Data in the Cloud Sam Fineberg HP Storage CT Office/ May 2013 Overview Introduction What is big data? Big Data I/O Hadoop/HDFS SAN Distributed FS Cloud Summary Research Areas

More information

Performance Management in Big Data Applica6ons. Michael Kopp, Technology Strategist @mikopp

Performance Management in Big Data Applica6ons. Michael Kopp, Technology Strategist @mikopp Performance Management in Big Data Applica6ons Michael Kopp, Technology Strategist NoSQL: High Volume/Low Latency DBs Web Java Key Challenges 1) Even Distribu6on 2) Correct Schema and Access paperns 3)

More information

CS2510 Computer Operating Systems

CS2510 Computer Operating Systems CS2510 Computer Operating Systems HADOOP Distributed File System Dr. Taieb Znati Computer Science Department University of Pittsburgh Outline HDF Design Issues HDFS Application Profile Block Abstraction

More information

CS2510 Computer Operating Systems

CS2510 Computer Operating Systems CS2510 Computer Operating Systems HADOOP Distributed File System Dr. Taieb Znati Computer Science Department University of Pittsburgh Outline HDF Design Issues HDFS Application Profile Block Abstraction

More information

Michael Kagan. michael@mellanox.com

Michael Kagan. michael@mellanox.com Virtualization in Data Center The Network Perspective Michael Kagan CTO, Mellanox Technologies michael@mellanox.com Outline Data Center Transition Servers S as a Service Network as a Service IO as a Service

More information

TCP/IP Implementation of Hadoop Acceleration. Cong Xu

TCP/IP Implementation of Hadoop Acceleration. Cong Xu TCP/IP Implementation of Hadoop Acceleration by Cong Xu A thesis submitted to the Graduate Faculty of Auburn University in partial fulfillment of the requirements for the Degree of Master of Science Auburn,

More information

HPC data becomes Big Data. Peter Braam peter.braam@braamresearch.com

HPC data becomes Big Data. Peter Braam peter.braam@braamresearch.com HPC data becomes Big Data Peter Braam peter.braam@braamresearch.com me 1983-2000 Academia Maths & Computer Science Entrepreneur with startups (5x) 4 startups sold Lustre emerged Held executive jobs with

More information

Designing Efficient FTP Mechanisms for High Performance Data-Transfer over InfiniBand

Designing Efficient FTP Mechanisms for High Performance Data-Transfer over InfiniBand Designing Efficient FTP Mechanisms for High Performance Data-Transfer over InfiniBand Ping Lai, Hari Subramoni, Sundeep Narravula, Amith Mamidala, Dhabaleswar K. Panda Department of Computer Science and

More information

Microsoft SMB 2.2 - Running Over RDMA in Windows Server 8

Microsoft SMB 2.2 - Running Over RDMA in Windows Server 8 Microsoft SMB 2.2 - Running Over RDMA in Windows Server 8 Tom Talpey, Architect Microsoft March 27, 2012 1 SMB2 Background The primary Windows filesharing protocol Initially shipped in Vista and Server

More information

Ant Rowstron. Joint work with Paolo Costa, Austin Donnelly and Greg O Shea Microsoft Research Cambridge. Hussam Abu-Libdeh, Simon Schubert Interns

Ant Rowstron. Joint work with Paolo Costa, Austin Donnelly and Greg O Shea Microsoft Research Cambridge. Hussam Abu-Libdeh, Simon Schubert Interns Ant Rowstron Joint work with Paolo Costa, Austin Donnelly and Greg O Shea Microsoft Research Cambridge Hussam Abu-Libdeh, Simon Schubert Interns Thinking of large-scale data centers Microsoft, Google,

More information

How To Monitor Infiniband Network Data From A Network On A Leaf Switch (Wired) On A Microsoft Powerbook (Wired Or Microsoft) On An Ipa (Wired/Wired) Or Ipa V2 (Wired V2)

How To Monitor Infiniband Network Data From A Network On A Leaf Switch (Wired) On A Microsoft Powerbook (Wired Or Microsoft) On An Ipa (Wired/Wired) Or Ipa V2 (Wired V2) INFINIBAND NETWORK ANALYSIS AND MONITORING USING OPENSM N. Dandapanthula 1, H. Subramoni 1, J. Vienne 1, K. Kandalla 1, S. Sur 1, D. K. Panda 1, and R. Brightwell 2 Presented By Xavier Besseron 1 Date:

More information

Can High-Performance Interconnects Benefit Hadoop Distributed File System?

Can High-Performance Interconnects Benefit Hadoop Distributed File System? Can High-Performance Interconnects Benefit Hadoop Distributed File System? Sayantan Sur, Hao Wang, Jian Huang, Xiangyong Ouyang and Dhabaleswar K. Panda Department of Computer Science and Engineering,

More information

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation

SMB Advanced Networking for Fault Tolerance and Performance. Jose Barreto Principal Program Managers Microsoft Corporation SMB Advanced Networking for Fault Tolerance and Performance Jose Barreto Principal Program Managers Microsoft Corporation Agenda SMB Remote File Storage for Server Apps SMB Direct (SMB over RDMA) SMB Multichannel

More information

Big Data With Hadoop

Big Data With Hadoop With Saurabh Singh singh.903@osu.edu The Ohio State University February 11, 2016 Overview 1 2 3 Requirements Ecosystem Resilient Distributed Datasets (RDDs) Example Code vs Mapreduce 4 5 Source: [Tutorials

More information

Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics

Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics Why Compromise? A discussion on RDMA versus Send/Receive and the difference between interconnect and application semantics Mellanox Technologies Inc. 2900 Stender Way, Santa Clara, CA 95054 Tel: 408-970-3400

More information