Data Center Specific Thermal and Energy Saving Techniques

Save this PDF as:

Size: px
Start display at page:

Download "Data Center Specific Thermal and Energy Saving Techniques"

Transcription

1 Data Center Specific Thermal and Energy Saving Techniques Tausif Muzaffar and Xiao Qin Department of Computer Science and Software Engineering Auburn University 1

2 Big Data 2

3 Data Centers In 2013, there are over 700 million square feet of data centers in united states Data centers account for 1.2% of all data power consumed in United States 3

4 Part 1 THERMAL MODEL 4

5 Data Center Power Usage 15% 15% 25% 45% Server Cooling Utilities Network 5

6 Thermal Recirculation Server i+1 Server i Server i-1 CRAC Workload 6

7 Thermal Recirculation Management Sensor Monitoring Thermal Simulations Thermal Model 7

8 Prior Thermal Models Some are based on power rather than workload Ignore I/O heavy applications Requires some sensor support Not easily ported to different platforms 8

9 Research Goal itad: making a simple and practical way to estimate the temperature of a data node based on CPU Utilization I/O Utilization Average Conditions of a Data Center 9

10 Our Focus To focus on each server separately and find the outlet temperature To estimate inlet temperature based on that outlet temperature Server Model Inlet Inlet Model Outlet T INIT Server i 11

11 Server Model Three factors affect the output temperature of a single node Inlet Temperature CPU Workload I/O Workload T out T INIT Server i 12 CPU Workload I/O Workload

12 Server Model Diagram Convective Heat Transfer T INIT T out Radiant Heat Server i Transfer 13 Workload

13 Server Model Equations (1) Convective Heat Transfer of Server (2) Radiant Heat Transfer of Server (3) Change in temperature 15

14 Server Model Equations (4) Set Radiant and Convection equal to each other and solve for Tout 16

15 Workload Model To assess how the CPU and I/O effect workload CPU Workload CPU Components I/O Workload I/O Components 17

16 Inlet Model After the first run we need to update the inlet temperature to do that we developed this model T s T INIT T in T out 18

17 Determining Parameters To implement this model we need to get the following constants Maximum I/O and CPU can affect the outlet temperature Z which is a collection of constants 19

18 Gathering Values We thermometers to gather inlet and outlet temperatures We used infrared thermometers to get the surface temperature 20

19 Test Machines 21

20 Data Capture We gathered surface temperature and stored the values like so CPU: 0-5% 100% 0-10% I/O: 0-5% 0-8% 100% Avg: ,

21 Determining Constants We observed the rate in changed with CPU and I/O We used the values to calculate Z 23

22 Verification After getting the constants we ran a live test where we had a computer run tasks and we measured actual outlet temperatures vs. model outlet temperature 24

23 Implementations MPI using itad to decisions 25

25 Part 2 HADOOP DISK ENERGY EFFICIENCY

26 Disk Energy Disk drives varies in energy Disks can be a significant part of a server 29

27 Single-Disk Server Power Usage Power Usage 30% 10% 33% 5% CPU Networking Power Supply Disks DRAM 22% 30

28 Scaling Server Disk # With every added disk, hard drive energy plays a bigger role 31

29 Disk Dynamic Power Disks tend to have different consumption modes Active Idle Standby 32

30 Hadoop Overview Parallel Processing Map Reduce Distributed Data 33

31 Hadoop Benefits Industry Standard Large Research Community I/O Heavy 34

32 Hadoop Architecture Hadoop creates multiple replicas Metadata is managed on name node Nodes can have multiple disks 35

33 Research Goal NAP E(N)ergy (A)ware Disks for Hadoo(P) Built for high energy efficiency Designed for Hadoop clusters 36

34 Setup 3-node cluster Each node identical 4 disks 4gb RAM Cloudera Hadoop Power meter 37

35 Optimizations We group disks together I/O Limits More time for disks to sleep 38

36 Naïve (Reactive) Algorithm Simply turn off all drive until needed 39

37 Proactive Algorithm Turn on next drive before its needed Threshold 40

38 Comparing the Algorithms Reactive Predictive 41

39 Speed Reactive does worse than proactive Time increase low 42

40 Block Size Effects how HDFS stores files Effects how fast it processes 43

41 File Size Effects how blocks are made Effect data locality 44

42 Map vs. Reduce Map is more I/O intensive usually Reduce was usually shorter 45

43 Map Heavy vs. Reduce Heavy Map Heavy is more I/O intensive Map and Reduce Heavy gets no gain 46

44 PRE-BUD Model Prefetching Energy-Efficient Parallel I/O Systems with buffer Disk 47

45 NAP Energy Model Find added energy by disks Group can either be standby or active Read and writes assumed same 48

46 Energy Saving Simulation 49

47 Summary itad: a simple and practical way to estimate the temperature of a data node NAP: an energy-saving technique for disks in Hadoop clusters 50

48 Questions 51

Energy-aware job scheduler for highperformance

Energy-aware job scheduler for highperformance computing 7.9.2011 Olli Mämmelä (VTT), Mikko Majanen (VTT), Robert Basmadjian (University of Passau), Hermann De Meer (University of Passau), André Giesler

IT Cost Savings Audit. Information Technology Report. Prepared for XYZ Inc. November 2009

IT Cost Savings Audit Information Technology Report Prepared for XYZ Inc. November 2009 Executive Summary This information technology report presents the technology audit resulting from the activities

Energy Efficient MapReduce

Energy Efficient MapReduce Motivation: Energy consumption is an important aspect of datacenters efficiency, the total power consumption in the united states has doubled from 2000 to 2005, representing

Effect of Rack Server Population on Temperatures in Data Centers

Effect of Rack Server Population on Temperatures in Data Centers Rajat Ghosh, Vikneshan Sundaralingam, Yogendra Joshi G.W. Woodruff School of Mechanical Engineering Georgia Institute of Technology, Atlanta,

http://bradhedlund.com/?p=3108 This article is Part 1 in series that will take a closer look at the architecture and methods of a Hadoop cluster, and how it relates to the network and server infrastructure.

Heterogeneous Workload Consolidation for Efficient Management of Data Centers in Cloud Computing

Heterogeneous Workload Consolidation for Efficient Management of Data Centers in Cloud Computing Deep Mann ME (Software Engineering) Computer Science and Engineering Department Thapar University Patiala-147004

Green HPC - Dynamic Power Management in HPC

Gr eenhpc Dynami cpower Management i nhpc AT ECHNOL OGYWHI T EP APER Green HPC Dynamic Power Management in HPC 2 Green HPC - Dynamic Power Management in HPC Introduction... 3 Green Strategies... 4 Implementation...

Big Fast Data Hadoop acceleration with Flash. June 2013

Big Fast Data Hadoop acceleration with Flash June 2013 Agenda The Big Data Problem What is Hadoop Hadoop and Flash The Nytro Solution Test Results The Big Data Problem Big Data Output Facebook Traditional

HADOOP AT NOKIA JOSH DEVINS, NOKIA HADOOP MEETUP, JANUARY 2011 BERLIN

HADOOP AT NOKIA JOSH DEVINS, NOKIA HADOOP MEETUP, JANUARY 2011 BERLIN Two parts: * technical setup * applications before starting Question: Hadoop experience levels from none to some to lots, and what

Performance Analysis of Mixed Distributed Filesystem Workloads

Performance Analysis of Mixed Distributed Filesystem Workloads Esteban Molina-Estolano, Maya Gokhale, Carlos Maltzahn, John May, John Bent, Scott Brandt Motivation Hadoop-tailored filesystems (e.g. CloudStore)

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

Big Data - Infrastructure Considerations

April 2014, HAPPIEST MINDS TECHNOLOGIES Big Data - Infrastructure Considerations Author Anand Veeramani / Deepak Shivamurthy SHARING. MINDFUL. INTEGRITY. LEARNING. EXCELLENCE. SOCIAL RESPONSIBILITY. Copyright

A Cloud Test Bed for China Railway Enterprise Data Center

A Cloud Test Bed for China Railway Enterprise Data Center BACKGROUND China Railway consists of eighteen regional bureaus, geographically distributed across China, with each regional bureau having their

ENERGY EFFICIENT VIRTUAL MACHINE ASSIGNMENT BASED ON ENERGY CONSUMPTION AND RESOURCE UTILIZATION IN CLOUD NETWORK

International Journal of Computer Engineering & Technology (IJCET) Volume 7, Issue 1, Jan-Feb 2016, pp. 45-53, Article ID: IJCET_07_01_006 Available online at http://www.iaeme.com/ijcet/issues.asp?jtype=ijcet&vtype=7&itype=1

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information

Deployment Planning Guide

Deployment Planning Guide Community 1.5.0 release The purpose of this document is to educate the user about the different strategies that can be adopted to optimize the usage of Jumbune on Hadoop and also

Introduction to IBM tools to manage energy consumption

Journée Thématique Emergente EDF Clamart, 13 janvier 2011 Les aspects énergétiques du calcul Introduction to IBM tools to manage energy consumption François Thomas, Luigi Brochard [ft,luigi.brochard]@fr.ibm.com

How High Temperature Data Centers & Intel Technologies save Energy, Money, Water and Greenhouse Gas Emissions

Intel Intelligent Power Management Intel How High Temperature Data Centers & Intel Technologies save Energy, Money, Water and Greenhouse Gas Emissions Power and cooling savings through the use of Intel

Data Warehousing and Analytics Infrastructure at Facebook. Ashish Thusoo & Dhruba Borthakur athusoo,dhruba@facebook.com

Data Warehousing and Analytics Infrastructure at Facebook Ashish Thusoo & Dhruba Borthakur athusoo,dhruba@facebook.com Overview Challenges in a Fast Growing & Dynamic Environment Data Flow Architecture,

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

Maximizing Hadoop Performance and Storage Capacity with AltraHD TM

Maximizing Hadoop Performance and Storage Capacity with AltraHD TM Executive Summary The explosion of internet data, driven in large part by the growth of more and more powerful mobile devices, has created

Performance and Energy Efficiency of. Hadoop deployment models

Performance and Energy Efficiency of Hadoop deployment models Contents Review: What is MapReduce Review: What is Hadoop Hadoop Deployment Models Metrics Experiment Results Summary MapReduce Introduced

Benchmarking Hadoop & HBase on Violin

Technical White Paper Report Technical Report Benchmarking Hadoop & HBase on Violin Harnessing Big Data Analytics at the Speed of Memory Version 1.0 Abstract The purpose of benchmarking is to show advantages

A Novel Cloud Based Elastic Framework for Big Data Preprocessing

School of Systems Engineering A Novel Cloud Based Elastic Framework for Big Data Preprocessing Omer Dawelbeit and Rachel McCrindle October 21, 2014 University of Reading 2008 www.reading.ac.uk Overview

Firebird meets NoSQL (Apache HBase) Case Study

Firebird meets NoSQL (Apache HBase) Case Study Firebird Conference 2011 Luxembourg 25.11.2011 26.11.2011 Thomas Steinmaurer DI +43 7236 3343 896 thomas.steinmaurer@scch.at www.scch.at Michael Zwick DI

Dynamic Resource allocation in Cloud

Dynamic Resource allocation in Cloud ABSTRACT: Cloud computing allows business customers to scale up and down their resource usage based on needs. Many of the touted gains in the cloud model come from

Performance And Scalability In Oracle9i And SQL Server 2000

Performance And Scalability In Oracle9i And SQL Server 2000 Presented By : Phathisile Sibanda Supervisor : John Ebden 1 Presentation Overview Project Objectives Motivation -Why performance & Scalability

Unprecedented Performance and Scalability Demonstrated For Meter Data Management:

Unprecedented Performance and Scalability Demonstrated For Meter Data Management: Ten Million Meters Scalable to One Hundred Million Meters For Five Billion Daily Meter Readings Performance testing results

A Performance Analysis of Distributed Indexing using Terrier

A Performance Analysis of Distributed Indexing using Terrier Amaury Couste Jakub Kozłowski William Martin Indexing Indexing Used by search

ICRI-CI Retreat Architecture track

ICRI-CI Retreat Architecture track Uri Weiser June 5 th 2015 - Funnel: Memory Traffic Reduction for Big Data & Machine Learning (Uri) - Accelerators for Big Data & Machine Learning (Ran) - Machine Learning

PARALLELS CLOUD STORAGE

PARALLELS CLOUD STORAGE Performance Benchmark Results 1 Table of Contents Executive Summary... Error! Bookmark not defined. Architecture Overview... 3 Key Features... 5 No Special Hardware Requirements...

Application of Predictive Analytics for Better Alignment of Business and IT

Application of Predictive Analytics for Better Alignment of Business and IT Boris Zibitsker, PhD bzibitsker@beznext.com July 25, 2014 Big Data Summit - Riga, Latvia About the Presenter Boris Zibitsker

Will They Blend?: Exploring Big Data Computation atop Traditional HPC NAS Storage

Will They Blend?: Exploring Big Data Computation atop Traditional HPC NAS Storage Ellis H. Wilson III 1,2 Mahmut Kandemir 1 Garth Gibson 2,3 1 Department of Computer Science and Engineering, The Pennsylvania

IMPROVED FAIR SCHEDULING ALGORITHM FOR TASKTRACKER IN HADOOP MAP-REDUCE

IMPROVED FAIR SCHEDULING ALGORITHM FOR TASKTRACKER IN HADOOP MAP-REDUCE Mr. Santhosh S 1, Mr. Hemanth Kumar G 2 1 PG Scholor, 2 Asst. Professor, Dept. Of Computer Science & Engg, NMAMIT, (India) ABSTRACT

Mixing Hadoop and HPC Workloads on Parallel Filesystems

Mixing Hadoop and HPC Workloads on Parallel Filesystems Esteban Molina-Estolano *, Maya Gokhale, Carlos Maltzahn *, John May, John Bent, Scott Brandt * * UC Santa Cruz, ISSDM, PDSI Lawrence Livermore National

News and trends in Data Warehouse Automation, Big Data and BI. Johan Hendrickx & Dirk Vermeiren

News and trends in Data Warehouse Automation, Big Data and BI Johan Hendrickx & Dirk Vermeiren Extreme Agility from Source to Analysis DWH Appliances & DWH Automation Typical Architecture 3 What Business

Importance of Data locality

Importance of Data Locality - Gerald Abstract Scheduling Policies Test Applications Evaluation metrics Tests in Hadoop Test environment Tests Observations Job run time vs. Mmax Job run time vs. number

Hadoop EKG: Using Heartbeats to Propagate Resource Utilization Data

Hadoop EKG: Using Heartbeats to Propagate Resource Utilization Data Trevor G. Reid Duke University tgr3@duke.edu Jian Wei Gan Duke University jg76@duke.edu Abstract Hadoop EKG is a modification to the

Hadoop Distributed File System. Jordan Prosch, Matt Kipps

Hadoop Distributed File System Jordan Prosch, Matt Kipps Outline - Background - Architecture - Comments & Suggestions Background What is HDFS? Part of Apache Hadoop - distributed storage What is Hadoop?

GraySort on Apache Spark by Databricks

GraySort on Apache Spark by Databricks Reynold Xin, Parviz Deyhim, Ali Ghodsi, Xiangrui Meng, Matei Zaharia Databricks Inc. Apache Spark Sorting in Spark Overview Sorting Within a Partition Range Partitioner

International Journal of Computer & Organization Trends Volume20 Number1 May 2015

Performance Analysis of Various Guest Operating Systems on Ubuntu 14.04 Prof. (Dr.) Viabhakar Pathak 1, Pramod Kumar Ram 2 1 Computer Science and Engineering, Arya College of Engineering, Jaipur, India.

The Hadoop Distributed File System

The Hadoop Distributed File System The Hadoop Distributed File System, Konstantin Shvachko, Hairong Kuang, Sanjay Radia, Robert Chansler, Yahoo, 2010 Agenda Topic 1: Introduction Topic 2: Architecture

Measuring Power in your Data Center: The Roadmap to your PUE and Carbon Footprint

Measuring Power in your Data Center: The Roadmap to your PUE and Carbon Footprint Energy usage in the data center Electricity Transformer/ UPS Air 10% Movement 12% Cooling 25% Lighting, etc. 3% IT Equipment

The CEETHERM Data Center Laboratory

The CEETHERM Data Center Laboratory A Platform for Transformative Research on Green Data Centers Yogendra Joshi and Pramod Kumar G.W. Woodruff School of Mechanical Engineering Georgia Institute of Technology

Energy Constrained Resource Scheduling for Cloud Environment

Energy Constrained Resource Scheduling for Cloud Environment 1 R.Selvi, 2 S.Russia, 3 V.K.Anitha 1 2 nd Year M.E.(Software Engineering), 2 Assistant Professor Department of IT KSR Institute for Engineering

Maximizing Hadoop Performance with Hardware Compression

Maximizing Hadoop Performance with Hardware Compression Robert Reiner Director of Marketing Compression and Security Exar Corporation November 2012 1 What is Big? sets whose size is beyond the ability

Big Data Analysis and Its Scheduling Policy Hadoop

IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 1, Ver. IV (Jan Feb. 2015), PP 36-40 www.iosrjournals.org Big Data Analysis and Its Scheduling Policy

Journal of science STUDY ON REPLICA MANAGEMENT AND HIGH AVAILABILITY IN HADOOP DISTRIBUTED FILE SYSTEM (HDFS)

Journal of science e ISSN 2277-3290 Print ISSN 2277-3282 Information Technology www.journalofscience.net STUDY ON REPLICA MANAGEMENT AND HIGH AVAILABILITY IN HADOOP DISTRIBUTED FILE SYSTEM (HDFS) S. Chandra

How High Temperature Data Centers & Intel Technologies save Energy, Money, Water and Greenhouse Gas Emissions

Intel Intelligent Power Management Intel How High Temperature Data Centers & Intel Technologies save Energy, Money, Water and Greenhouse Gas Emissions Power savings through the use of Intel s intelligent

HiBench Introduction. Carson Wang (carson.wang@intel.com) Software & Services Group

HiBench Introduction Carson Wang (carson.wang@intel.com) Agenda Background Workloads Configurations Benchmark Report Tuning Guide Background WHY Why we need big data benchmarking systems? WHAT What is

Evaluating HDFS I/O Performance on Virtualized Systems

Evaluating HDFS I/O Performance on Virtualized Systems Xin Tang xtang@cs.wisc.edu University of Wisconsin-Madison Department of Computer Sciences Abstract Hadoop as a Service (HaaS) has received increasing

The Google File System

The Google File System By Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung (Presented at SOSP 2003) Introduction Google search engine. Applications process lots of data. Need good file system. Solution:

MAPREDUCE [1] is proposed by Google in 2004 and

IEEE TRANSACTIONS ON COMPUTERS 1 Improving MapReduce Performance Using Smart Speculative Execution Strategy Qi Chen, Cheng Liu, and Zhen Xiao, Senior Member, IEEE Abstract MapReduce is a widely used parallel

Cost-Efficient Job Scheduling Strategies

Cost-Efficient Job Scheduling Strategies Alexander Leib Arbeitsbereich Wissenschaftliches Rechnen Fachbereich Informatik Fakultät für Mathematik, Informatik und Naturwissenschaften Universität Hamburg

External Sorting. Why Sort? 2-Way Sort: Requires 3 Buffers. Chapter 13

External Sorting Chapter 13 Database Management Systems 3ed, R. Ramakrishnan and J. Gehrke 1 Why Sort? A classic problem in computer science! Data requested in sorted order e.g., find students in increasing

Power Efficiency Metrics for the Top500. Shoaib Kamil and John Shalf CRD/NERSC Lawrence Berkeley National Lab

Power Efficiency Metrics for the Top500 Shoaib Kamil and John Shalf CRD/NERSC Lawrence Berkeley National Lab Power for Single Processors HPC Concurrency on the Rise Total # of Processors in Top15 350000

Dynamic resource management for energy saving in the cloud computing environment

Dynamic resource management for energy saving in the cloud computing environment Liang-Teh Lee, Kang-Yuan Liu, and Hui-Yang Huang Department of Computer Science and Engineering, Tatung University, Taiwan

HPPR Data Center Improvements Projects Towards Energy Efficiency (CFD Analysis, Monitoring, etc.)

HPPR Data Center Improvements Projects Towards Energy Efficiency (CFD Analysis, Monitoring, etc.) Felipe Visbal / March 27, 2013 HPPR Data Center Technologies Team Services Data Centers: Thermal Assessments

Thermal Aware Scheduling in Hadoop MapReduce Framework. Sayan Kole

Thermal Aware Scheduling in Hadoop MapReduce Framework by Sayan Kole A Thesis Presented in Partial Fulfillment of the Requirement for the Degree Master of Science Approved November 2013 by the Graduate

Hadoop Overview Data analytics has become a key element of the business decision process over the last decade. Classic reporting on a dataset stored in a database was sufficient until recently, but yesterday

Dell Reference Configuration for Hortonworks Data Platform

Dell Reference Configuration for Hortonworks Data Platform A Quick Reference Configuration Guide Armando Acosta Hadoop Product Manager Dell Revolutionary Cloud and Big Data Group Kris Applegate Solution

Data-intensive HPC: opportunities and challenges. Patrick Valduriez

Data-intensive HPC: opportunities and challenges Patrick Valduriez Big Data Landscape Multi-\$billion market! Big data = Hadoop = MapReduce? No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard,

HADOOP ON ORACLE ZFS STORAGE A TECHNICAL OVERVIEW

HADOOP ON ORACLE ZFS STORAGE A TECHNICAL OVERVIEW 757 Maleta Lane, Suite 201 Castle Rock, CO 80108 Brett Weninger, Managing Director brett.weninger@adurant.com Dave Smelker, Managing Principal dave.smelker@adurant.com

Cloud Storage. Parallels. Performance Benchmark Results. White Paper. www.parallels.com

Parallels Cloud Storage White Paper Performance Benchmark Results www.parallels.com Table of Contents Executive Summary... 3 Architecture Overview... 3 Key Features... 4 No Special Hardware Requirements...

Efficient Data Replication Scheme based on Hadoop Distributed File System

, pp. 177-186 http://dx.doi.org/10.14257/ijseia.2015.9.12.16 Efficient Data Replication Scheme based on Hadoop Distributed File System Jungha Lee 1, Jaehwa Chung 2 and Daewon Lee 3* 1 Division of Supercomputing,

An Oracle White Paper June 2012. High Performance Connectors for Load and Access of Data from Hadoop to Oracle Database

An Oracle White Paper June 2012 High Performance Connectors for Load and Access of Data from Hadoop to Oracle Database Executive Overview... 1 Introduction... 1 Oracle Loader for Hadoop... 2 Oracle Direct

Thermal Management of Datacenter

Thermal Management of Datacenter Qinghui Tang 1 Preliminaries What is data center What is thermal management Why does Intel Care Why Computer Science 2 Typical layout of a datacenter Rack outlet temperature

SoSe 2014 Dozenten: Prof. Dr. Thomas Ludwig, Dr. Manuel Dolz Vorgetragen von Hakob Aridzanjan 03.06.2014

Total Cost of Ownership in High Performance Computing HPC datacenter energy efficient techniques: job scheduling, best programming practices, energy-saving hardware/software mechanisms SoSe 2014 Dozenten:

Lecture 5: GFS & HDFS! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl

Big Data Processing, 2014/15 Lecture 5: GFS & HDFS!! Claudia Hauff (Web Information Systems)! ti2736b-ewi@tudelft.nl 1 Course content Introduction Data streams 1 & 2 The MapReduce paradigm Looking behind

IOmark Suite. Benchmarking Storage with Applica4on Workloads August, 2013. 2013 Evaluator Group, Inc.

IOmark Suite Benchmarking Storage with Applica4on Workloads August, 2013 1 What is IOmark Suite?! A storage specific benchmark for applicaaon workloads Tests storage only Supports VDI and Virtual Machine

DSS. Diskpool and cloud storage benchmarks used in IT-DSS. Data & Storage Services. Geoffray ADDE

DSS Data & Diskpool and cloud storage benchmarks used in IT-DSS CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/it Geoffray ADDE DSS Outline I- A rational approach to storage systems evaluation

Power Efficiency Comparison: Cisco UCS 5108 Blade Server Chassis and IBM FlexSystem Enterprise Chassis

White Paper Power Efficiency Comparison: Cisco UCS 5108 Blade Server Chassis and IBM FlexSystem Enterprise Chassis White Paper March 2014 2014 Cisco and/or its affiliates. All rights reserved. This document

HP reference configuration for entry-level SAS Grid Manager solutions

HP reference configuration for entry-level SAS Grid Manager solutions Up to 864 simultaneous SAS jobs and more than 3 GB/s I/O throughput Technical white paper Table of contents Executive summary... 2

A Study of Data Management Technology for Handling Big Data

Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 9, September 2014,

Hadoop Distributed File System (HDFS) Overview

2012 coreservlets.com and Dima May Hadoop Distributed File System (HDFS) Overview Originals of slides and source code for examples: http://www.coreservlets.com/hadoop-tutorial/ Also see the customized

Introducing EEMBC Cloud and Big Data Server Benchmarks

Introducing EEMBC Cloud and Big Data Server Benchmarks Quick Background: Industry-Standard Benchmarks for the Embedded Industry EEMBC formed in 1997 as non-profit consortium Defining and developing application-specific

Using CFD for optimal thermal management and cooling design in data centers

www.siemens.com/datacenters Using CFD for optimal thermal management and cooling design in data centers Introduction As the power density of IT equipment within a rack increases and energy costs rise,

Apache Hadoop. Alexandru Costan

1 Apache Hadoop Alexandru Costan Big Data Landscape No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard, except Hadoop 2 Outline What is Hadoop? Who uses it? Architecture HDFS MapReduce Open

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray ware 2 Agenda The Hadoop Journey Why Virtualize Hadoop? Elasticity and Scalability Performance Tests Storage Reference

Data Center Operating Cost Savings Realized by Air Flow Management and Increased Rack Inlet Temperatures

Data Center Operating Cost Savings Realized by Air Flow Management and Increased Rack Inlet Temperatures William Seeber Stephen Seeber Mid Atlantic Infrared Services, Inc. 5309 Mohican Road Bethesda, MD

Quantcast Petabyte Storage at Half Price with QFS!

9-131 Quantcast Petabyte Storage at Half Price with QFS Presented by Silvius Rus, Director, Big Data Platforms September 2013 Quantcast File System (QFS) A high performance alternative to the Hadoop Distributed

High Performance Computing in CST STUDIO SUITE

High Performance Computing in CST STUDIO SUITE Felix Wolfheimer GPU Computing Performance Speedup 18 16 14 12 10 8 6 4 2 0 Promo offer for EUC participants: 25% discount for K40 cards Speedup of Solver

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Alexandra Carpen-Amarie Diana Moise Bogdan Nicolae KerData Team, INRIA Outline

Shareability and Locality Aware Scheduling Algorithm in Hadoop for Mobile Cloud Computing

Shareability and Locality Aware Scheduling Algorithm in Hadoop for Mobile Cloud Computing Hsin-Wen Wei 1,2, Che-Wei Hsu 2, Tin-Yu Wu 3, Wei-Tsong Lee 1 1 Department of Electrical Engineering, Tamkang University

Hadoop and Map-Reduce. Swati Gore

Hadoop and Map-Reduce Swati Gore Contents Why Hadoop? Hadoop Overview Hadoop Architecture Working Description Fault Tolerance Limitations Why Map-Reduce not MPI Distributed sort Why Hadoop? Existing Data

Efficient Load Balancing using VM Migration by QEMU-KVM

International Journal of Computer Science and Telecommunications [Volume 5, Issue 8, August 2014] 49 ISSN 2047-3338 Efficient Load Balancing using VM Migration by QEMU-KVM Sharang Telkikar 1, Shreyas Talele

Characterizing Task Usage Shapes in Google s Compute Clusters

Characterizing Task Usage Shapes in Google s Compute Clusters Qi Zhang 1, Joseph L. Hellerstein 2, Raouf Boutaba 1 1 University of Waterloo, 2 Google Inc. Introduction Cloud computing is becoming a key

ENERGY STAR Program Requirements Product Specification for Computer Servers. Test Method Rev. Apr-2013

ENERGY STAR Program Requirements Product Specification for Computer Servers Test Method Rev. Apr-2013 1 OVERVIEW The following test method shall be used for determining compliance with requirements in

HADOOP MOCK TEST HADOOP MOCK TEST I

http://www.tutorialspoint.com HADOOP MOCK TEST Copyright tutorialspoint.com This section presents you various set of Mock Tests related to Hadoop Framework. You can download these sample mock tests at

A Dynamic Resource Management with Energy Saving Mechanism for Supporting Cloud Computing

A Dynamic Resource Management with Energy Saving Mechanism for Supporting Cloud Computing Liang-Teh Lee, Kang-Yuan Liu, Hui-Yang Huang and Chia-Ying Tseng Department of Computer Science and Engineering,

Benefits of Cold Aisle Containment During Cooling Failure

Benefits of Cold Aisle Containment During Cooling Failure Introduction Data centers are mission-critical facilities that require constant operation because they are at the core of the customer-business

Running a typical ROOT HEP analysis on Hadoop/MapReduce. Stefano Alberto Russo Michele Pinamonti Marina Cobal

Running a typical ROOT HEP analysis on Hadoop/MapReduce Stefano Alberto Russo Michele Pinamonti Marina Cobal CHEP 2013 Amsterdam 14-18/10/2013 Topics The Hadoop/MapReduce model Hadoop and High Energy Physics

Optimizing Dell PowerEdge Configurations for Hadoop

Optimizing Dell PowerEdge Configurations for Hadoop Understanding how to get the most out of Hadoop running on Dell hardware A Dell technical white paper July 2013 Michael Pittaro Principal Architect,

DATA MINING WITH HADOOP AND HIVE Introduction to Architecture

DATA MINING WITH HADOOP AND HIVE Introduction to Architecture Dr. Wlodek Zadrozny (Most slides come from Prof. Akella s class in 2014) 2015-2025. Reproduction or usage prohibited without permission of

The Comprehensive Performance Rating for Hadoop Clusters on Cloud Computing Platform

The Comprehensive Performance Rating for Hadoop Clusters on Cloud Computing Platform Fong-Hao Liu, Ya-Ruei Liou, Hsiang-Fu Lo, Ko-Chin Chang, and Wei-Tsong Lee Abstract Virtualization platform solutions

Enabling High performance Big Data platform with RDMA

Enabling High performance Big Data platform with RDMA Tong Liu HPC Advisory Council Oct 7 th, 2014 Shortcomings of Hadoop Administration tooling Performance Reliability SQL support Backup and recovery

Big Data Analytics Using R

October 23, 2014 Table of contents BIG DATA DEFINITION 1 BIG DATA DEFINITION Definition Characteristics Scaling Challange 2 Divide and Conquer Amdahl s and Gustafson s Law Life experience Where to parallelize?