Cloud Management: Knowing is Half The Battle

Size: px
Start display at page:

Download "Cloud Management: Knowing is Half The Battle"

Transcription

1 Cloud Management: Knowing is Half The Battle Raouf BOUTABA David R. Cheriton School of Computer Science University of Waterloo Joint work with Qi Zhang, Faten Zhani (University of Waterloo) and Joseph L. Hellerstein (Google Inc.) NOMS 2014, Krakow (Poland), May 5-9, 2014

2 Outline Introduction to Cloud computing The heterogeneity challenge Google Cluster Data Set Research Questions/Opportunities Dynamic Capacity Provisioning with Harmony Conclusions

3 The rise of Internet-scale Applications

4 Infrastructure/Data Scale Large scale infrastructure Google: 200+ clusters, hundreds of thousands machines Facebook: machines Yahoo: machines Huge volume of data (a.k.a. big data) Google: 20PB data per day (2008) Facebook: 36 PB of stored data, processing 80-90TB per day (2010) Yahoo: 170 PB data stored spread across the globe. Processing 3 PB per day (2010)

5 Cloud Computing A model designed for running large applications in a scalable and cost-efficient manner Harnessing massive resource capacities in the computing platforms, e.g. data centers Sharing resources among applications based on usage in an on-demand fashion Roles in a cloud computing environment Cloud providers (a.k.a. infrastructure providers) Service providers End users

6 Benefits of Cloud Computing Economical Cheap, commodity hardware Leveraging economies of scale Highly scalable Illusion of infinite resources on demand Start small, then scale resources up/down as needed Highly flexible Customizable CPU, memory, storage & networking capabilities Customizable software stack Easy access Access resources from any machine connected to the Internet Deploy applications from anywhere at anytime

7 Resource Management Resource management is a central activity of any cloud computing environment Service-level management Dynamic (i.e., on-demand) performance management and service provisioning Infrastructure-level management Monitoring Scheduling and resource allocation Fault detection and management Energy Management

8 The Heterogeneity Challenge Cloud resource management is difficult! A key reason: Both Cloud resources and applications are heterogeneous Machines have heterogeneous processing capacities and capabilities Different processor architecture, hardware features, processor speed, memory size and energy consumption model. Applications have heterogeneous sizes, durations, priorities and performance objectives

9 Outline Introduction to Cloud computing The heterogeneity challenge Google Cluster Data Set Research Questions/Opportunities Dynamic Capacity Provisioning with Harmony Conclusions

10 Google s Case Study Google s compute clusters execute millions of tasks on a daily basis Carrying out management activities requires an understanding of the performance impact of management activities Evaluating the performance of a new scheduling algorithm Capacity upgrade: what type of machines do we need? Current solution: sophisticated simulations High overhead Difficult to understand evaluation results Difficult to analyze what-if scenarios Characterizing the heterogeneity can improve resource management effectiveness and lower maintenance overhead

11 Google Data Set Workload traces collected from a production compute cluster in Google over 29 days ~ 12,000 machines ~ 2,012,242 jobs 25,462,157 tasks Applications are represented by jobs Each Job consists of one or more tasks 12 priorities divided into 3 priority groups Gratis (0-1): low priority batch jobs (e.g., MapReduce jobs) Other (2-8) : medium priority jobs (e.g., monitoring) Production (9-11) : high priority applications (e.g., user facing)

12 Machine Heterogeneity Histogram of machine capacities Machine availability over 24 hours Machines in production data centers often consist of multiple types E.g. multiple generations of machines purchased over time Machine failures are common in the compute cluster

13 Application Heterogeneity: Job Priority & Size Percentage of jobs per priority group CDF of Number of tasks per job Most of the jobs have low priority Almost 50% of the jobs consist of <10 tasks, but a few of them have more than 1000 tasks

14 Application Heterogeneity: Task Size Task size (Gratis) Task size (Other) Task size (Production) Tasks in production compute clusters are very heterogeneous in size

15 Task Duration and Scheduling Delay CDF of Task Duration CDF of Scheduling delay Most of the tasks are short (<10 min), a few tasks are really long More than 30% of the tasks are scheduled immediately, however other tasks can wait for days to be scheduled

16 Job Arrival Rate Arrival rate of jobs varies highly from time to time Inter-arrival time exhibits an on-off pattern according to the time of the day During day time the job arrival can be quite intense, as around 40% job inter-arrival time is less than 10s. At night time, job arrival intervals can be very long The task arrival rate can be very spiky Due to uneven distribution of both jobs size and arrival rate

17 Outline Introduction to Cloud computing The heterogeneity challenge Google Cluster Data Set Research Questions/Opportunities If Knowing is Half the Battle, What is the Other Half? Dynamic Capacity Provisioning with Harmony Conclusions

18 Research Questions/Opportunities Performance modeling for heterogonous workloads How to capture task and job performance characteristics (e.g. queuing delay, pre-emption rate) when both workload and machines are heterogeneous? Scheduling Algorithms for heterogeneous workloads How to design scheduling algorithms that consider workload and machine heterogeneity? MapReduce jobs and user facing jobs have completely different performance objectives, thus different scheduling policies should be used How can we take job performance objectives (e.g. deadlines for MapReduce jobs) into account when making scheduling decisions? Are there good bin-packing algorithms for task scheduling, given the distribution of task sizes? How to avoid frequent preemption of long running tasks?

19 Research Questions/Opportunities (cont) Optimizing workload performance and resource efficiency using migration Live migration is a well known technique for online workload management Reduce resource contention (e.g., network hot spots) Reduce resource fragmentation Minimize energy consumption (i.e., cost) How to use migration effectively given heterogeneous workload and machine characteristics? Energy management How to leverage machine heterogeneity and job arrival patterns to save energy, while meeting job performance objectives?

20 Outline Introduction to Cloud computing The heterogeneity challenge Google Cluster Data Set Research Questions/Opportunities Dynamic Capacity Provisioning with Harmony Conclusions

21 HARMONY: Dynamic Heterogeneity-Aware Capacity Provisioning Energy cost is an important concern in data centers Accounts for 12% of data center operational cost [Gartner Report 2010] Governments policies for building energy-efficient (i.e. Green ) ICT Minimize energy cost by turning off servers An idle server consumes as much as 60% of its peak energy demand

22 Resource Demand - Google s Data Set Fluctuation of resource demand in data centers creates opportunities for dynamically turning on and off servers CPU Demand over 30 days Memory Demand over 30 days Figure: Total resource demand in Google s Cluster Data Set

23 Important Factors To dynamically control data center capacity, one must consider the following factors: Heterogeneity of machines Heterogeneity of task size and duration Variability of task arrival rate Workload performance requirement Scheduling delay Cost of turning on and off servers Wear-tear effect Fluctuating energy prices

24 Solution Approach Classify tasks based on their size and duration using k-means clustering algorithm Capture the run-time workload composition in terms of arrival rate for each task class Predict the arrival rate of each type of tasks Define container as a logical allocation of resources to a task that belongs to a task class Use containers to reserve resources for each task class Using task arrival rate to estimate the number of required containers of each type of task

25 System Architecture

26 Optimization Optimal Capacity Provisioning can be formulated as the following integer program: Where: (Performance objective) (Energy cost) (Switching cost) Subject to constraints: (Machine state constraint) (Capacity constraint)

27 Optimization (cont) Optimal Capacity Provisioning is NP-hard We relax the integer program, then devise two solutions Container-Based Scheduling (CBS) Statically allocate containers in physical machines At run-time, schedule tasks into containers Container-Based Provisioning (CBP) Use the estimated number of containers to provision machines At run-time, schedule tasks using existing VM scheduling algorithms such as first-fit (FF)

28 Experiment Set Up Task classification Classify tasks based on size Categorize into short and long tasks Number of tasks (gratis) Task size (gratis) Task duration (gratis)

29 Experiments Set Up (cont) Machine energy consumption model Aggregated task arrival rates Number of required containers

30 Experiment Results Number of machines (the baseline) Number of machines (CBS and CBP) Comparison of Energy Consumption

31 Experiment Results (cont) Baseline CBP CBS

32 Outline Introduction to Cloud computing The heterogeneity challenge Google Cluster Data Set Research Questions/Opportunities Dynamic Capacity Provisioning with Harmony Conclusions

33 Take Away Message Cloud computing is becoming an integral part of today s IT infrastructure Heterogeneity is a major yet overlooked challenge for resource management in Cloud computing environments Machines have heterogeneous capacities and capabilities Applications have diverse resource characteristics, priority and performance objectives We have presented a characterization of workload found in production cloud environments. Traces can be dowloaded at: Many research opportunities exist for designing heterogeneityaware resource management schemes, with higher potential for practical impact.

34 Questions

Characterizing Task Usage Shapes in Google s Compute Clusters

Characterizing Task Usage Shapes in Google s Compute Clusters Characterizing Task Usage Shapes in Google s Compute Clusters Qi Zhang 1, Joseph L. Hellerstein 2, Raouf Boutaba 1 1 University of Waterloo, 2 Google Inc. Introduction Cloud computing is becoming a key

More information

Dynamic Workload Management in Heterogeneous Cloud Computing Environments

Dynamic Workload Management in Heterogeneous Cloud Computing Environments Dynamic Workload Management in Heterogeneous Cloud Computing Environments Qi Zhang and Raouf Boutaba University of Waterloo IEEE/IFIP Network Operations and Management Symposium Krakow, Poland May 7, 2014

More information

CQNCR: Optimal VM Migration Planning in Cloud Data Centers

CQNCR: Optimal VM Migration Planning in Cloud Data Centers CQNCR: Optimal VM Migration Planning in Cloud Data Centers Presented By Reaz Ahmed David R. Cheriton School of Computer science University of Waterloo Joint work with Md. Faizul Bari, Mohamed Faten Zhani,

More information

Characterizing Task Usage Shapes in Google s Compute Clusters

Characterizing Task Usage Shapes in Google s Compute Clusters Characterizing Task Usage Shapes in Google s Compute Clusters Qi Zhang University of Waterloo qzhang@uwaterloo.ca Joseph L. Hellerstein Google Inc. jlh@google.com Raouf Boutaba University of Waterloo rboutaba@uwaterloo.ca

More information

Dynamic Resource allocation in Cloud

Dynamic Resource allocation in Cloud Dynamic Resource allocation in Cloud ABSTRACT: Cloud computing allows business customers to scale up and down their resource usage based on needs. Many of the touted gains in the cloud model come from

More information

Relational Databases in the Cloud

Relational Databases in the Cloud Contact Information: February 2011 zimory scale White Paper Relational Databases in the Cloud Target audience CIO/CTOs/Architects with medium to large IT installations looking to reduce IT costs by creating

More information

Energy Efficient MapReduce

Energy Efficient MapReduce Energy Efficient MapReduce Motivation: Energy consumption is an important aspect of datacenters efficiency, the total power consumption in the united states has doubled from 2000 to 2005, representing

More information

International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 36 ISSN 2229-5518

International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 36 ISSN 2229-5518 International Journal of Scientific & Engineering Research, Volume 6, Issue 4, April-2015 36 An Efficient Approach for Load Balancing in Cloud Environment Balasundaram Ananthakrishnan Abstract Cloud computing

More information

Power Management in Cloud Computing using Green Algorithm. -Kushal Mehta COP 6087 University of Central Florida

Power Management in Cloud Computing using Green Algorithm. -Kushal Mehta COP 6087 University of Central Florida Power Management in Cloud Computing using Green Algorithm -Kushal Mehta COP 6087 University of Central Florida Motivation Global warming is the greatest environmental challenge today which is caused by

More information

Keywords Distributed Computing, On Demand Resources, Cloud Computing, Virtualization, Server Consolidation, Load Balancing

Keywords Distributed Computing, On Demand Resources, Cloud Computing, Virtualization, Server Consolidation, Load Balancing Volume 5, Issue 1, January 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Survey on Load

More information

ABSTRACT. KEYWORDS: Cloud Computing, Load Balancing, Scheduling Algorithms, FCFS, Group-Based Scheduling Algorithm

ABSTRACT. KEYWORDS: Cloud Computing, Load Balancing, Scheduling Algorithms, FCFS, Group-Based Scheduling Algorithm A REVIEW OF THE LOAD BALANCING TECHNIQUES AT CLOUD SERVER Kiran Bala, Sahil Vashist, Rajwinder Singh, Gagandeep Singh Department of Computer Science & Engineering, Chandigarh Engineering College, Landran(Pb),

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

CHAPTER 1 INTRODUCTION

CHAPTER 1 INTRODUCTION CHAPTER 1 INTRODUCTION 1.1 Background The command over cloud computing infrastructure is increasing with the growing demands of IT infrastructure during the changed business scenario of the 21 st Century.

More information

Analysis of Resource Allocation Approaches in Cloud Computing

Analysis of Resource Allocation Approaches in Cloud Computing Analysis of Resource Allocation Approaches in Cloud Computing Dr. N. Nagadeepa M.Sc., MCA, M.Phil., Ph.D., H.O.D, Department of Computer Science and Computer Applications, Jairams Arts and Sciemce College,

More information

Energy Constrained Resource Scheduling for Cloud Environment

Energy Constrained Resource Scheduling for Cloud Environment Energy Constrained Resource Scheduling for Cloud Environment 1 R.Selvi, 2 S.Russia, 3 V.K.Anitha 1 2 nd Year M.E.(Software Engineering), 2 Assistant Professor Department of IT KSR Institute for Engineering

More information

Introduction to Cloud Computing

Introduction to Cloud Computing Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services

More information

What Is It? Business Architecture Research Challenges Bibliography. Cloud Computing. Research Challenges Overview. Carlos Eduardo Moreira dos Santos

What Is It? Business Architecture Research Challenges Bibliography. Cloud Computing. Research Challenges Overview. Carlos Eduardo Moreira dos Santos Research Challenges Overview May 3, 2010 Table of Contents I 1 What Is It? Related Technologies Grid Computing Virtualization Utility Computing Autonomic Computing Is It New? Definition 2 Business Business

More information

Load Distribution in Large Scale Network Monitoring Infrastructures

Load Distribution in Large Scale Network Monitoring Infrastructures Load Distribution in Large Scale Network Monitoring Infrastructures Josep Sanjuàs-Cuxart, Pere Barlet-Ros, Gianluca Iannaccone, and Josep Solé-Pareta Universitat Politècnica de Catalunya (UPC) {jsanjuas,pbarlet,pareta}@ac.upc.edu

More information

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information

More information

Resource-Diversity Tolerant: Resource Allocation in the Cloud Infrastructure Services

Resource-Diversity Tolerant: Resource Allocation in the Cloud Infrastructure Services IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 5, Ver. III (Sep. Oct. 2015), PP 19-25 www.iosrjournals.org Resource-Diversity Tolerant: Resource Allocation

More information

Cloud, Community and Collaboration Airline benefits of using the Amadeus community cloud

Cloud, Community and Collaboration Airline benefits of using the Amadeus community cloud Cloud, Community and Collaboration Airline benefits of using the Amadeus community cloud Index Index... 2 Overview... 3 What is cloud computing?... 3 The benefit to businesses... 4 The downsides of public

More information

1. Simulation of load balancing in a cloud computing environment using OMNET

1. Simulation of load balancing in a cloud computing environment using OMNET Cloud Computing Cloud computing is a rapidly growing technology that allows users to share computer resources according to their need. It is expected that cloud computing will generate close to 13.8 million

More information

A Novel Cloud Based Elastic Framework for Big Data Preprocessing

A Novel Cloud Based Elastic Framework for Big Data Preprocessing School of Systems Engineering A Novel Cloud Based Elastic Framework for Big Data Preprocessing Omer Dawelbeit and Rachel McCrindle October 21, 2014 University of Reading 2008 www.reading.ac.uk Overview

More information

International Journal Of Engineering Research & Management Technology

International Journal Of Engineering Research & Management Technology International Journal Of Engineering Research & Management Technology March- 2014 Volume-1, Issue-2 PRIORITY BASED ENHANCED HTV DYNAMIC LOAD BALANCING ALGORITHM IN CLOUD COMPUTING Srishti Agarwal, Research

More information

Energy Conscious Virtual Machine Migration by Job Shop Scheduling Algorithm

Energy Conscious Virtual Machine Migration by Job Shop Scheduling Algorithm Energy Conscious Virtual Machine Migration by Job Shop Scheduling Algorithm Shanthipriya.M 1, S.T.Munusamy 2 ProfSrinivasan. R 3 M.Tech (IT) Student, Department of IT, PSV College of Engg & Tech, Krishnagiri,

More information

International Journal of Engineering Research & Management Technology

International Journal of Engineering Research & Management Technology International Journal of Engineering Research & Management Technology March- 2015 Volume 2, Issue-2 Survey paper on cloud computing with load balancing policy Anant Gaur, Kush Garg Department of CSE SRM

More information

Environments, Services and Network Management for Green Clouds

Environments, Services and Network Management for Green Clouds Environments, Services and Network Management for Green Clouds Carlos Becker Westphall Networks and Management Laboratory Federal University of Santa Catarina MARCH 3RD, REUNION ISLAND IARIA GLOBENET 2012

More information

A dynamic optimization model for power and performance management of virtualized clusters

A dynamic optimization model for power and performance management of virtualized clusters A dynamic optimization model for power and performance management of virtualized clusters Vinicius Petrucci, Orlando Loques Univ. Federal Fluminense Niteroi, Rio de Janeiro, Brasil Daniel Mossé Univ. of

More information

INCREASING SERVER UTILIZATION AND ACHIEVING GREEN COMPUTING IN CLOUD

INCREASING SERVER UTILIZATION AND ACHIEVING GREEN COMPUTING IN CLOUD INCREASING SERVER UTILIZATION AND ACHIEVING GREEN COMPUTING IN CLOUD M.Rajeswari 1, M.Savuri Raja 2, M.Suganthy 3 1 Master of Technology, Department of Computer Science & Engineering, Dr. S.J.S Paul Memorial

More information

Cost-effective Strategies for Building the Next-generation Data Center

Cost-effective Strategies for Building the Next-generation Data Center White Paper Cost-effective Strategies for Building the Next-generation Data Center Custom-made servers bearing energy-efficient processors are key to today s cloud computing-inspired architectures. Tom

More information

Migration of Virtual Machines for Better Performance in Cloud Computing Environment

Migration of Virtual Machines for Better Performance in Cloud Computing Environment Migration of Virtual Machines for Better Performance in Cloud Computing Environment J.Sreekanth 1, B.Santhosh Kumar 2 PG Scholar, Dept. of CSE, G Pulla Reddy Engineering College, Kurnool, Andhra Pradesh,

More information

Capacity Planning Fundamentals. Support Business Growth with a Better Approach to Scaling Your Data Center

Capacity Planning Fundamentals. Support Business Growth with a Better Approach to Scaling Your Data Center Capacity Planning Fundamentals Support Business Growth with a Better Approach to Scaling Your Data Center Executive Summary As organizations scale, planning for greater application workload demand is critical.

More information

Hadoop in the Hybrid Cloud

Hadoop in the Hybrid Cloud Presented by Hortonworks and Microsoft Introduction An increasing number of enterprises are either currently using or are planning to use cloud deployment models to expand their IT infrastructure. Big

More information

Load Balancing in cloud computing

Load Balancing in cloud computing Load Balancing in cloud computing 1 Foram F Kherani, 2 Prof.Jignesh Vania Department of computer engineering, Lok Jagruti Kendra Institute of Technology, India 1 kheraniforam@gmail.com, 2 jigumy@gmail.com

More information

Newsletter 4/2013 Oktober 2013. www.soug.ch

Newsletter 4/2013 Oktober 2013. www.soug.ch SWISS ORACLE US ER GRO UP www.soug.ch Newsletter 4/2013 Oktober 2013 Oracle 12c Consolidation Planer Data Redaction & Transparent Sensitive Data Protection Oracle Forms Migration Oracle 12c IDENTITY table

More information

An Oracle White Paper November 2010. Leveraging Massively Parallel Processing in an Oracle Environment for Big Data Analytics

An Oracle White Paper November 2010. Leveraging Massively Parallel Processing in an Oracle Environment for Big Data Analytics An Oracle White Paper November 2010 Leveraging Massively Parallel Processing in an Oracle Environment for Big Data Analytics 1 Introduction New applications such as web searches, recommendation engines,

More information

A Taxonomy and Survey of Energy-Efficient Data Centers and Cloud Computing Systems

A Taxonomy and Survey of Energy-Efficient Data Centers and Cloud Computing Systems A Taxonomy and Survey of Energy-Efficient Data Centers and Cloud Computing Systems Anton Beloglazov, Rajkumar Buyya, Young Choon Lee, and Albert Zomaya Present by Leping Wang 1/25/2012 Outline Background

More information

White Paper. How to Achieve Best-in-Class Performance Monitoring for Distributed Java Applications

White Paper. How to Achieve Best-in-Class Performance Monitoring for Distributed Java Applications White Paper How to Achieve Best-in-Class Performance Monitoring for Distributed Java Applications July / 2012 Introduction Critical Java business applications have been deployed for some time. However,

More information

Cost-effective Resource Provisioning for MapReduce in a Cloud

Cost-effective Resource Provisioning for MapReduce in a Cloud 1 -effective Resource Provisioning for MapReduce in a Cloud Balaji Palanisamy, Member, IEEE, Aameek Singh, Member, IEEE Ling Liu, Senior Member, IEEE Abstract This paper presents a new MapReduce cloud

More information

Efficient and Enhanced Load Balancing Algorithms in Cloud Computing

Efficient and Enhanced Load Balancing Algorithms in Cloud Computing , pp.9-14 http://dx.doi.org/10.14257/ijgdc.2015.8.2.02 Efficient and Enhanced Load Balancing Algorithms in Cloud Computing Prabhjot Kaur and Dr. Pankaj Deep Kaur M. Tech, CSE P.H.D prabhjotbhullar22@gmail.com,

More information

can you effectively plan for the migration and management of systems and applications on Vblock Platforms?

can you effectively plan for the migration and management of systems and applications on Vblock Platforms? SOLUTION BRIEF CA Capacity Management and Reporting Suite for Vblock Platforms can you effectively plan for the migration and management of systems and applications on Vblock Platforms? agility made possible

More information

The International Journal Of Science & Technoledge (ISSN 2321 919X) www.theijst.com

The International Journal Of Science & Technoledge (ISSN 2321 919X) www.theijst.com THE INTERNATIONAL JOURNAL OF SCIENCE & TECHNOLEDGE Efficient Parallel Processing on Public Cloud Servers using Load Balancing Manjunath K. C. M.Tech IV Sem, Department of CSE, SEA College of Engineering

More information

SCHEDULING IN CLOUD COMPUTING

SCHEDULING IN CLOUD COMPUTING SCHEDULING IN CLOUD COMPUTING Lipsa Tripathy, Rasmi Ranjan Patra CSA,CPGS,OUAT,Bhubaneswar,Odisha Abstract Cloud computing is an emerging technology. It process huge amount of data so scheduling mechanism

More information

CloudRank-D:A Benchmark Suite for Private Cloud Systems

CloudRank-D:A Benchmark Suite for Private Cloud Systems CloudRank-D:A Benchmark Suite for Private Cloud Systems Jing Quan Institute of Computing Technology, Chinese Academy of Sciences and University of Science and Technology of China HVC tutorial in conjunction

More information

Dynamic Resource Allocation in Software Defined and Virtual Networks: A Comparative Analysis

Dynamic Resource Allocation in Software Defined and Virtual Networks: A Comparative Analysis Dynamic Resource Allocation in Software Defined and Virtual Networks: A Comparative Analysis Felipe Augusto Nunes de Oliveira - GRR20112021 João Victor Tozatti Risso - GRR20120726 Abstract. The increasing

More information

159.735. Final Report. Cluster Scheduling. Submitted by: Priti Lohani 04244354

159.735. Final Report. Cluster Scheduling. Submitted by: Priti Lohani 04244354 159.735 Final Report Cluster Scheduling Submitted by: Priti Lohani 04244354 1 Table of contents: 159.735... 1 Final Report... 1 Cluster Scheduling... 1 Table of contents:... 2 1. Introduction:... 3 1.1

More information

Cloud Computing for Universities: A Prototype Suggestion and use of Cloud Computing in Academic Institutions

Cloud Computing for Universities: A Prototype Suggestion and use of Cloud Computing in Academic Institutions Cloud Computing for Universities: A Prototype Suggestion and use of Cloud Computing in Academic Institutions Sarvesh Kumar Computer Science & Engineering LPU, India Omkara Murthy Asst.Professor,CSE LPU,India

More information

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof.

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof. CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing University of Florida, CISE Department Prof. Daisy Zhe Wang Cloud Computing and Amazon Web Services Cloud Computing Amazon

More information

Efficient Service Broker Policy For Large-Scale Cloud Environments

Efficient Service Broker Policy For Large-Scale Cloud Environments www.ijcsi.org 85 Efficient Service Broker Policy For Large-Scale Cloud Environments Mohammed Radi Computer Science Department, Faculty of Applied Science Alaqsa University, Gaza Palestine Abstract Algorithms,

More information

The Total Cost of (Non) Ownership of Web Applications in the Cloud

The Total Cost of (Non) Ownership of Web Applications in the Cloud The Total Cost of (Non) Ownership of Web Applications in the Cloud Jinesh Varia August 2012 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 30 Abstract

More information

Networking in the Hadoop Cluster

Networking in the Hadoop Cluster Hadoop and other distributed systems are increasingly the solution of choice for next generation data volumes. A high capacity, any to any, easily manageable networking layer is critical for peak Hadoop

More information

Efficient Virtual Machine Sizing For Hosting Containers as a Service

Efficient Virtual Machine Sizing For Hosting Containers as a Service 1 Efficient Virtual Machine Sizing For Hosting Containers as a Service Sareh Fotuhi Piraghaj, Amir Vahid Dastjerdi, Rodrigo N. Calheiros, and Rajkumar Buyya Cloud Computing and Distributed Systems (CLOUDS)

More information

A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems

A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems Aysan Rasooli Department of Computing and Software McMaster University Hamilton, Canada Email: rasooa@mcmaster.ca Douglas G. Down

More information

Load balancing model for Cloud Data Center ABSTRACT:

Load balancing model for Cloud Data Center ABSTRACT: Load balancing model for Cloud Data Center ABSTRACT: Cloud data center management is a key problem due to the numerous and heterogeneous strategies that can be applied, ranging from the VM placement to

More information

Group Based Load Balancing Algorithm in Cloud Computing Virtualization

Group Based Load Balancing Algorithm in Cloud Computing Virtualization Group Based Load Balancing Algorithm in Cloud Computing Virtualization Rishi Bhardwaj, 2 Sangeeta Mittal, Student, 2 Assistant Professor, Department of Computer Science, Jaypee Institute of Information

More information

Cloud Computing. Walfredo Cirne

Cloud Computing. Walfredo Cirne Cloud Computing Walfredo Cirne Agenda What is the cloud? o Motivation o Cloud provisioning o SLO variation o IaaS, PaaS, SaaS Using the cloud o Dependency and locking o Pets x Cattle Trust & security What

More information

Fair Scheduling Algorithm with Dynamic Load Balancing Using In Grid Computing

Fair Scheduling Algorithm with Dynamic Load Balancing Using In Grid Computing Research Inventy: International Journal Of Engineering And Science Vol.2, Issue 10 (April 2013), Pp 53-57 Issn(e): 2278-4721, Issn(p):2319-6483, Www.Researchinventy.Com Fair Scheduling Algorithm with Dynamic

More information

A Novel Approach for Efficient Load Balancing in Cloud Computing Environment by Using Partitioning

A Novel Approach for Efficient Load Balancing in Cloud Computing Environment by Using Partitioning A Novel Approach for Efficient Load Balancing in Cloud Computing Environment by Using Partitioning 1 P. Vijay Kumar, 2 R. Suresh 1 M.Tech 2 nd Year, Department of CSE, CREC Tirupati, AP, India 2 Professor

More information

Coho Data s DataStream Clustered NAS System

Coho Data s DataStream Clustered NAS System Technology Insight Paper Coho Data s DataStream Clustered NAS System Bridging a Gap Between Webscale and Enterprise IT Storage By John Webster November, 2014 Enabling you to make the best technology decisions

More information

Data Centers and Cloud Computing

Data Centers and Cloud Computing Data Centers and Cloud Computing CS377 Guest Lecture Tian Guo 1 Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Case Study: Amazon EC2 2 Data Centers

More information

Cloud Computing, Virtualization & Green IT

Cloud Computing, Virtualization & Green IT Cloud Computing, Virtualization & Green IT Cloud computing can change how IT supports business Consider the following: As much as 85% of computing capacity sits idle in distributed computing environments.

More information

The Importance of Software License Server Monitoring

The Importance of Software License Server Monitoring The Importance of Software License Server Monitoring NetworkComputer How Shorter Running Jobs Can Help In Optimizing Your Resource Utilization White Paper Introduction Semiconductor companies typically

More information

ASCETiC Whitepaper. Motivation. ASCETiC Toolbox Business Goals. Approach

ASCETiC Whitepaper. Motivation. ASCETiC Toolbox Business Goals. Approach ASCETiC Whitepaper Motivation The increased usage of ICT, together with growing energy costs and the need to reduce greenhouse gases emissions call for energy-efficient technologies that decrease the overall

More information

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

More information

IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures

IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures IaaS Cloud Architectures: Virtualized Data Centers to Federated Cloud Infrastructures Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Introduction

More information

Cloud Computing. Chapter 8 Virtualization

Cloud Computing. Chapter 8 Virtualization Cloud Computing Chapter 8 Virtualization Learning Objectives Define and describe virtualization. Discuss the history of virtualization. Describe various types of virtualization. List the pros and cons

More information

Efficient Parallel Processing on Public Cloud Servers Using Load Balancing

Efficient Parallel Processing on Public Cloud Servers Using Load Balancing Efficient Parallel Processing on Public Cloud Servers Using Load Balancing Valluripalli Srinath 1, Sudheer Shetty 2 1 M.Tech IV Sem CSE, Sahyadri College of Engineering & Management, Mangalore. 2 Asso.

More information

Computing Load Aware and Long-View Load Balancing for Cluster Storage Systems

Computing Load Aware and Long-View Load Balancing for Cluster Storage Systems 215 IEEE International Conference on Big Data (Big Data) Computing Load Aware and Long-View Load Balancing for Cluster Storage Systems Guoxin Liu and Haiying Shen and Haoyu Wang Department of Electrical

More information

Steps to Migrating to a Private Cloud

Steps to Migrating to a Private Cloud Deploying and Managing Private Clouds The Essentials Series Steps to Migrating to a Private Cloud sponsored by Introduction to Realtime Publishers by Don Jones, Series Editor For several years now, Realtime

More information

LOAD BALANCING ALGORITHM REVIEW s IN CLOUD ENVIRONMENT

LOAD BALANCING ALGORITHM REVIEW s IN CLOUD ENVIRONMENT LOAD BALANCING ALGORITHM REVIEW s IN CLOUD ENVIRONMENT K.Karthika, K.Kanakambal, R.Balasubramaniam PG Scholar,Dept of Computer Science and Engineering, Kathir College Of Engineering/ Anna University, India

More information

A two-level scheduler to dynamically schedule a stream of batch jobs in large-scale grids

A two-level scheduler to dynamically schedule a stream of batch jobs in large-scale grids Managed by A two-level scheduler to dynamically schedule a stream of batch jobs in large-scale grids M. Pasquali, R. Baraglia, G. Capannini, L. Ricci, and D. Laforenza 7th Meeting of the Institute on Resource

More information

SPARC Enterprise s Approach to Virtualization and Its Contribution to ICT Society

SPARC Enterprise s Approach to Virtualization and Its Contribution to ICT Society SPARC Enterprise s Approach to Virtualization and Its Contribution to ICT Society Masaru Nukada Akio Satori In recent years, it has become common practice to implement virtualization technology across

More information

OPTIMIZING PERFORMANCE IN AMAZON EC2 INTRODUCTION: LEVERAGING THE PUBLIC CLOUD OPPORTUNITY WITH AMAZON EC2. www.boundary.com

OPTIMIZING PERFORMANCE IN AMAZON EC2 INTRODUCTION: LEVERAGING THE PUBLIC CLOUD OPPORTUNITY WITH AMAZON EC2. www.boundary.com OPTIMIZING PERFORMANCE IN AMAZON EC2 While the business decision to migrate to Amazon public cloud services can be an easy one, tracking and managing performance in these environments isn t so clear cut.

More information

IMPROVED FAIR SCHEDULING ALGORITHM FOR TASKTRACKER IN HADOOP MAP-REDUCE

IMPROVED FAIR SCHEDULING ALGORITHM FOR TASKTRACKER IN HADOOP MAP-REDUCE IMPROVED FAIR SCHEDULING ALGORITHM FOR TASKTRACKER IN HADOOP MAP-REDUCE Mr. Santhosh S 1, Mr. Hemanth Kumar G 2 1 PG Scholor, 2 Asst. Professor, Dept. Of Computer Science & Engg, NMAMIT, (India) ABSTRACT

More information

Chapter 2: Cloud Basics Chapter 3: Cloud Architecture

Chapter 2: Cloud Basics Chapter 3: Cloud Architecture Chapter 2: Cloud Basics Chapter 3: Cloud Architecture Service provider s job is supplying abstraction layer Users and developers are isolated from complexity of IT technology: Virtualization Service-oriented

More information

A Comparative Survey on Various Load Balancing Techniques in Cloud Computing

A Comparative Survey on Various Load Balancing Techniques in Cloud Computing 2015 IJSRSET Volume 1 Issue 6 Print ISSN : 2395-1990 Online ISSN : 2394-4099 Themed Section: Engineering and Technology A Comparative Survey on Various Load Balancing Techniques in Cloud Computing Patel

More information

International Journal of Emerging Technology & Research

International Journal of Emerging Technology & Research International Journal of Emerging Technology & Research A Study Based on the Survey of Optimized Dynamic Resource Allocation Techniques in Cloud Computing Sharvari J N 1, Jyothi S 2, Neetha Natesh 3 1,

More information

An Approach to Load Balancing In Cloud Computing

An Approach to Load Balancing In Cloud Computing An Approach to Load Balancing In Cloud Computing Radha Ramani Malladi Visiting Faculty, Martins Academy, Bangalore, India ABSTRACT: Cloud computing is a structured model that defines computing services,

More information

Using VMware VMotion with Oracle Database and EMC CLARiiON Storage Systems

Using VMware VMotion with Oracle Database and EMC CLARiiON Storage Systems Using VMware VMotion with Oracle Database and EMC CLARiiON Storage Systems Applied Technology Abstract By migrating VMware virtual machines from one physical environment to another, VMware VMotion can

More information

Thesis Proposal: Dynamic optimization of power and performance for virtualized server clusters. Vinicius Petrucci September 2010

Thesis Proposal: Dynamic optimization of power and performance for virtualized server clusters. Vinicius Petrucci September 2010 Thesis Proposal: Dynamic optimization of power and performance for virtualized server clusters Vinicius Petrucci September 2010 Supervisor: Orlando Loques (IC-UFF) Institute of Computing Fluminense Federal

More information

Web Applications Engineering: Performance Analysis: Operational Laws

Web Applications Engineering: Performance Analysis: Operational Laws Web Applications Engineering: Performance Analysis: Operational Laws Service Oriented Computing Group, CSE, UNSW Week 11 Material in these Lecture Notes is derived from: Performance by Design: Computer

More information

ADAPTIVE LOAD BALANCING ALGORITHM USING MODIFIED RESOURCE ALLOCATION STRATEGIES ON INFRASTRUCTURE AS A SERVICE CLOUD SYSTEMS

ADAPTIVE LOAD BALANCING ALGORITHM USING MODIFIED RESOURCE ALLOCATION STRATEGIES ON INFRASTRUCTURE AS A SERVICE CLOUD SYSTEMS ADAPTIVE LOAD BALANCING ALGORITHM USING MODIFIED RESOURCE ALLOCATION STRATEGIES ON INFRASTRUCTURE AS A SERVICE CLOUD SYSTEMS Lavanya M., Sahana V., Swathi Rekha K. and Vaithiyanathan V. School of Computing,

More information

On-Demand Virtual System Service

On-Demand Virtual System Service On-Demand System Service Yasutaka Taniuchi Cloud computing, which enables information and communications technology (ICT) capacity to be used over the network, is entering a genuine expansion phase for

More information

Datacenters and Cloud Computing. Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html

Datacenters and Cloud Computing. Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html Datacenters and Cloud Computing Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html What is Cloud Computing? A model for enabling ubiquitous, convenient, ondemand network

More information

CS 695 Topics in Virtualization and Cloud Computing and Storage Systems. Introduction

CS 695 Topics in Virtualization and Cloud Computing and Storage Systems. Introduction CS 695 Topics in Virtualization and Cloud Computing and Storage Systems Introduction Hot or not? source: Gartner Hype Cycle for Emerging Technologies, 2014 2 Source: http://geekandpoke.typepad.com/ 3 Cloud

More information

RESOURCE MANAGEMENT IN CLOUD COMPUTING ENVIRONMENT

RESOURCE MANAGEMENT IN CLOUD COMPUTING ENVIRONMENT RESOURCE MANAGEMENT IN CLOUD COMPUTING ENVIRONMENT A.Chermaraj 1, Dr.P.Marikkannu 2 1 PG Scholar, 2 Assistant Professor, Department of IT, Anna University Regional Centre Coimbatore, Tamilnadu (India)

More information

Comparison of PBRR Scheduling Algorithm with Round Robin and Heuristic Priority Scheduling Algorithm in Virtual Cloud Environment

Comparison of PBRR Scheduling Algorithm with Round Robin and Heuristic Priority Scheduling Algorithm in Virtual Cloud Environment www.ijcsi.org 99 Comparison of PBRR Scheduling Algorithm with Round Robin and Heuristic Priority Scheduling Algorithm in Cloud Environment Er. Navreet Singh 1 1 Asst. Professor, Computer Science Department

More information

Simplifying Storage Operations By David Strom (published 3.15 by VMware) Introduction

Simplifying Storage Operations By David Strom (published 3.15 by VMware) Introduction Simplifying Storage Operations By David Strom (published 3.15 by VMware) Introduction There are tectonic changes to storage technology that the IT industry hasn t seen for many years. Storage has been

More information

Cloud based Holdfast Electronic Sports Game Platform

Cloud based Holdfast Electronic Sports Game Platform Case Study Cloud based Holdfast Electronic Sports Game Platform Intel and Holdfast work together to upgrade Holdfast Electronic Sports Game Platform with cloud technology Background Shanghai Holdfast Online

More information

RevoScaleR Speed and Scalability

RevoScaleR Speed and Scalability EXECUTIVE WHITE PAPER RevoScaleR Speed and Scalability By Lee Edlefsen Ph.D., Chief Scientist, Revolution Analytics Abstract RevoScaleR, the Big Data predictive analytics library included with Revolution

More information

Cloud-Scale Datacenters

Cloud-Scale Datacenters Cloud-Scale Datacenters Cloud-Scale Datacenters Page 1 Cloud-Scale Datacenters Delivering services at cloud-scale requires a radically different approach to designing, building, deploying, and operating

More information

Setting deadlines and priorities to the tasks to improve energy efficiency in cloud computing

Setting deadlines and priorities to the tasks to improve energy efficiency in cloud computing Setting deadlines and priorities to the tasks to improve energy efficiency in cloud computing Problem description Cloud computing is a technology used more and more every day, requiring an important amount

More information

Grid Computing Approach for Dynamic Load Balancing

Grid Computing Approach for Dynamic Load Balancing International Journal of Computer Sciences and Engineering Open Access Review Paper Volume-4, Issue-1 E-ISSN: 2347-2693 Grid Computing Approach for Dynamic Load Balancing Kapil B. Morey 1*, Sachin B. Jadhav

More information

Rackspace Cloud Databases and Container-based Virtualization

Rackspace Cloud Databases and Container-based Virtualization Rackspace Cloud Databases and Container-based Virtualization August 2012 J.R. Arredondo @jrarredondo Page 1 of 6 INTRODUCTION When Rackspace set out to build the Cloud Databases product, we asked many

More information

CHAPTER 7 SUMMARY AND CONCLUSION

CHAPTER 7 SUMMARY AND CONCLUSION 179 CHAPTER 7 SUMMARY AND CONCLUSION This chapter summarizes our research achievements and conclude this thesis with discussions and interesting avenues for future exploration. The thesis describes a novel

More information

A Review on Load Balancing In Cloud Computing 1

A Review on Load Balancing In Cloud Computing 1 www.ijecs.in International Journal Of Engineering And Computer Science ISSN:2319-7242 Volume 4 Issue 6 June 2015, Page No. 12333-12339 A Review on Load Balancing In Cloud Computing 1 Peenaz Pathak, 2 Er.Kamna

More information

Enhancing the Scalability of Virtual Machines in Cloud

Enhancing the Scalability of Virtual Machines in Cloud Enhancing the Scalability of Virtual Machines in Cloud Chippy.A #1, Ashok Kumar.P #2, Deepak.S #3, Ananthi.S #4 # Department of Computer Science and Engineering, SNS College of Technology Coimbatore, Tamil

More information

Task Scheduling in Hadoop

Task Scheduling in Hadoop Task Scheduling in Hadoop Sagar Mamdapure Munira Ginwala Neha Papat SAE,Kondhwa SAE,Kondhwa SAE,Kondhwa Abstract Hadoop is widely used for storing large datasets and processing them efficiently under distributed

More information