A Batch Computing Service for the Spot Market. Supreeth Subramanya, Tian Guo, Prateek Sharma, David Irwin, Prashant Shenoy

Size: px
Start display at page:

Download "A Batch Computing Service for the Spot Market. Supreeth Subramanya, Tian Guo, Prateek Sharma, David Irwin, Prashant Shenoy"

Transcription

1 SpotOn: A Batch Computing Service for the Spot Market Supreeth Subramanya, Tian Guo, Prateek Sharma, David Irwin, Prashant Shenoy University of Massachusetts Amherst

2 infrastructure cloud Cost vs. Availability Tradeoff in IaaS Clouds On-demand Cost (per hour) Cheap Expensive Reserved Spot Spot Instances Preemptible VM Guaranteed, Non-revocable Not guaranteed, Non-revocable Not guaranteed, Revocable Availability

3 spot markets Amazon EC2 Spot User bid Bid in a 2nd price auction Acquire when bid > spot price Price Terminate when spot price > bid Time of the day How do we mitigate the impact of revocation? 1. Raise the bid 2. Employ fault-tolerance mechanisms mechanisms

4 challenges spot market complexity Amazon EC2 operates ~4000 spot markets Selecting an instance that yields lowest cost per unit of computation while also considering the probability of revocation is complex Scatterplot of ranks for EC2 spot markets

5 challenges application complexity CPU : IO 1:1 Working Set 8GB Disk Type Remote Running Time 1 hour Resource Vector Cost 20% On-demand Revocation Rate 2.4 per day Fault-tolerance Mechanism Checkpoint (every 900s) Spot VM Cost ( ) Completion time (s) 0 On-demand Checkpoint 0

6 challenges application complexity Resource Vector CPU : IO 1:1 Working Set 8GB Disk Type Remote Local Running Time 1 hour Spot VM Cost 20% On-demand Revocation Rate 2.4 per day Fault-tolerance Mechanism Checkpoint (every 900s) Replicate (deg=2) Cost ( ) Completion time (s) 0 On-demand Checkpoint Replicate 0

7 challenges application complexity CPU : IO 1:1 Working Set 8GB Disk Type Local Running Time 1 hour Resource Vector Spot VM Cost 20% On-demand Revocation Rate 2.4 per day >24 per day Fault-tolerance Mechanism Checkpoint (every 900s) Cost ( ) Completion time (s) 0 0 On-demand Checkpoint Replicate Replicate (Revoked)

8 Spoton: A batch computing service Service that accepts batch jobs from users and runs them on spot instances SpotOn Manages application and spot market complexity transparently Run batch jobs at on-demand performance but paying spot market price

9 greedy selection algorithm Selecting the best spot market and fault-tolerance mechanism Migrate Dup Chkp Spot Markets Fault Tolerance S i F t J b $ Batch Job Minimum cost of running J b using F t on S i Repeat on spot revocation (until J b finishes) Acquire S i

10 fault tolerance (1/3) Reactive Migration Migrated T M size of memory + local disk remote disk bandwidth Z k Random variable measuring time to revocation P k Probability job gets revoked before completion Cost = E[Price k ] = [(1 Pk) * T + Pk * (E(Zk) + T M )] * spot-price E[Time k ] (1 Pk) * T + Pk * E(Zk)

11 fault tolerance (2/3) Proactive Checkpoint Restored T C size of memory + local disk remote disk bandwidth T L (τ/2) τ: checkpoint frequency Total Overhead = (T /τ) * T c + T L Cost overhead is primarily a function of job s resource usage

12 fault tolerance (3/3) Spot Replication T L market volatility Total Overhead (Replication factor, T L ) Cost overhead is primarily a function of market characteristics

13 evaluation (1/3) SpotOn Prototype App emulator Built on Linux Containers for efficient checkpointing / migration App Emulator to create synthetic jobs with varying resource usage SpotOn job manager daemon daemon

14 evaluation (2/3) Effect of Application and Spot Market Characteristics Best choice of fault-tolerance mechanism is a function of the spot market and job characteristics

15 evaluation (3/3) Effect of Cost-aware Selection 2x better than just checkpointing On Google cluster trace, Cost-aware selection achieved 91.9% savings with little impact on performance

16 conclusion Spot markets offer arbitrage opportunities SpotOn manages application and market complexities We model fault tolerance and propose a selection algorithm Prototype on Amazon EC2 Achieves ~90% cost savings Thank you!

17 backup slides

18 bidding Spot price distribution has long tail Spot price changes are peaky Bidding does affect volatility but not drastically SpotOn always bids at the on-demand price If spot price goes above on-demand, SpotOn would choose on-demand

SpotCheck: Designing a Derivative IaaS Cloud on the Spot Market

SpotCheck: Designing a Derivative IaaS Cloud on the Spot Market SpotCheck: Designing a Derivative IaaS Cloud on the Spot Market Prateek Sharma Stephen Lee Tian Guo David Irwin Prashant Shenoy University of Massachusetts Amherst {prateeks,stephenlee,tian,shenoy}@cs.umass.edu

More information

Data Centers and Cloud Computing

Data Centers and Cloud Computing Data Centers and Cloud Computing CS377 Guest Lecture Tian Guo 1 Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Case Study: Amazon EC2 2 Data Centers

More information

Improving MapReduce Performance in Heterogeneous Environments

Improving MapReduce Performance in Heterogeneous Environments UC Berkeley Improving MapReduce Performance in Heterogeneous Environments Matei Zaharia, Andy Konwinski, Anthony Joseph, Randy Katz, Ion Stoica University of California at Berkeley Motivation 1. MapReduce

More information

Cutting the Cost of Hosting Online Services Using Cloud Spot Markets

Cutting the Cost of Hosting Online Services Using Cloud Spot Markets Cutting the Cost of Hosting Online Services Using Cloud Spot Markets Xin He, Prashant Shenoy, Ramesh Sitaraman and David Irwin University of Massachusetts Amherst {xhe, shenoy, ramesh}@cs.umass.edu, irwin@ecs.umass.edu

More information

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2 DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing Slide 1 Slide 3 A style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet.

More information

Amazon CloudWatch to monitor cloud resource usage

Amazon CloudWatch to monitor cloud resource usage DiCloud Deliverable 2f May 15, 2010 Amazon CloudWatch to monitor cloud resource usage Michael Zink, Prashant Shenoy, David Irwin and Emmanuel Cecchet {zink, shenoy, irwin, cecchet}@cs.umass.edu University

More information

Black-box and Gray-box Strategies for Virtual Machine Migration

Black-box and Gray-box Strategies for Virtual Machine Migration Black-box and Gray-box Strategies for Virtual Machine Migration Wood, et al (UMass), NSDI07 Context: Virtual Machine Migration 1 Introduction Want agility in server farms to reallocate resources devoted

More information

Cloud Computing and Amazon Web Services

Cloud Computing and Amazon Web Services Cloud Computing and Amazon Web Services Gary A. McGilvary edinburgh data.intensive research 1 OUTLINE 1. An Overview of Cloud Computing 2. Amazon Web Services 3. Amazon EC2 Tutorial 4. Conclusions 2 CLOUD

More information

Chapter 19 Cloud Computing for Multimedia Services

Chapter 19 Cloud Computing for Multimedia Services Chapter 19 Cloud Computing for Multimedia Services 19.1 Cloud Computing Overview 19.2 Multimedia Cloud Computing 19.3 Cloud-Assisted Media Sharing 19.4 Computation Offloading for Multimedia Services 19.5

More information

A Very Brief Introduction To Cloud Computing. Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman

A Very Brief Introduction To Cloud Computing. Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman A Very Brief Introduction To Cloud Computing Jens Vöckler, Gideon Juve, Ewa Deelman, G. Bruce Berriman What is The Cloud Cloud computing refers to logical computational resources accessible via a computer

More information

ADAPTIVE LIVE VM MIGRATION OVER A WAN MODELING AND IMPLEMENTATION

ADAPTIVE LIVE VM MIGRATION OVER A WAN MODELING AND IMPLEMENTATION ADAPTIVE LIVE VM MIGRATION OVER A WAN MODELING AND IMPLEMENTATION Weida Zhang, King Tin Lam, Cho-Li Wang Department of Computer Science, The University of Hong Kong Live Migration of VMs Live migration:

More information

Ensuring Collective Availability in Volatile Resource Pools via Forecasting

Ensuring Collective Availability in Volatile Resource Pools via Forecasting Ensuring Collective Availability in Volatile Resource Pools via Forecasting Artur Andrzejak andrzejak[at]zib.de Derrick Kondo David P. Anderson Zuse Institute Berlin (ZIB) INRIA UC Berkeley Motivation

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information

CHAPTER 8 CLOUD COMPUTING

CHAPTER 8 CLOUD COMPUTING CHAPTER 8 CLOUD COMPUTING SE 458 SERVICE ORIENTED ARCHITECTURE Assist. Prof. Dr. Volkan TUNALI Faculty of Engineering and Natural Sciences / Maltepe University Topics 2 Cloud Computing Essential Characteristics

More information

Disaster Recovery as a Cloud Service: Economic Benefits and Deployment Challenges

Disaster Recovery as a Cloud Service: Economic Benefits and Deployment Challenges Disaster Recovery as a Cloud Service: Economic Benefits and Deployment Challenges Tim Wood, Emmanuel Cecchet, KK Ramakrishnan*, Prashant Shenoy, Kobus van der Merwe*, and Arun Venkataramani UMass Amherst

More information

Google File System. Web and scalability

Google File System. Web and scalability Google File System Web and scalability The web: - How big is the Web right now? No one knows. - Number of pages that are crawled: o 100,000 pages in 1994 o 8 million pages in 2005 - Crawlable pages might

More information

Data Centers and Cloud Computing. Data Centers

Data Centers and Cloud Computing. Data Centers Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises

More information

Characterizing Task Usage Shapes in Google s Compute Clusters

Characterizing Task Usage Shapes in Google s Compute Clusters Characterizing Task Usage Shapes in Google s Compute Clusters Qi Zhang 1, Joseph L. Hellerstein 2, Raouf Boutaba 1 1 University of Waterloo, 2 Google Inc. Introduction Cloud computing is becoming a key

More information

Data Centers and Cloud Computing. Data Centers

Data Centers and Cloud Computing. Data Centers Data Centers and Cloud Computing Slides courtesy of Tim Wood 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises for server applications Internet

More information

Datacenters and Cloud Computing. Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html

Datacenters and Cloud Computing. Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html Datacenters and Cloud Computing Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html What is Cloud Computing? A model for enabling ubiquitous, convenient, ondemand network

More information

Ground up Introduction to In-Memory Data (Grids)

Ground up Introduction to In-Memory Data (Grids) Ground up Introduction to In-Memory Data (Grids) QCON 2015 NEW YORK, NY 2014 Hazelcast Inc. Why you here? 2014 Hazelcast Inc. Java Developer on a quest for scalability frameworks Architect on low-latency

More information

Part 1: Price Comparison Among The 10 Top Iaas Providers

Part 1: Price Comparison Among The 10 Top Iaas Providers Part 1: Price Comparison Among The 10 Top Iaas Providers Table of Contents Executive Summary 3 Estimating Cloud Spending 3 About the Pricing Report 3 Key Findings 3 The IaaS Providers 3 Provider Characteristics

More information

Multilevel Communication Aware Approach for Load Balancing

Multilevel Communication Aware Approach for Load Balancing Multilevel Communication Aware Approach for Load Balancing 1 Dipti Patel, 2 Ashil Patel Department of Information Technology, L.D. College of Engineering, Gujarat Technological University, Ahmedabad 1

More information

Data Centers and Cloud Computing. Data Centers. MGHPCC Data Center. Inside a Data Center

Data Centers and Cloud Computing. Data Centers. MGHPCC Data Center. Inside a Data Center Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises

More information

Who moved my cloud? Part II: What s behind the cloud vendors AWS, GCE and Azure?

Who moved my cloud? Part II: What s behind the cloud vendors AWS, GCE and Azure? Who moved my cloud? Part II: What s behind the cloud vendors AWS, GCE and Azure? Part II of an ebook series covering cloud infrastructure and platform fundamentals not to be missed when preparing to make

More information

Dynamic Resource Distribution Across Clouds

Dynamic Resource Distribution Across Clouds University of Victoria Faculty of Engineering Winter 2010 Work Term Report Dynamic Resource Distribution Across Clouds Department of Physics University of Victoria Victoria, BC Michael Paterson V00214440

More information

CloudFTP: A free Storage Cloud

CloudFTP: A free Storage Cloud CloudFTP: A free Storage Cloud ABSTRACT: The cloud computing is growing rapidly for it offers on-demand computing power and capacity. The power of cloud enables dynamic scalability of applications facing

More information

Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, and Satoshi Sekiguchi

Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, and Satoshi Sekiguchi Takahiro Hirofuchi, Hidemoto Nakada, Satoshi Itoh, and Satoshi Sekiguchi National Institute of Advanced Industrial Science and Technology (AIST), Japan VTDC2011, Jun. 8 th, 2011 1 Outline What is dynamic

More information

International Journal of Computer & Organization Trends Volume21 Number1 June 2015 A Study on Load Balancing in Cloud Computing

International Journal of Computer & Organization Trends Volume21 Number1 June 2015 A Study on Load Balancing in Cloud Computing A Study on Load Balancing in Cloud Computing * Parveen Kumar * Er.Mandeep Kaur Guru kashi University,Talwandi Sabo Guru kashi University,Talwandi Sabo Abstract: Load Balancing is a computer networking

More information

Cloud Panel Service Evaluation Scenarios

Cloud Panel Service Evaluation Scenarios Cloud Panel Service Evaluation Scenarios August 2014 Service Evaluation Scenarios The scenarios below are provided as a sample of how Finance may approach the evaluation of a particular service offered

More information

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. 3, NO. X, XXXXX 2015 1. Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds

IEEE TRANSACTIONS ON CLOUD COMPUTING, VOL. 3, NO. X, XXXXX 2015 1. Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds TRANSACTIONS ON CLOUD COMPUTING, VOL. 3, NO. X, XXXXX 2015 1 Monetary Cost Optimizations for Hosting Workflow-as-a-Service in IaaS Clouds Amelie Chi Zhou, Bingsheng He, and Cheng Liu Abstract Recently,

More information

Proactive, Resource-Aware, Tunable Real-time Fault-tolerant Middleware

Proactive, Resource-Aware, Tunable Real-time Fault-tolerant Middleware Proactive, Resource-Aware, Tunable Real-time Fault-tolerant Middleware Priya Narasimhan T. Dumitraş, A. Paulos, S. Pertet, C. Reverte, J. Slember, D. Srivastava Carnegie Mellon University Problem Description

More information

Fujitsu Managed Hosting Delivers your Cloud Infrastructure as a Service environment with confidence

Fujitsu Managed Hosting Delivers your Cloud Infrastructure as a Service environment with confidence Fujitsu Managed Hosting Delivers your Cloud Infrastructure as a Service environment with confidence Fujitsu supports and accelerates your adoption of cloud through a range of managed hosting services.

More information

Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT

Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT Experimental Investigation Decentralized IaaS Cloud Architecture Open Stack with CDT S. Gobinath, S. Saravanan PG Scholar, CSE Dept, M.Kumarasamy College of Engineering, Karur, India 1 Assistant Professor,

More information

Database high availability

Database high availability Database high availability Seminar in Data and Knowledge Engineering Viktorija Šukvietytė December 27, 2014 1. Introduction The contemporary world is overflowed with data. In 2010, Eric Schmidt one of

More information

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda

How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda How swift is your Swift? Ning Zhang, OpenStack Engineer at Zmanda Chander Kant, CEO at Zmanda 1 Outline Build a cost-efficient Swift cluster with expected performance Background & Problem Solution Experiments

More information

The Case for Enterprise Ready Virtual Private Clouds

The Case for Enterprise Ready Virtual Private Clouds The Case for Enterprise Ready Virtual Private Clouds Timothy Wood, Alexandre Gerber *, K.K. Ramakrishnan *, Jacobus van der Merwe *, and Prashant Shenoy University of Massachusetts Amherst *AT&T Research

More information

Process Replication for HPC Applications on the Cloud

Process Replication for HPC Applications on the Cloud Process Replication for HPC Applications on the Cloud Scott Purdy and Pete Hunt Advised by Prof. David Bindel December 17, 2010 1 Abstract Cloud computing has emerged as a new paradigm in large-scale computing.

More information

HPC performance applications on Virtual Clusters

HPC performance applications on Virtual Clusters Panagiotis Kritikakos EPCC, School of Physics & Astronomy, University of Edinburgh, Scotland - UK pkritika@epcc.ed.ac.uk 4 th IC-SCCE, Athens 7 th July 2010 This work investigates the performance of (Java)

More information

How To Create A Cloud Based System For Aaas (Networking)

How To Create A Cloud Based System For Aaas (Networking) 1 3.1 IaaS Definition IaaS: Infrastructure as a Service Through the internet, provide IT server, storage, computing power and other infrastructure capacity to the end users and the service fee based on

More information

Last time. Data Center as a Computer. Today. Data Center Construction (and management)

Last time. Data Center as a Computer. Today. Data Center Construction (and management) Last time Data Center Construction (and management) Johan Tordsson Department of Computing Science 1. Common (Web) application architectures N-tier applications Load Balancers Application Servers Databases

More information

Coding Techniques for Efficient, Reliable Networked Distributed Storage in Data Centers

Coding Techniques for Efficient, Reliable Networked Distributed Storage in Data Centers Coding Techniques for Efficient, Reliable Networked Distributed Storage in Data Centers Anwitaman Datta Joint work with Frédérique Oggier (SPMS) School of Computer Engineering Nanyang Technological University

More information

Cloud Services. May 28 th, 2014 Athens, Greece

Cloud Services. May 28 th, 2014 Athens, Greece Cloud Services May 28 th, 2014 Athens, Greece Cloud Services? Cloud services and PT PT is Virtualization technology and delivery leader Well known as storage & data protection integrator Chosen by RedHat

More information

Protecting your SQL database with Hybrid Cloud Backup and Recovery. Session Code CL02

Protecting your SQL database with Hybrid Cloud Backup and Recovery. Session Code CL02 Protecting your SQL database with Hybrid Cloud Backup and Recovery Session Code CL02 ARCserve True Hybrid Data Protection ARCserve Backup Data protection for complex environments Disk to Disk to-tape Disk

More information

Network Attached Storage. Jinfeng Yang Oct/19/2015

Network Attached Storage. Jinfeng Yang Oct/19/2015 Network Attached Storage Jinfeng Yang Oct/19/2015 Outline Part A 1. What is the Network Attached Storage (NAS)? 2. What are the applications of NAS? 3. The benefits of NAS. 4. NAS s performance (Reliability

More information

Towards a New Model for the Infrastructure Grid

Towards a New Model for the Infrastructure Grid INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS Cetraro (Italy), June 30 - July 4, 2008 Panel: From Grids to Cloud Services Towards a New Model for the Infrastructure Grid

More information

Preview of Oracle Database 12c In-Memory Option. Copyright 2013, Oracle and/or its affiliates. All rights reserved.

Preview of Oracle Database 12c In-Memory Option. Copyright 2013, Oracle and/or its affiliates. All rights reserved. Preview of Oracle Database 12c In-Memory Option 1 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any

More information

Server Virtualization and Cloud Computing

Server Virtualization and Cloud Computing Server Virtualization and Cloud Computing Four hidden impacts on uptime and availability by Stratus Technologies November, 2011 1 Abstract As x86 virtualization has become established and cloud computing

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

- Behind The Cloud -

- Behind The Cloud - - Behind The Cloud - Infrastructure and Technologies used for Cloud Computing Alexander Huemer, 0025380 Johann Taferl, 0320039 Florian Landolt, 0420673 Seminar aus Informatik, University of Salzburg Overview

More information

Traditional v/s CONVRGD

Traditional v/s CONVRGD Traditional v/s CONVRGD Traditional Virtualization Stack Converged Virtualization Infrastructure with HCE/HSE Data protection software applications PDU Backup Servers + Virtualization Storage Switch HA

More information

On the Use of Fuzzy Modeling in Virtualized Data Center Management Jing Xu, Ming Zhao, José A. B. Fortes, Robert Carpenter*, Mazin Yousif*

On the Use of Fuzzy Modeling in Virtualized Data Center Management Jing Xu, Ming Zhao, José A. B. Fortes, Robert Carpenter*, Mazin Yousif* On the Use of Fuzzy Modeling in Virtualized Data Center Management Jing Xu, Ming Zhao, José A. B. Fortes, Robert Carpenter*, Mazin Yousif* Electrical and Computer Engineering, University of Florida *Intel

More information

The Virtual Grid Application Development Software (VGrADS) Project

The Virtual Grid Application Development Software (VGrADS) Project The Virtual Grid Application Development Software (VGrADS) Project VGrADS: Enabling e-science Workflows on Grids and Clouds with Fault Tolerance http://vgrads.rice.edu/ VGrADS Goal: Distributed Problem

More information

Best Practices for Using MySQL in the Cloud

Best Practices for Using MySQL in the Cloud Best Practices for Using MySQL in the Cloud Luis Soares, Sr. Software Engineer, MySQL Replication, Oracle Lars Thalmann, Director Replication, Backup, Utilities and Connectors THE FOLLOWING IS INTENDED

More information

Security management in the internet era

Security management in the internet era Security management in the internet era Cloud Security (1) Septemberr 29, 2011 Jun Murai Keio University! Suguru Yamaguchi! Nara Institute of Science and Technology! Schedule 01st (09/22) Course Description

More information

Hadoop Architecture. Part 1

Hadoop Architecture. Part 1 Hadoop Architecture Part 1 Node, Rack and Cluster: A node is simply a computer, typically non-enterprise, commodity hardware for nodes that contain data. Consider we have Node 1.Then we can add more nodes,

More information

Web Application Deployment in the Cloud Using Amazon Web Services From Infancy to Maturity

Web Application Deployment in the Cloud Using Amazon Web Services From Infancy to Maturity P3 InfoTech Solutions Pvt. Ltd http://www.p3infotech.in July 2013 Created by P3 InfoTech Solutions Pvt. Ltd., http://p3infotech.in 1 Web Application Deployment in the Cloud Using Amazon Web Services From

More information

Cloud 101. Mike Gangl, Caltech/JPL, michael.e.gangl@jpl.nasa.gov 2015 California Institute of Technology. Government sponsorship acknowledged

Cloud 101. Mike Gangl, Caltech/JPL, michael.e.gangl@jpl.nasa.gov 2015 California Institute of Technology. Government sponsorship acknowledged Cloud 101 Mike Gangl, Caltech/JPL, michael.e.gangl@jpl.nasa.gov 2015 California Institute of Technology. Government sponsorship acknowledged Outline What is cloud computing? Cloud service models Deployment

More information

Cloud Computing. Summary

Cloud Computing. Summary Cloud Computing Lecture 1 2011-2012 https://fenix.ist.utl.pt/disciplinas/cn Summary Teaching Staff. Rooms and Schedule. Goals. Context. Syllabus. Reading Material. Assessment and Grading. Important Dates.

More information

Virtualization and Cloud Computing

Virtualization and Cloud Computing Written by Zakir Hossain, CS Graduate (OSU) CEO, Data Group Fed Certifications: PFA (Programming Foreign Assistance), COR (Contracting Officer), AOR (Assistance Officer) Oracle Certifications: OCP (Oracle

More information

Last time. Today. IaaS Providers. Amazon Web Services, overview

Last time. Today. IaaS Providers. Amazon Web Services, overview Last time General overview, motivation, expected outcomes, other formalities, etc. Please register for course Online (if possible), or talk to Yvonne@CS Course evaluation forgotten Please assign one volunteer

More information

Amazon EC2 Product Details Page 1 of 5

Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Product Details Page 1 of 5 Amazon EC2 Functionality Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of

More information

Building an Internal Cloud that is ready for the external Cloud

Building an Internal Cloud that is ready for the external Cloud Building an Internal Cloud that is ready for the external Cloud Luca ZERMINIANI, Senior Systems Engineer, VMware Italy Athens, February 2010 2009 VMware Inc. All rights reserved Agenda How virtualization

More information

Volunteer Computing, Grid Computing and Cloud Computing: Opportunities for Synergy. Derrick Kondo INRIA, France

Volunteer Computing, Grid Computing and Cloud Computing: Opportunities for Synergy. Derrick Kondo INRIA, France Volunteer Computing, Grid Computing and Cloud Computing: Opportunities for Synergy Derrick Kondo INRIA, France Outline Cloud Grid Volunteer Computing Cloud Background Vision Hide complexity of hardware

More information

The Google File System

The Google File System The Google File System By Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung (Presented at SOSP 2003) Introduction Google search engine. Applications process lots of data. Need good file system. Solution:

More information

Analyzing the Applicatbility of Airline Booking Systems for Cloud Computing Offerings

Analyzing the Applicatbility of Airline Booking Systems for Cloud Computing Offerings ISGC 2010 Analyzing the Applicatbility of Airline Booking Systems for Cloud Computing Offerings Johannes Watzl, Nils gentschen Felde, Dieter Kranzlmüller Aufgabensteller: Prof. Dr. H.-G. Hegering Grid

More information

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández

DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING. Carlos de Alfonso Andrés García Vicente Hernández DESIGN OF A PLATFORM OF VIRTUAL SERVICE CONTAINERS FOR SERVICE ORIENTED CLOUD COMPUTING Carlos de Alfonso Andrés García Vicente Hernández 2 INDEX Introduction Our approach Platform design Storage Security

More information

Mesos: A Platform for Fine- Grained Resource Sharing in Data Centers (II)

Mesos: A Platform for Fine- Grained Resource Sharing in Data Centers (II) UC BERKELEY Mesos: A Platform for Fine- Grained Resource Sharing in Data Centers (II) Anthony D. Joseph LASER Summer School September 2013 My Talks at LASER 2013 1. AMP Lab introduction 2. The Datacenter

More information

Cloud Computing through Virtualization and HPC technologies

Cloud Computing through Virtualization and HPC technologies Cloud Computing through Virtualization and HPC technologies William Lu, Ph.D. 1 Agenda Cloud Computing & HPC A Case of HPC Implementation Application Performance in VM Summary 2 Cloud Computing & HPC HPC

More information

Scheduling. Yücel Saygın. These slides are based on your text book and on the slides prepared by Andrew S. Tanenbaum

Scheduling. Yücel Saygın. These slides are based on your text book and on the slides prepared by Andrew S. Tanenbaum Scheduling Yücel Saygın These slides are based on your text book and on the slides prepared by Andrew S. Tanenbaum 1 Scheduling Introduction to Scheduling (1) Bursts of CPU usage alternate with periods

More information

Oracle Solutions on Top of VMware vsphere 4. Saša Hederić VMware Adriatic

Oracle Solutions on Top of VMware vsphere 4. Saša Hederić VMware Adriatic Oracle Solutions on Top of VMware vsphere 4 Saša Hederić VMware Adriatic The Problem Where the IT Budget Goes 5% Infrastructure Investment 23% Application Investment 42% Infrastructure Maintenance Overwhelming

More information

Virtual Machine Synchronization for High Availability Clusters

Virtual Machine Synchronization for High Availability Clusters Virtual Machine Synchronization for High Availability Clusters Yoshiaki Tamura, Koji Sato, Seiji Kihara, Satoshi Moriai NTT Cyber Space Labs. 2007/4/17 Consolidating servers using VM Internet services

More information

A New Strategy for Data Management in a Time of Hyper Change

A New Strategy for Data Management in a Time of Hyper Change 1 A New Strategy for Data Management in a Time of Hyper Change 2 Everyone Claims to Do Data Protection How Do Enterprises Protect Data Today? In Medium to Large Enterprises, traditional backup is today

More information

SERVER VIRTUALIZATION IN MANUFACTURING

SERVER VIRTUALIZATION IN MANUFACTURING SERVER VIRTUALIZATION IN MANUFACTURING White Paper 2 Do s and Don ts for Your Most Critical Manufacturing Systems Abstract While the benefits of server virtualization at the corporate data center are receiving

More information

IBM Spectrum Protect in the Cloud

IBM Spectrum Protect in the Cloud IBM Spectrum Protect in the Cloud. Disclaimer IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM s sole discretion. Information regarding

More information

Azure VM Performance Considerations Running SQL Server

Azure VM Performance Considerations Running SQL Server Azure VM Performance Considerations Running SQL Server Your company logo here Vinod Kumar M @vinodk_sql http://blogs.extremeexperts.com Session Objectives And Takeaways Session Objective(s): Learn the

More information

Dynamic Load Balancing: Improve Efficiency in Cloud Computing Argha Roy * M.Tech CSE Netaji Subhash Engineering College West Bengal, India.

Dynamic Load Balancing: Improve Efficiency in Cloud Computing Argha Roy * M.Tech CSE Netaji Subhash Engineering College West Bengal, India. Dynamic Load Balancing: Improve Efficiency in Cloud Computing Argha Roy * M.Tech CSE Netaji Subhash Engineering College West Bengal, India. Diptam Dutta M.Tech CSE Heritage Institute of Technology West

More information

DISTRIBUTED SYSTEMS AND CLOUD COMPUTING. A Comparative Study

DISTRIBUTED SYSTEMS AND CLOUD COMPUTING. A Comparative Study DISTRIBUTED SYSTEMS AND CLOUD COMPUTING A Comparative Study Geographically distributed resources, such as storage devices, data sources, and computing power, are interconnected as a single, unified resource

More information

Big data blue print for cloud architecture

Big data blue print for cloud architecture Big data blue print for cloud architecture -COGNIZANT Image Area Prabhu Inbarajan Srinivasan Thiruvengadathan Muralicharan Gurumoorthy Praveen Codur 2012, Cognizant Next 30 minutes Big Data / Cloud challenges

More information

Data Management in the Cloud: Limitations and Opportunities. Annies Ductan

Data Management in the Cloud: Limitations and Opportunities. Annies Ductan Data Management in the Cloud: Limitations and Opportunities Annies Ductan Discussion Outline: Introduc)on Overview Vision of Cloud Compu8ng Managing Data in The Cloud Cloud Characteris8cs Data Management

More information

JOHNSON COUNTY COMMUNITY COLLEGE 12345 College Blvd., Overland Park, KS 66210 Ph. 913-469-3812 Fax 913-469-4429

JOHNSON COUNTY COMMUNITY COLLEGE 12345 College Blvd., Overland Park, KS 66210 Ph. 913-469-3812 Fax 913-469-4429 JOHNSON COUNTY COMMUNITY COLLEGE 12345 College Blvd., Overland Park, KS 66210 Ph. 913-469-3812 Fax 913-469-4429 ADDENDUM #1 September 21, 2015 REQUEST FOR PROPOSALS #16-033 FOR CLOUD BASED BACKUP & RECOVERY

More information

Cloud Management: Knowing is Half The Battle

Cloud Management: Knowing is Half The Battle Cloud Management: Knowing is Half The Battle Raouf BOUTABA David R. Cheriton School of Computer Science University of Waterloo Joint work with Qi Zhang, Faten Zhani (University of Waterloo) and Joseph

More information

Comparison of Open Source Cloud System for Small and Medium Sized Enterprises

Comparison of Open Source Cloud System for Small and Medium Sized Enterprises , pp.276-282 http://dx.doi.org/10.14257/astl.2014.51.63 Comparison of Open Source Cloud System for Small and Medium Sized Enterprises Yasuo Uchida 1, Seigo Matsuno 1, Makoto Sakamoto 2 1 Ube National College

More information

Introduction to AWS Economics

Introduction to AWS Economics Introduction to AWS Economics Reducing Costs and Complexity May 2015 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved. Notices This document is provided for informational purposes

More information

TGL VMware Presentation. Guangzhou Macau Hong Kong Shanghai Beijing

TGL VMware Presentation. Guangzhou Macau Hong Kong Shanghai Beijing TGL VMware Presentation Guangzhou Macau Hong Kong Shanghai Beijing The Path To IT As A Service Existing Apps Future Apps Private Cloud Lots of Hardware and Plumbing Today IT TODAY Internal Cloud Federation

More information

Cloud Server Performance A Comparative Analysis of 5 Large Cloud IaaS Providers

Cloud Server Performance A Comparative Analysis of 5 Large Cloud IaaS Providers Cloud Server Performance A Comparative Analysis of 5 Large Cloud IaaS Providers Cloud Spectator Study June 5, 2013 With a lack of standardization in the IaaS industry, providers freely use unique terminology

More information

Network Infrastructure Services CS848 Project

Network Infrastructure Services CS848 Project Quality of Service Guarantees for Cloud Services CS848 Project presentation by Alexey Karyakin David R. Cheriton School of Computer Science University of Waterloo March 2010 Outline 1. Performance of cloud

More information

PeerMon: A Peer-to-Peer Network Monitoring System

PeerMon: A Peer-to-Peer Network Monitoring System PeerMon: A Peer-to-Peer Network Monitoring System Tia Newhall, Janis Libeks, Ross Greenwood, Jeff Knerr Computer Science Department Swarthmore College Swarthmore, PA USA newhall@cs.swarthmore.edu Target:

More information

High Availability for Database Systems in Cloud Computing Environments. Ashraf Aboulnaga University of Waterloo

High Availability for Database Systems in Cloud Computing Environments. Ashraf Aboulnaga University of Waterloo High Availability for Database Systems in Cloud Computing Environments Ashraf Aboulnaga University of Waterloo Acknowledgments University of Waterloo Prof. Kenneth Salem Umar Farooq Minhas Rui Liu (post-doctoral

More information

Daniel J. Adabi. Workshop presentation by Lukas Probst

Daniel J. Adabi. Workshop presentation by Lukas Probst Daniel J. Adabi Workshop presentation by Lukas Probst 3 characteristics of a cloud computing environment: 1. Compute power is elastic, but only if workload is parallelizable 2. Data is stored at an untrusted

More information

Outline. MCSA: Server Virtualization

Outline. MCSA: Server Virtualization MCSA: Server Virtualization Description Get hands-on instruction and practice implementing Microsoft Server Virtualization with Windows Server 2012 R2 Hyper-V and System Center 2012 R2 Virtual Machine

More information

An Energy Efficient Server Load Balancing Algorithm

An Energy Efficient Server Load Balancing Algorithm An Energy Efficient Server Load Balancing Algorithm Rima M. Shah 1, Dr. Priti Srinivas Sajja 2 1 Assistant Professor in Master of Computer Application,ITM Universe,Vadodara, India 2 Professor at Post Graduate

More information

The Easiest Way to Run Spark Jobs. How-To Guide

The Easiest Way to Run Spark Jobs. How-To Guide The Easiest Way to Run Spark Jobs How-To Guide The Easiest Way to Run Spark Jobs Recently, Databricks added a new feature, Jobs, to our cloud service. You can find a detailed overview of this feature in

More information

High Availability for Citrix XenServer

High Availability for Citrix XenServer WHITE PAPER Citrix XenServer High Availability for Citrix XenServer Enhancing XenServer Fault Tolerance with High Availability www.citrix.com Contents Contents... 2 Heartbeating for availability... 4 Planning

More information

A Secure Strategy using Weighted Active Monitoring Load Balancing Algorithm for Maintaining Privacy in Multi-Cloud Environments

A Secure Strategy using Weighted Active Monitoring Load Balancing Algorithm for Maintaining Privacy in Multi-Cloud Environments IJSTE - International Journal of Science Technology & Engineering Volume 1 Issue 10 April 2015 ISSN (online): 2349-784X A Secure Strategy using Weighted Active Monitoring Load Balancing Algorithm for Maintaining

More information

Cloud Computing #6 - Virtualization

Cloud Computing #6 - Virtualization Cloud Computing #6 - Virtualization Main source: Smith & Nair, Virtual Machines, Morgan Kaufmann, 2005 Today What do we mean by virtualization? Why is it important to cloud? What is the penalty? Current

More information

Millennium Systems on the Cloud:

Millennium Systems on the Cloud: Millennium Systems on the Cloud: Experience on System Migration from SUN V490 to VMWare Platform Li Yiu On, Systems Section, Library Tsang Jimmy, Office of Information Technology Hong Kong Baptist University

More information

Box Leangsuksun+ * Thammasat University, Patumtani, Thailand # Oak Ridge National Laboratory, Oak Ridge, TN, USA + Louisiana Tech University, Ruston,

Box Leangsuksun+ * Thammasat University, Patumtani, Thailand # Oak Ridge National Laboratory, Oak Ridge, TN, USA + Louisiana Tech University, Ruston, N. Saragol * Hong Ong# Box Leangsuksun+ K. Chanchio* * Thammasat University, Patumtani, Thailand # Oak Ridge National Laboratory, Oak Ridge, TN, USA + Louisiana Tech University, Ruston, LA, USA Introduction

More information

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud) Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University

More information