Towards Predictable Datacenter Networks

Size: px
Start display at page:

Download "Towards Predictable Datacenter Networks"

Transcription

1 Towards Predictable Datacenter Networks Hitesh Ballani, Paolo Costa, Thomas Karagiannis and Ant Rowstron Microsoft Research, Cambridge

2 This talk is about Guaranteeing network performance for tenants in multi-tenant datacenters Multi-tenant datacenters Datacenters with multiple (possibly competing) tenants Private datacenters Run by organizations like Facebook, Intel, etc. Tenants: Product groups and applications Cloud datacenters Amazon EC2, Microsoft Azure, Rackspace, etc. Tenants: Users renting virtual machines

3 Tenant Cloud datacenters 101 Simple interface: Tenants ask for a set of VMs Amazon EC2 Interface Request VMs Charging is per-vm, per-hour Amazon EC2 small instances: $0.085/hour No (intra-cloud) network cost Network performance is not guaranteed Bandwidth between a tenant s VMs depends on their placement, network load, protocols used, etc.

4 Performance variability in Up the to 5x wild variability Study Study Provider Duration A [Giurgui 10] Amazon EC2 n/a B [Schad 10] Amazon EC2 31 days C/D/E [Li 10] (Azure, EC2, Rackspace) 1 day F/G [Yu 10] Amazon EC2 1 day H [Mangot 09] Amazon EC2 1 day

5 Network performance can vary... so what? Tenant Tenant Map Reduce Job Data analytics on an isolated cluster Enterprise Results Data analytics in a multi-tenant datacenter Completion Time 4 hours Unpredictability of application performance and tenant costs is a key hindrance to cloud adoption Key Map Contributor: Reduce Network Results performance variation Job Datacenter Completion Time hours Variable tenant costs Expected cost (based on 4 hour completion time) = $100 Actual cost = $

6 Predictable datacenter networks Extend the tenant-provider interface to account for the network Tenant Request Request Virtual Network # of VMs and network demands # of VMs and network demands Contributions- VM 1 VM 2 VM N Virtual network abstractions To capture tenant network demands Oktopus: Proof of concept system Implements virtual networks in multi-tenant datacenters Can be incrementally deployed today! Key Idea: Tenants are offered a virtual network with bandwidth guarantees This decouples tenant performance from provider infrastructure

7 Introduction Talk Outline Virtual network abstractions Oktopus Allocating virtual networks Enforcing virtual networks Evaluation

8 Abstraction 1: Virtual Cluster (VC) Motivation: In enterprises, tenants run applications on dedicated Ethernet clusters Virtual Switch Total bandwidth = N * B VM 1 VM 2 VM N Request <N, B> N VMs. Each VM can send and receive at B Mbps Tenants get a network with no oversubscription Suitable for data-intensive apps. (MapReduce, BLAST) Moderate provider flexibility

9 Abstraction 2: Virtual Oversubscribed Cluster (VOC) VMs can send traffic to group members at B Mbps Group Virtual Switch Root Virtual Switch Total bandwidth at root = N * B / O Total bandwidth at VMs = N * B VM 1 Group 1 VM N S VM 1 Group 2 VM S. VM 1 VM S Group N/S Motivation: Many Request applications <N, B, moving S, O> to the cloud have N VOC VMs capitalizes in groups of size S. Oversubscription factor O. localized on communication tenant communication patterns patterns Applications No oversubscription Suitable are composed for typical for of intra-group applications groups with (though communication more traffic not all) within (captures Intra-group the Improved sparseness groups communication than of provider across inter-group is the groups flexibility common communication) case! Oversubscription factor O for inter-group communication

10 Introduction Talk Outline Virtual network abstractions Oktopus Allocating virtual networks Enforcing virtual networks Evaluation

11 Oktopus Offers virtual networks to tenants in datacenters Two main components Management plane: Allocation of tenant requests Allocates tenant requests to physical infrastructure Accounts for tenant network bandwidth requirements Data plane: Enforcement of virtual networks Enforces tenant bandwidth requirements Achieved through rate limiting at end hosts

12 Allocating Virtual Clusters Request : <3 VMs, 100 Mbps> Max Sending Rate = 2*100 = 200 VM What for an bandwidth existing Max Receive tenant Rate = needs to be 1*100 = 100 reserved for the B/W tenant needed on this on link? = Min (200, 100) = 100Mbps For a virtual cluster <N,B>, bandwidth needed on a link that How Datacenter to Tenant find Physical a valid Request allocation? Topology connects m VMs to the remaining (N-m) VMs is = Min (m, N-m) * B An allocation Link divides of tenant virtual VMs tree to into physical two parts machines Allocate 4 physical a tenant machines, asking for 2 3 VM VMs slots arranged per machine a virtual cluster Tenant Consider with traffic all traffic 100 Mbps traverses from each, the the i.e. highlighted left to right <3 VMs, 100Mbps> links part Bandwidth needed <= Link s Residual Bandwidth For a valid allocation:

13 Allocation Algorithm Request : <3 VMs, 100 Mbps> How Solution many VMs can At most be allocated 1 VM for to this tenant machine? can be 2 VMs allocated here 2 VMs 2 VMs 1 VM 1 VM Constraints for Allocation # of VMs (m) is that fast can and be allocated efficient to the machine- 3 VMs Greedy Key allocation intuition Packing algorithm Traverse Validity up the conditions VMs together hierarchy can motivated and be determine used to by determine the fact that the lowest the level at number datacenter of VMs networks which that all can are 3 VMs be can allocated typically oversubscribed be allocated to any level of the Allocation datacenter; can be extended machines, for goals racks like failure and so resiliency, on etc. 1. VMs can only be allocated to empty slots m <= VMs are requested m <= 3 3. Enough b/w on outbound link min (m, 3-m)*100 <= 200

14 Introduction Talk Outline Virtual network abstractions Oktopus Allocating virtual networks Enforcing virtual networks Evaluation

15 Enforcement in Oktopus: Key highlights Oktopus enforces virtual networks at end hosts Use egress rate limiters at end hosts Implement on hypervisor/vmm Oktopus can be deployed today No changes to tenant applications No network support Tenants without virtual networks can be supported Good for incremental roll out

16 Introduction Talk Outline Virtual network abstractions Oktopus Allocating virtual networks Enforcing virtual networks Evaluation

17 Datacenter Simulator Flow-based simulator 16,000 servers and 4 VMs/server 64,000 VMs Three-tier network topology (10:1 oversubscription) Tenants submit requests for VMs and execute jobs Job: VMs process and shuffle data between each other Baseline: representative of today s setup Tenants simply ask for VMs VMs are allocated in a locality-aware fashion Virtual network request Tenants ask for Virtual Cluster (VC) or Virtual Oversubscribed Cluster (VOC)

18 Worse Private datacenters VC is Virtual Cluster VOC-10 is Virtual Oversubscribed Cluster with oversubscription=10 Execute a batch of 10,000 tenant jobs Jobs vary in network intensiveness (bandwidth at which a job can generate data) Better Virtual networks improve completion time VC: 50% of Baseline VOC-10: 31% of Baseline Jobs become more network intensive

19 Cloud Datacenters Amazon EC2 s reported target utilization Worse Tenant job requests arrive over time Jobs are rejected if they cannot be accommodated on arrival (representative of cloud datacenters) Better Job requests arrive faster Rejected Requests Baseline: 31% VC: 15% VOC-10: 5%

20 Tenant Costs What should tenants pay to ensure provider revenue neutrality, i.e. provider revenue remains the same with all approaches Based on today s EC2 prices, i.e. $0.085/hour for each VM Provider revenue increases while tenants pay less At 70% target utilization, provider revenue increases by 20% and median tenant cost reduces by 42%

21 Oktopus Deployment Implementation scales well and imposes low overhead Allocation of virtual networks is fast In a datacenter with 10 5 machines, median allocation time is 0.35ms Enforcement of virtual networks is cheap Use Traffic Control API to enforce rate limits at end hosts Deployment on testbed with 25 end hosts End hosts arranged in five racks

22 Oktopus Deployment Cross-validation of simulation results Completion time for jobs in the simulator matches that on the testbed

23 Summary Proposal: Offer virtual networks to tenants Virtual network abstractions Resemble physical networks in enterprises Make transition easier for tenants Proof of concept: Oktopus Tenants get guaranteed network performance Sufficient multiplexing for providers Win-win: tenants pay less, providers earn more! How to determine tenant network demands? Ongoing work: Map high-level goals (like desired completion time) to Oktopus abstractions

24 Thank you

Predictable Data Centers

Predictable Data Centers Predictable Data Centers Thomas Karagiannis Hitesh Ballani, Paolo Costa, Fahad Dogar, Keon Jang, Greg O Shea, Eno Thereska, and Ant Rowstron Systems & Networking Microsoft Research, Cambridge http://research.microsoft.com/datacenters/

More information

Chatty Tenants and the Cloud Network Sharing Problem

Chatty Tenants and the Cloud Network Sharing Problem Chatty Tenants and the Cloud Network Sharing Problem Hitesh Ballani, Keon Jang, Thomas Karagiannis Changhoon Kim, Dinan Gunawardena, Greg O Shea MSR Cambridge, Windows Azure This talk is about... How to

More information

Towards Predictable Datacenter Networks

Towards Predictable Datacenter Networks Towards Predictable Datacenter Networks Hitesh Ballani Paolo Costa Thomas Karagiannis Ant Rowstron hiballan@microsoft.com costa@imperial.ac.uk thomkar@microsoft.com antr@microsoft.com Microsoft Research

More information

The Price Is Right: Towards Location-Independent Costs in Datacenters

The Price Is Right: Towards Location-Independent Costs in Datacenters The Is Right: Towards Location-Independent Costs in Datacenters Hitesh Ballani Paolo Costa Thomas Karagiannis Ant Rowstron hiballan@microsoft.com costa@imperial.ac.uk thomkar@microsoft.com antr@microsoft.com

More information

Beyond the Stars: Revisiting Virtual Cluster Embeddings

Beyond the Stars: Revisiting Virtual Cluster Embeddings Beyond the Stars: Revisiting Virtual Cluster Embeddings Matthias Rost Technische Universität Berlin September 7th, 2015, Télécom-ParisTech Joint work with Carlo Fuerst, Stefan Schmid Published in ACM SIGCOMM

More information

Decentralized Task-Aware Scheduling for Data Center Networks

Decentralized Task-Aware Scheduling for Data Center Networks Decentralized Task-Aware Scheduling for Data Center Networks Fahad R. Dogar, Thomas Karagiannis, Hitesh Ballani, Ant Rowstron Presented by Eric Dong (yd2dong) October 30, 2015 Tasks in data centers Applications

More information

Gatekeeper: Supporting Bandwidth Guarantees for Multi-tenant Datacenter Networks

Gatekeeper: Supporting Bandwidth Guarantees for Multi-tenant Datacenter Networks Gatekeeper: Supporting Bandwidth Guarantees for Multi-tenant Datacenter Networks Henrique Rodrigues, Yoshio Turner, Jose Renato Santos, Paolo Victor, Dorgival Guedes HP Labs WIOV 2011, Portland, OR The

More information

Chatty Tenants and the Cloud Network Sharing Problem

Chatty Tenants and the Cloud Network Sharing Problem Chatty Tenants and the Cloud Network Sharing Problem Hitesh Ballani Keon Jang Thomas Karagiannis Changhoon Kim Dinan Gunawardena Greg O Shea Microsoft Research Windows Azure Cambridge, UK USA Abstract

More information

Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load

Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load Payment minimization and Error-tolerant Resource Allocation for Cloud System Using equally spread current execution load Pooja.B. Jewargi Prof. Jyoti.Patil Department of computer science and engineering,

More information

Cloud Computing Trends

Cloud Computing Trends UT DALLAS Erik Jonsson School of Engineering & Computer Science Cloud Computing Trends What is cloud computing? Cloud computing refers to the apps and services delivered over the internet. Software delivered

More information

Silo: Predictable Message Completion Time in the Cloud

Silo: Predictable Message Completion Time in the Cloud Silo: Predictable Message Completion Time in the Cloud Keon Jang Justine Sherry Hitesh Ballani Toby Moncaster Microsoft Research UC Berkeley University of Cambridge September, 2013 Technical Report MSR-TR-2013-95

More information

CLOUD NETWORKING THE NEXT CHAPTER FLORIN BALUS

CLOUD NETWORKING THE NEXT CHAPTER FLORIN BALUS CLOUD NETWORKING THE NEXT CHAPTER FLORIN BALUS COMMON APPLICATION VIEW OF THE NETWORK Fallacies of Distributed Computing 1. The network is reliable. 2. Latency is zero. 3. Bandwidth is infinite. 4. The

More information

STRATEGIC WHITE PAPER. The next step in server virtualization: How containers are changing the cloud and application landscape

STRATEGIC WHITE PAPER. The next step in server virtualization: How containers are changing the cloud and application landscape STRATEGIC WHITE PAPER The next step in server virtualization: How containers are changing the cloud and application landscape Abstract Container-based server virtualization is gaining in popularity, due

More information

IoNCloud: exploring application affinity to improve utilization and predictability in datacenters

IoNCloud: exploring application affinity to improve utilization and predictability in datacenters IoNCloud: exploring application affinity to improve utilization and predictability in datacenters Daniel S. Marcon, Miguel C. Neves, Rodrigo R. Oliveira, Leonardo R. Bays, Raouf Boutaba, Luciano P. Gaspary,

More information

Lecture 7: Data Center Networks"

Lecture 7: Data Center Networks Lecture 7: Data Center Networks" CSE 222A: Computer Communication Networks Alex C. Snoeren Thanks: Nick Feamster Lecture 7 Overview" Project discussion Data Centers overview Fat Tree paper discussion CSE

More information

Impact of Virtualization on Cloud Networking Arista Networks Whitepaper

Impact of Virtualization on Cloud Networking Arista Networks Whitepaper Overview: Virtualization takes IT by storm The adoption of virtualization in datacenters creates the need for a new class of networks designed to support elasticity of resource allocation, increasingly

More information

Elasticity-Aware Virtual Machine Placement in K-ary Cloud Data Centers

Elasticity-Aware Virtual Machine Placement in K-ary Cloud Data Centers Elasticity-Aware Virtual Machine Placement in K-ary Cloud Data Centers Kangkang Li *1, Jie Wu 2, Adam Blaisse 3 1, 2, 3 Department of Computer and Information Sciences, Temple University, Philadelphia,

More information

Data Centers and Cloud Computing. Data Centers

Data Centers and Cloud Computing. Data Centers Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises

More information

Towards an understanding of oversubscription in cloud

Towards an understanding of oversubscription in cloud IBM Research Towards an understanding of oversubscription in cloud Salman A. Baset, Long Wang, Chunqiang Tang sabaset@us.ibm.com IBM T. J. Watson Research Center Hawthorne, NY Outline Oversubscription

More information

Falloc: Fair Network Bandwidth Allocation in IaaS Datacenters via a Bargaining Game Approach

Falloc: Fair Network Bandwidth Allocation in IaaS Datacenters via a Bargaining Game Approach Falloc: Fair Network Bandwidth Allocation in IaaS Datacenters via a Bargaining Game Approach Fangming Liu 1,2 In collaboration with Jian Guo 1,2, Haowen Tang 1,2, Yingnan Lian 1,2, Hai Jin 2 and John C.S.

More information

Core and Pod Data Center Design

Core and Pod Data Center Design Overview The Core and Pod data center design used by most hyperscale data centers is a dramatically more modern approach than traditional data center network design, and is starting to be understood by

More information

Microsoft Private Cloud Fast Track

Microsoft Private Cloud Fast Track Microsoft Private Cloud Fast Track Microsoft Private Cloud Fast Track is a reference architecture designed to help build private clouds by combining Microsoft software with Nutanix technology to decrease

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information

CS 91: Cloud Systems & Datacenter Networks. The Datacenter Environment. Kevin Webb Swarthmore College September 3, 2014

CS 91: Cloud Systems & Datacenter Networks. The Datacenter Environment. Kevin Webb Swarthmore College September 3, 2014 CS 91: Cloud Systems & Datacenter Networks The Datacenter Environment Kevin Webb Swarthmore College September 3, 2014 Announcements Sign up for piazza If you didn t just add the course, you should have

More information

The Software Defined Hybrid Packet Optical Datacenter Network SDN AT LIGHT SPEED TM. 2012-13 CALIENT Technologies www.calient.

The Software Defined Hybrid Packet Optical Datacenter Network SDN AT LIGHT SPEED TM. 2012-13 CALIENT Technologies www.calient. The Software Defined Hybrid Packet Optical Datacenter Network SDN AT LIGHT SPEED TM 2012-13 CALIENT Technologies www.calient.net 1 INTRODUCTION In datacenter networks, video, mobile data, and big data

More information

Data Centers and Cloud Computing

Data Centers and Cloud Computing Data Centers and Cloud Computing CS377 Guest Lecture Tian Guo 1 Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Case Study: Amazon EC2 2 Data Centers

More information

Data Centers and Cloud Computing. Data Centers. MGHPCC Data Center. Inside a Data Center

Data Centers and Cloud Computing. Data Centers. MGHPCC Data Center. Inside a Data Center Data Centers and Cloud Computing Intro. to Data centers Virtualization Basics Intro. to Cloud Computing Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises

More information

MS 20246C Monitoring and Operating a Private Cloud

MS 20246C Monitoring and Operating a Private Cloud MS 20246C Monitoring and Operating a Private Cloud Description: Days: 5 Prerequisites: This course describes how to monitor and operate a cloud with Microsoft System Center 2012 R2. This course focuses

More information

Comparison of Windows IaaS Environments

Comparison of Windows IaaS Environments Comparison of Windows IaaS Environments Comparison of Amazon Web Services, Expedient, Microsoft, and Rackspace Public Clouds January 5, 215 TABLE OF CONTENTS Executive Summary 2 vcpu Performance Summary

More information

Towards Rack-scale Computing Challenges and Opportunities

Towards Rack-scale Computing Challenges and Opportunities Towards Rack-scale Computing Challenges and Opportunities Paolo Costa paolo.costa@microsoft.com joint work with Raja Appuswamy, Hitesh Ballani, Sergey Legtchenko, Dushyanth Narayanan, Ant Rowstron Hardware

More information

Optimizing Data Center Networks for Cloud Computing

Optimizing Data Center Networks for Cloud Computing PRAMAK 1 Optimizing Data Center Networks for Cloud Computing Data Center networks have evolved over time as the nature of computing changed. They evolved to handle the computing models based on main-frames,

More information

Data Centers and Cloud Computing. Data Centers

Data Centers and Cloud Computing. Data Centers Data Centers and Cloud Computing Slides courtesy of Tim Wood 1 Data Centers Large server and storage farms 1000s of servers Many TBs or PBs of data Used by Enterprises for server applications Internet

More information

Managing Network Reservation for Tenants in Oversubscribed Clouds

Managing Network Reservation for Tenants in Oversubscribed Clouds 213 IEEE 21st International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems Managing Network Reservation for Tenants in Oversubscribed Clouds Mayank Mishra IIT Bombay,

More information

Configuring and Deploying a Private Cloud. Day(s): 5. Overview

Configuring and Deploying a Private Cloud. Day(s): 5. Overview Configuring and Deploying a Private Cloud Day(s): 5 Course Code: M20247 Overview This course equips students with the skills they require to configure and deploy a cloud using Microsoft System Center 2012

More information

MS 20247C Configuring and Deploying a Private Cloud

MS 20247C Configuring and Deploying a Private Cloud MS 20247C Configuring and Deploying a Private Cloud Description: Days: 5 Prerequisites: This course equips students with the skills they require to configure and deploy a cloud using Microsoft System Center

More information

Outline. VL2: A Scalable and Flexible Data Center Network. Problem. Introduction 11/26/2012

Outline. VL2: A Scalable and Flexible Data Center Network. Problem. Introduction 11/26/2012 VL2: A Scalable and Flexible Data Center Network 15744: Computer Networks, Fall 2012 Presented by Naveen Chekuri Outline Introduction Solution Approach Design Decisions Addressing and Routing Evaluation

More information

20247D: Configuring and Deploying a Private Cloud

20247D: Configuring and Deploying a Private Cloud 20247D: Configuring and Deploying a Private Course Details Course Code: Duration: Notes: 20247D 5 days This course syllabus should be used to determine whether the course is appropriate for the students,

More information

Emerging Technology for the Next Decade

Emerging Technology for the Next Decade Emerging Technology for the Next Decade Cloud Computing Keynote Presented by Charles Liang, President & CEO Super Micro Computer, Inc. What is Cloud Computing? Cloud computing is Internet-based computing,

More information

Enhancing Application Robustness in Cloud Data Centers

Enhancing Application Robustness in Cloud Data Centers Enhancing Application Robustness in Cloud Data Centers Madalin Mihailescu, Andres Rodriguez, Cristiana Amza University of Toronto Dmitrijs Palcikovs, Gabriel Iszlai, Andrew Trossman, Joanna Ng IBM Center

More information

On Tackling Virtual Data Center Embedding Problem

On Tackling Virtual Data Center Embedding Problem On Tackling Virtual Data Center Embedding Problem Md Golam Rabbani, Rafael Esteves, Maxim Podlesny, Gwendal Simon Lisandro Zambenedetti Granville, Raouf Boutaba D.R. Cheriton School of Computer Science,

More information

IPOP-TinCan: User-defined IP-over-P2P Virtual Private Networks

IPOP-TinCan: User-defined IP-over-P2P Virtual Private Networks IPOP-TinCan: User-defined IP-over-P2P Virtual Private Networks Renato Figueiredo Advanced Computing and Information Systems Lab University of Florida ipop-project.org Unit 3: Intra-cloud Virtual Networks

More information

Security and Billing for Azure Pack. Presented by 5nine Software and Cloud Cruiser

Security and Billing for Azure Pack. Presented by 5nine Software and Cloud Cruiser Security and Billing for Azure Pack Presented by 5nine Software and Cloud Cruiser Meet our Speakers Symon Perriman VP of Business Development 5nine Software symon@5nine.com @SymonPerriman Paul Zinn Senior

More information

Xiaoqiao Meng, Vasileios Pappas, Li Zhang IBM T.J. Watson Research Center Presented by: Payman Khani

Xiaoqiao Meng, Vasileios Pappas, Li Zhang IBM T.J. Watson Research Center Presented by: Payman Khani Improving the Scalability of Data Center Networks with Traffic-aware Virtual Machine Placement Xiaoqiao Meng, Vasileios Pappas, Li Zhang IBM T.J. Watson Research Center Presented by: Payman Khani Overview:

More information

Software-Defined Networking Architecture Framework for Multi-Tenant Enterprise Cloud Environments

Software-Defined Networking Architecture Framework for Multi-Tenant Enterprise Cloud Environments Software-Defined Networking Architecture Framework for Multi-Tenant Enterprise Cloud Environments Aryan TaheriMonfared Department of Electrical Engineering and Computer Science University of Stavanger

More information

Ant Rowstron. Joint work with Paolo Costa, Austin Donnelly and Greg O Shea Microsoft Research Cambridge. Hussam Abu-Libdeh, Simon Schubert Interns

Ant Rowstron. Joint work with Paolo Costa, Austin Donnelly and Greg O Shea Microsoft Research Cambridge. Hussam Abu-Libdeh, Simon Schubert Interns Ant Rowstron Joint work with Paolo Costa, Austin Donnelly and Greg O Shea Microsoft Research Cambridge Hussam Abu-Libdeh, Simon Schubert Interns Thinking of large-scale data centers Microsoft, Google,

More information

Improving MapReduce Performance in Heterogeneous Environments

Improving MapReduce Performance in Heterogeneous Environments UC Berkeley Improving MapReduce Performance in Heterogeneous Environments Matei Zaharia, Andy Konwinski, Anthony Joseph, Randy Katz, Ion Stoica University of California at Berkeley Motivation 1. MapReduce

More information

Cloud Optimize Your IT

Cloud Optimize Your IT Cloud Optimize Your IT Windows Server 2012 The information contained in this presentation relates to a pre-release product which may be substantially modified before it is commercially released. This pre-release

More information

Data Center Virtualization and Cloud QA Expertise

Data Center Virtualization and Cloud QA Expertise Data Center Virtualization and Cloud QA Expertise Highlights Broad Functional QA Experience Deep understanding of Switching and Routing Protocols Strong hands on experience in multiple hyper-visors like

More information

Webpage: www.ijaret.org Volume 3, Issue XI, Nov. 2015 ISSN 2320-6802

Webpage: www.ijaret.org Volume 3, Issue XI, Nov. 2015 ISSN 2320-6802 An Effective VM scheduling using Hybrid Throttled algorithm for handling resource starvation in Heterogeneous Cloud Environment Er. Navdeep Kaur 1 Er. Pooja Nagpal 2 Dr.Vinay Guatum 3 1 M.Tech Student,

More information

Flexible Building Blocks for Software Defined Network Function Virtualization (Tenant-Programmable Virtual Networks)

Flexible Building Blocks for Software Defined Network Function Virtualization (Tenant-Programmable Virtual Networks) Flexible Building Blocks for Software Defined Network Function Virtualization (Tenant-Programmable Virtual Networks) Aryan TaheriMonfared Chunming Rong Department of Electrical Engineering and Computer

More information

Scaling IP Mul-cast on Datacenter Topologies. Xiaozhou Li Mike Freedman

Scaling IP Mul-cast on Datacenter Topologies. Xiaozhou Li Mike Freedman Scaling IP Mul-cast on Datacenter Topologies Xiaozhou Li Mike Freedman IP Mul0cast Applica0ons Publish- subscribe services Clustered applica0ons servers Distributed caching infrastructures IP Mul0cast

More information

Scaling up Applications over Distributed Clouds with Dynamic Layer-2 Exchange and Broadcast Service

Scaling up Applications over Distributed Clouds with Dynamic Layer-2 Exchange and Broadcast Service Scaling up Applications over Distributed Clouds with Dynamic Layer-2 Exchange and Broadcast Service Yufeng Xin, Ilya Baldin, Chris Heermann, Anirban Mandal, and Paul Ruth Renci, University of North Carolina

More information

BM 465E Distributed Systems

BM 465E Distributed Systems BM 465E Distributed Systems Lecture 4 Networking (cont.) Mehmet Demirci Today Overlay networks Data centers Content delivery networks Overlay Network A virtual network built on top of another network Overlay

More information

Sistemi Operativi e Reti. Cloud Computing

Sistemi Operativi e Reti. Cloud Computing 1 Sistemi Operativi e Reti Cloud Computing Facoltà di Scienze Matematiche Fisiche e Naturali Corso di Laurea Magistrale in Informatica Osvaldo Gervasi ogervasi@computer.org 2 Introduction Technologies

More information

Networking Issues For Big Data

Networking Issues For Big Data Networking Issues For Big Data. Raj Jain Washington University in Saint Louis Saint Louis, MO 63130 Jain@cse.wustl.edu These slides and audio/video recordings of this class lecture are at: http://www.cse.wustl.edu/~jain/cse570-13/

More information

CS 695 Topics in Virtualization and Cloud Computing. Introduction

CS 695 Topics in Virtualization and Cloud Computing. Introduction CS 695 Topics in Virtualization and Cloud Computing Introduction This class What does virtualization and cloud computing mean? 2 Cloud Computing The in-vogue term Everyone including his/her dog want something

More information

CS 695 Topics in Virtualization and Cloud Computing and Storage Systems. Introduction

CS 695 Topics in Virtualization and Cloud Computing and Storage Systems. Introduction CS 695 Topics in Virtualization and Cloud Computing and Storage Systems Introduction Hot or not? source: Gartner Hype Cycle for Emerging Technologies, 2014 2 Source: http://geekandpoke.typepad.com/ 3 Cloud

More information

What is SDN all about?

What is SDN all about? What is SDN all about? Emil Gągała Juniper Networks Piotr Jabłoński Cisco Systems In the beginning there was a chaos CLOUD BUILDING BLOCKS CAN I VIRTUALIZE MY Compute Network? Storage Where is my money?

More information

SINGLE-TOUCH ORCHESTRATION FOR PROVISIONING, END-TO-END VISIBILITY AND MORE CONTROL IN THE DATA CENTER

SINGLE-TOUCH ORCHESTRATION FOR PROVISIONING, END-TO-END VISIBILITY AND MORE CONTROL IN THE DATA CENTER SINGLE-TOUCH ORCHESTRATION FOR PROVISIONING, END-TO-END VISIBILITY AND MORE CONTROL IN THE DATA CENTER JOINT SDN SOLUTION BY ALCATEL-LUCENT ENTERPRISE AND NEC APPLICATION NOTE EXECUTIVE SUMMARY Server

More information

Networking Architectures for Big-Data Applications

Networking Architectures for Big-Data Applications Networking Architectures for Big-Data Applications Raj Jain Washington University in Saint Louis Saint Louis, MO 63130 Jain@cse.wustl.edu Eighth Annual Microsoft Research Networking Summit Woodinville,

More information

Building Scalable Multi-Tenant Cloud Networks with OpenFlow and OpenStack

Building Scalable Multi-Tenant Cloud Networks with OpenFlow and OpenStack Building Scalable Multi-Tenant Cloud Networks with OpenFlow and OpenStack Dave Tucker Hewlett-Packard April 2013 1 About Me Dave Tucker WW Technical Marketing HP Networking dave.j.tucker@hp.com Twitter:

More information

Paolo Costa costa@imperial.ac.uk

Paolo Costa costa@imperial.ac.uk joint work with Ant Rowstron, Austin Donnelly, and Greg O Shea (MSR Cambridge) Hussam Abu-Libdeh, Simon Schubert (Interns) Paolo Costa costa@imperial.ac.uk Paolo Costa CamCube - Rethinking the Data Center

More information

Cloud computing. Examples

Cloud computing. Examples Cloud computing Cloud computing Web Systems and Algorithms Cloud Computing Chris Brooks Department of Computer Science University of San Francisco What is cloud computing? What separates it from: grid

More information

Network-Aware Scheduling of MapReduce Framework on Distributed Clusters over High Speed Networks

Network-Aware Scheduling of MapReduce Framework on Distributed Clusters over High Speed Networks Network-Aware Scheduling of MapReduce Framework on Distributed Clusters over High Speed Networks Praveenkumar Kondikoppa, Chui-Hui Chiu, Cheng Cui, Lin Xue and Seung-Jong Park Department of Computer Science,

More information

Software-Defined Networks Powered by VellOS

Software-Defined Networks Powered by VellOS WHITE PAPER Software-Defined Networks Powered by VellOS Agile, Flexible Networking for Distributed Applications Vello s SDN enables a low-latency, programmable solution resulting in a faster and more flexible

More information

Resource Provisioning of Web Applications in Heterogeneous Cloud. Jiang Dejun Supervisor: Guillaume Pierre 2011-04-10

Resource Provisioning of Web Applications in Heterogeneous Cloud. Jiang Dejun Supervisor: Guillaume Pierre 2011-04-10 Resource Provisioning of Web Applications in Heterogeneous Cloud Jiang Dejun Supervisor: Guillaume Pierre -04-10 Background Cloud is an attractive hosting platform for startup Web applications On demand

More information

Zadara Storage Cloud A whitepaper. @ZadaraStorage

Zadara Storage Cloud A whitepaper. @ZadaraStorage Zadara Storage Cloud A whitepaper @ZadaraStorage Zadara delivers two solutions to its customers: On- premises storage arrays Storage as a service from 31 locations globally (and counting) Some Zadara customers

More information

Panel : Future Data Center Networks

Panel : Future Data Center Networks Vijoy Pandey, Ph.D. CTO, Network IBM Distinguished Engineer vijoy.pandey@us.ibm.com Panel : Future Data Center Networks 2012 IBM Corporation Networking folks were poor Custom silicon or poor functionality

More information

Windows Server 2008 R2 Hyper-V Live Migration

Windows Server 2008 R2 Hyper-V Live Migration Windows Server 2008 R2 Hyper-V Live Migration Table of Contents Overview of Windows Server 2008 R2 Hyper-V Features... 3 Dynamic VM storage... 3 Enhanced Processor Support... 3 Enhanced Networking Support...

More information

Network Virtualization and Application Delivery Using Software Defined Networking

Network Virtualization and Application Delivery Using Software Defined Networking Network Virtualization and Application Delivery Using Software Defined Networking Project Leader: Subharthi Paul Washington University in Saint Louis Saint Louis, MO 63130 Jain@cse.wustl.edu Keynote at

More information

Installing Intercloud Fabric Firewall

Installing Intercloud Fabric Firewall This chapter contains the following sections: Information About the Intercloud Fabric Firewall, page 1 Prerequisites, page 1 Guidelines and Limitations, page 2 Basic Topology, page 2 Intercloud Fabric

More information

Testing Software Defined Network (SDN) For Data Center and Cloud VERYX TECHNOLOGIES

Testing Software Defined Network (SDN) For Data Center and Cloud VERYX TECHNOLOGIES Testing Software Defined Network (SDN) For Data Center and Cloud VERYX TECHNOLOGIES Table of Contents Introduction... 1 SDN - An Overview... 2 SDN: Solution Layers and its Key Requirements to be validated...

More information

Network Virtualization

Network Virtualization Network Virtualization What is Network Virtualization? Abstraction of the physical network Support for multiple logical networks running on a common shared physical substrate A container of network services

More information

Multi-Datacenter Replication

Multi-Datacenter Replication www.basho.com Multi-Datacenter Replication A Technical Overview & Use Cases Table of Contents Table of Contents... 1 Introduction... 1 How It Works... 1 Default Mode...1 Advanced Mode...2 Architectural

More information

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Solution Brief Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Introduction Accelerating time to market, increasing IT agility to enable business strategies, and improving

More information

Datacenters and Cloud Computing. Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html

Datacenters and Cloud Computing. Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html Datacenters and Cloud Computing Jia Rao Assistant Professor in CS http://cs.uccs.edu/~jrao/cs5540/spring2014/index.html What is Cloud Computing? A model for enabling ubiquitous, convenient, ondemand network

More information

Cloud Design and Implementation. Cheng Li MPI-SWS Nov 9 th, 2010

Cloud Design and Implementation. Cheng Li MPI-SWS Nov 9 th, 2010 Cloud Design and Implementation Cheng Li MPI-SWS Nov 9 th, 2010 1 Modern Computing CPU, Mem, Disk Academic computation Chemistry, Biology Large Data Set Analysis Online service Shopping Website Collaborative

More information

All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman.

All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman. WHITE PAPER All-Flash Arrays Weren t Built for Dynamic Environments. Here s Why... This whitepaper is based on content originally posted at www.frankdenneman.nl 1 Monolithic shared storage architectures

More information

International Journal of Scientific & Engineering Research, Volume 6, Issue 3, March-2015 575 ISSN 2229-5518

International Journal of Scientific & Engineering Research, Volume 6, Issue 3, March-2015 575 ISSN 2229-5518 International Journal of Scientific & Engineering Research, Volume 6, Issue 3, March-2015 575 Simulation-Based Approaches For Evaluating Load Balancing In Cloud Computing With Most Significant Broker Policy

More information

: Tiering Storage for Data Analytics in the Cloud

: Tiering Storage for Data Analytics in the Cloud : Tiering Storage for Data Analytics in the Cloud Yue Cheng, M. Safdar Iqbal, Aayush Gupta, Ali R. Butt Virginia Tech, IBM Research Almaden Cloud enables cost-efficient data analytics Amazon EMR Cloud

More information

Stuart Berman, CEO Jeda Networks September, 2013

Stuart Berman, CEO Jeda Networks September, 2013 Stuart Berman, CEO Jeda Networks September, 2013 Emerging Cloud model Resource pooling, dynamic allocation of resources, multi-tenant, software deployments/automation/management Consumer cloud companies

More information

SDN: A NEW PARADIGM. Kireeti Kompella CTO, JDI

SDN: A NEW PARADIGM. Kireeti Kompella CTO, JDI SDN: A NEW PARADIGM Kireeti Kompella CTO, JDI AGENDA What is SDN? Definition and goals of SDN Analogy with Compute Virtualization Orchestration for Agile Provisioning Unified SDN What parts of the network

More information

Introduction to OpenStack

Introduction to OpenStack Introduction to OpenStack Carlo Vallati PostDoc Reseracher Dpt. Information Engineering University of Pisa carlo.vallati@iet.unipi.it Cloud Computing - Definition Cloud Computing is a term coined to refer

More information

MS-20246: Monitoring and Operating a Private Cloud

MS-20246: Monitoring and Operating a Private Cloud MS-20246: Monitoring and Operating a Private Cloud Description This course describes how to monitor and operate a cloud with Microsoft System Center 2012 R2. This course focuses on how to manage and administer

More information

Cloud Computing Architecture: A Survey

Cloud Computing Architecture: A Survey Cloud Computing Architecture: A Survey Abstract Now a day s Cloud computing is a complex and very rapidly evolving and emerging area that affects IT infrastructure, network services, data management and

More information

CLOUD COMPUTING. When It's smarter to rent than to buy

CLOUD COMPUTING. When It's smarter to rent than to buy CLOUD COMPUTING When It's smarter to rent than to buy Is it new concept? Nothing new In 1990 s, WWW itself Grid Technologies- Scientific applications Online banking websites More convenience Not to visit

More information

CloudLink - The On-Ramp to the Cloud Security, Management and Performance Optimization for Multi-Tenant Private and Public Clouds

CloudLink - The On-Ramp to the Cloud Security, Management and Performance Optimization for Multi-Tenant Private and Public Clouds - The On-Ramp to the Cloud Security, Management and Performance Optimization for Multi-Tenant Private and Public Clouds February 2011 1 Introduction Today's business environment requires organizations

More information

Virtualization of the MS Exchange Server Environment

Virtualization of the MS Exchange Server Environment MS Exchange Server Acceleration Maximizing Users in a Virtualized Environment with Flash-Powered Consolidation Allon Cohen, PhD OCZ Technology Group Introduction Microsoft (MS) Exchange Server is one of

More information

Network Virtualization for the Enterprise Data Center. Guido Appenzeller Open Networking Summit October 2011

Network Virtualization for the Enterprise Data Center. Guido Appenzeller Open Networking Summit October 2011 Network Virtualization for the Enterprise Data Center Guido Appenzeller Open Networking Summit October 2011 THE ENTERPRISE DATA CENTER! Major Trends change Enterprise Data Center Networking Trends in the

More information

OpenFlow/SDN for IaaS Providers

OpenFlow/SDN for IaaS Providers OpenFlow/SDN for IaaS Providers Open Networking Summit 2011 Stanford University Paul Lappas & Ivan Batanov The Public Cloud Our Definition Shared infrastructure operated by a service provider where no

More information

Cost-effective Resource Provisioning for MapReduce in a Cloud

Cost-effective Resource Provisioning for MapReduce in a Cloud 1 -effective Resource Provisioning for MapReduce in a Cloud Balaji Palanisamy, Member, IEEE, Aameek Singh, Member, IEEE Ling Liu, Senior Member, IEEE Abstract This paper presents a new MapReduce cloud

More information

ProgrammableFlow for Open Virtualized Data Center Network

ProgrammableFlow for Open Virtualized Data Center Network ProgrammableFlow for Open Virtualized Data Center Network Samrat Ganguly NEC Corporation of America Challenges in Today s Data Center and Cloud Networks Performance scaling concerns Complexity scaling

More information

Virtualizing Apache Hadoop. June, 2012

Virtualizing Apache Hadoop. June, 2012 June, 2012 Table of Contents EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 VIRTUALIZING APACHE HADOOP... 4 INTRODUCTION TO VSPHERE TM... 4 USE CASES AND ADVANTAGES OF VIRTUALIZING HADOOP... 4 MYTHS ABOUT RUNNING

More information

Choreo: Network-Aware Task Placement for Cloud Applications

Choreo: Network-Aware Task Placement for Cloud Applications Choreo: Network-Aware Task Placement for Cloud Applications Katrina LaCurts, Shuo Deng, Ameesh Goyal, Hari Balakrishnan MIT Computer Science and Artificial Intelligence Lab Cambridge, Massachusetts, U.S.A.

More information

Data Center Manager (DCM)

Data Center Manager (DCM) DATA SHEET Data Center Manager (DCM) Unified Virtual/Physical Data Center Fabric Management Benefits LOWER OPERATIONAL COSTS High degree of automation within physical and virtual environments to streamline

More information

A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems

A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems Danilo Ardagna 1, Barbara Panicucci 1, Mauro Passacantando 2 1 Politecnico di Milano,, Italy 2 Università di Pisa, Dipartimento

More information

Figure 1. The cloud scales: Amazon EC2 growth [2].

Figure 1. The cloud scales: Amazon EC2 growth [2]. - Chung-Cheng Li and Kuochen Wang Department of Computer Science National Chiao Tung University Hsinchu, Taiwan 300 shinji10343@hotmail.com, kwang@cs.nctu.edu.tw Abstract One of the most important issues

More information

Configuring and Deploying a Private Cloud

Configuring and Deploying a Private Cloud Course Code: M20247 Vendor: Microsoft Course Overview Duration: 5 RRP: 2,025 Configuring and Deploying a Private Cloud Overview This course equips students with the skills they require to configure and

More information

Big Data Trends and HDFS Evolution

Big Data Trends and HDFS Evolution Big Data Trends and HDFS Evolution Sanjay Radia Founder & Architect Hortonworks Inc Page 1 Hello Founder, Hortonworks Part of the Hadoop team at Yahoo! since 2007 Chief Architect of Hadoop Core at Yahoo!

More information

NCTA Cloud Architecture

NCTA Cloud Architecture NCTA Cloud Architecture Course Specifications Course Number: 093019 Course Length: 5 days Course Description Target Student: This course is designed for system administrators who wish to plan, design,

More information