# Dominant Resource Fairness: Fair Allocation of Multiple Resource Types

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Dominant Resource Fairness: Fair Allocation of Multiple Resource Types Ali Ghodsi Matei Zaharia Benjamin Hindman Andrew Konwinski Scott Shenker Ion Stoica Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS March 6, 2011

8 microeconomic domain [21, 29, 32]. We compare these policies with DRF in Section Asset Fairness The idea behind Asset Fairness is that equal shares of different resources are worth the same, i.e., that 1% of all CPUs worth is the same as 1% of memory and 1% of I/O bandwidth. Asset Fairness then tries to equalize the aggregate resource value allocated to each user. In particular, Asset Fairness computes for each user i the aggregate share x i = j s i,j, where s i,j is the share of resource j given to user i. It then applies max-min across users aggregate shares, i.e., it repeatedly launches tasks for the user with the minimum aggregate share. Consider the example in Section 4.1. Since there are twice as many GB of RAM as CPUs (i.e., 9 CPUs and 18 GB RAM), one CPU is worth twice as much as one GB of RAM. Supposing that one GB is worth \$1 and one CPU is worth \$2, it follows that user A spends \$6 for each task, while user B spends \$7. Let x and y be the number of tasks allocated by Asset Fairness to users A and B, respectively. Then the asset-fair allocation is given by the solution to the following optimization problem: max (x, y) subject to (Maximize allocations) x + 3y 9 (CPU constraint) 4x + y 18 (Memory constraint) 6x = 7y (Every user spends the same) Solving the above problem yields x = 2.52 and y = Thus, user A gets 2.5 CPUs, 10.1 GB, while user B gets 6.5 CPUs, 2.2 GB, respectively. While this allocation policy seems compelling in its simplicity, it has a significant drawback: it violates the sharing incentive property. As we show in Section 6.3, asset fairness can result in one user getting less than 1/n of all resources, where n is the total number of users. 5.2 Competitive Equilibrium from Equal Incomes In microeconomic theory, the preferred method to fairly divide resources is Competitive Equilibrium from Equal Incomes (CEEI) [21, 29, 32]. With CEEI, each user receives initially 1 n of every resource, and subsequently, each user trades her resources with other users in a perfectly competitive market. 3 The outcome of CEEI is both envy-free and Pareto efficient [29]. 3 A perfect market satisfies the price-taking (i.e., no single user affects prices) and market-clearance (i.e., matching supply and demand via price adjustment) assumptions. 100% 50% 0% User A 100% 50% User B 0% 0% CPU Mem CPU Mem CPU Mem a) DRF b) Asset Fairness 100% 50% c) CEEI Figure 4: Allocations given by DRF, Asset Fairness and CEEI in the example scenario in Section 4.1. More precisely, the CEEI allocation is given by the Nash bargaining solution 4 [21, 22]. The Nash bargaining solution picks the feasible allocation that maximizes i u i(a i ), where u i (a i ) is the utility that user i gets from her allocation a i. To simplify the comparison, we assume that the utility that a user gets from her allocation is simply her dominant share, s i. Consider again the two-user example in Section 4.1. Recall that the dominant share of user A is 4x/18 = 2x/9 while the dominant share of user B is 3y/9 = y/3, where x is the number of tasks given to A and y is the number of tasks given to B. Maximizing the product of the dominant shares is equivalent to maximizing the product x y. Thus, CEEI aims to solve the following optimization problem: max (x y) subject to (maximize Nash product) x + 3y 9 (CPU constraint) 4x + y 18 (Memory constraint) Solving the above problem yields x = 45/11 and y = 18/11. Thus, user A gets 4.1 CPUs, 16.4 GB, while user B gets 4.9 CPUs, 1.6 GB. Unfortunately, while CEEI is envy-free and Pareto efficient, it turns out that it is not strategy-proof, as we will show in Section 6.4. Thus, users can increase their allocations by lying about their resource demands. 5.3 Comparison with DRF To give the reader an intuitive understanding of Asset Fairness and CEEI, we compare their allocations for the example in Section 4.1 to that of DRF in Figure 4. 4 For this to hold, utilities have to be homogeneous, i.e., u(α x) = α u(x) for α > 0, which is true in our case. 6

10 dominant share of s 1 = s 2 = 0.6. As a side note, none of the dominant resources are fully utilized in this example. 6.2 Properties of DRF We start with a preliminary result. Lemma 2 Every user in a DRF allocation has at least one saturated resource. Proof Assume this is not the case, i.e., none of the resources used by user i is saturated. However, this contradicts the assumption that progressive filling has completed the computation of the DRF allocation. Indeed, as long as none of the resources of user i are saturated, progressive filling will continue to increase the allocations of user i (and of all the other users sharing only non-saturated resources). Recall that progressive filling always allocates the resources to a user proportionally to the user s demand vector. More precisely, let D i = d i,1, d i,2,..., d i,m be the demand vector of user i. Then, at any time t during the progressive filling process, the allocation of user i is proportional to the demand vector, A i (t) = α i (t) D i = α i (t) d i,1, d i,2,..., d i,m (1) where α i (t) is a positive scalar. Now, we are in position to prove the DRF properties. Theorem 3 DRF is Pareto efficient. Proof Assume user i can increase her dominant share, s i, without decreasing the dominant share of any other user j. According to Lemma 2, user i has at least one saturated resource. If no other user is using the saturated resource, then we are done as it would be impossible to increase i s share of the saturated resource. If other users are using the saturated resource, then increasing the allocation of i would result in decreasing the allocation of at least another user j sharing the same saturated resource. Since under progressive filling, the resources allocated by any user are proportional to her demand vector (see Eq. 1), decreasing the allocation of any resource used by user i will also decrease i s dominant share. This contradicts our hypothesis, and therefore proves the result. Theorem 4 DRF satisfies the sharing incentive and bottleneck fairness properties. Proof Consider a system consisting of n users. Assume resource k is the first one being saturated by using progressive filling. Let i be the user allocating the largest share on resource k, and let t i,k denote her share of k. Since resource k is saturated, we have trivially t i,k 1 n. Furthermore, by the definition of the dominant share, we have s i t i,k 1 n. Since progressive filling increases the allocation of each user s dominant resource at the same rate, it follows that each user gets at least 1 n of her dominant resource. Thus, DRF satisfies the sharing incentive property. If all users have the same dominant resource, each user gets exactly 1 n of that resource. As a result, DRF satisfies the bottleneck fairness property as well. Theorem 5 Every DRF allocation is envy-free. Proof For a user i to envy another user j, j must have a strictly higher share of every resource that i wants; otherwise i cannot run more tasks under j s allocation. From the max-min theorem it follows that i must have a bottleneck resource R k and none of the users using R k have a higher dominant share than i. However, this is impossible as j has a higher share of R k than i and must therefore have a higher dominant share than i. Thus, i cannot envy any other user. Theorem 6 (Strategy-proofness) A user cannot increase her dominant share in DRF by altering her true demand vector. Proof Assume user i can increase her dominant share by using a demand vector ˆd i d i. Let a i,j and â i,j denote the amount of resource j user i is allocated using progressive filling when the user uses the vector d i and ˆd i, respectively. For user i to be better off using ˆd i, we need that â i,k > a i,k for every resource k where d i,k > 0. Let r denote the first resource that becomes saturated for user i when she uses the demand vector d i. If no other user is allocated resource r (a j,r = 0 for all j i), this contradicts the hypothesis as user i is already allocated the entire resource r, and thus cannot increase her allocation of r using another demand vector ˆd i. Thus, assume there are other users that have been allocated r (a j,r > 0 for some j i). In this case, progressive filling will eventually saturate r at time t when using d i, and at time t when using demand ˆd i. Recall that the dominant share is the maximum of a user s shares, thus i must have a higher dominant share in the allocation â than in a. Thus, t > t, as progressive filling increases the dominant share at a constant rate. This implies that i when using ˆd does not saturate any resource before time t, and hence does not affect other user s allocation before time t. Thus, when i uses ˆd, any user m using resource r has allocation a m,r at time t. Therefore, at time t, there is only a i,r amount of r left for user i, which contradicts the assumption that â i,r > a i,r. The strategy-proofness of DRF shows that a user will not be better off by demanding resources that she does 8

18 [15] B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. H. Katz, S. Shenker, and I. Stoica. Mesos: A platform for fine-grained resource sharing in the data center. In NSDI, [16] M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In EuroSys 07, [17] M. Isard, V. Prabhakaran, J. Currey, U. Wieder, K. Talwar, and A. Goldberg. Quincy: Fair scheduling for distributed computing clusters. In SOSP 09, [18] E. Kalai and M. Smorodinsky. Other Solutions to Nash s Bargaining Problem. Econometrica, 43(3): , [19] J. M. Kleinberg, Y. Rabani, and É. Tardos. Fairness in routing and load balancing. J. Comput. Syst. Sci., 63(1):2 20, [20] Y. Liu and E. W. Knightly. Opportunistic fair scheduling over multiple wireless channels. In IN- FOCOM, [21] H. Moulin. Fair Division and Collective Welfare. The MIT Press, [22] J. Nash. The Bargaining Problem. Econometrica, 18(2): , April [23] A. Parekh and R. Gallager. A generalized processor sharing approach to flow control - the single node case. ACM/IEEE Transactions on Networking, 1(3): , June [24] E. A. Pazner and D. Schmeidler. Egalitarian equivalent allocations: A new concept of economic equity. Quarterly Journal of Economics, 92: , [25] A. Sen. Rawls Versus Bentham: An Axiomatic Examination of the Pure Distribution Problem. Theory and Decision, 4(1): , [26] M. Shreedhar and G. Varghese. Efficient fair queuing using deficit round robin. IEEE Trans. Net, [27] I. Stoica, H. Abdel-Wahab, K. Jeffay, S. Baruah, J. Gehrke, and G. Plaxton. A proportional share resource allocation algorithm for real-time, timeshared systems. In IEEE RTSS 96, [28] I. Stoica, S. Shenker, and H. Zhang. Core-stateless fair queueing: Achieving approximately fair bandwidth allocations in high speed networks. In SIG- COMM, [29] H. Varian. Equity, envy, and efficiency. Journal of Economic Theory, 9(1):63 91, [30] C. A. Waldspurger. Lottery and Stride Scheduling: Flexible Proportional Share Resource Management. PhD thesis, MIT, Laboratory of Computer Science, Sept MIT/LCS/TR-667. [31] C. A. Waldspurger and W. E. Weihl. Lottery scheduling: flexible proportional-share resource management. In OSDI 94, [32] H. P. Young. Equity: in theory and practice. Princeton University Press, [33] M. Zaharia, D. Borthakur, J. Sen Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling. In EuroSys 10, [34] D. Zhu, D. Mossé, and R. G. Melhem. Multiple- Resource Periodic Scheduling Problem: how much fairness is necessary? In IEEE RTSS,

### Manipulability of the Price Mechanism for Data Centers

Manipulability of the Price Mechanism for Data Centers Greg Bodwin 1, Eric Friedman 2,3,4, and Scott Shenker 3,4 1 Department of Computer Science, Tufts University, Medford, Massachusetts 02155 2 School

### Dominant Resource Fairness in Cloud Computing Systems with Heterogeneous Servers

IEEE INFOCOM 24 - IEEE Conference on Computer Communications Dominant Resource Fairness in Cloud Computing Systems with Heterogeneous Servers Wei Wang, Baochun Li, Ben Liang Department of Electrical and

### Chapter 7. Sealed-bid Auctions

Chapter 7 Sealed-bid Auctions An auction is a procedure used for selling and buying items by offering them up for bid. Auctions are often used to sell objects that have a variable price (for example oil)

### Mesos: A Platform for Fine- Grained Resource Sharing in Data Centers (II)

UC BERKELEY Mesos: A Platform for Fine- Grained Resource Sharing in Data Centers (II) Anthony D. Joseph LASER Summer School September 2013 My Talks at LASER 2013 1. AMP Lab introduction 2. The Datacenter

### Improving MapReduce Performance in Heterogeneous Environments

UC Berkeley Improving MapReduce Performance in Heterogeneous Environments Matei Zaharia, Andy Konwinski, Anthony Joseph, Randy Katz, Ion Stoica University of California at Berkeley Motivation 1. MapReduce

### Introduction to Apache YARN Schedulers & Queues

Introduction to Apache YARN Schedulers & Queues In a nutshell, YARN was designed to address the many limitations (performance/scalability) embedded into Hadoop version 1 (MapReduce & HDFS). Some of the

IMPROVED FAIR SCHEDULING ALGORITHM FOR TASKTRACKER IN HADOOP MAP-REDUCE Mr. Santhosh S 1, Mr. Hemanth Kumar G 2 1 PG Scholor, 2 Asst. Professor, Dept. Of Computer Science & Engg, NMAMIT, (India) ABSTRACT

### Market-based Mechanisms for Resource Scheduling

Market-based Mechanisms for Resource Scheduling Paul Christiano, Frank Li, Richard Shin {paulfchristiano, frankli, ricshin}@cs.berkeley.edu ABSTRACT Existing cluster managers typically allocate resources

### Unraveling versus Unraveling: A Memo on Competitive Equilibriums and Trade in Insurance Markets

Unraveling versus Unraveling: A Memo on Competitive Equilibriums and Trade in Insurance Markets Nathaniel Hendren January, 2014 Abstract Both Akerlof (1970) and Rothschild and Stiglitz (1976) show that

### Performance of networks containing both MaxNet and SumNet links

Performance of networks containing both MaxNet and SumNet links Lachlan L. H. Andrew and Bartek P. Wydrowski Abstract Both MaxNet and SumNet are distributed congestion control architectures suitable for

### Pricing the Cloud: Resource Allocations, Fairness, and Revenue

Pricing the Cloud: Resource Allocations, Fairness, and Revenue Carlee Joe-Wong and Soumya Sen Princeton University and University of Minnesota Emails: coe@princeton.edu, ssen@umn.edu Abstract As more businesses

### X Mathematical Frameworks for Pricing in the Cloud: Revenue, Fairness, and Resource Allocations

X Mathematical Frameworks for Pricing in the Cloud: Revenue, Fairness, and Resource Allocations CARLEE JOE-WONG, Princeton University SOUMYA SEN, Princeton University As more and more users begin to use

### Multi-Resource Fair Queueing for Packet Processing

Multi-Resource Fair Queueing for Packet Processing Ali Ghodsi,, Vyas Sekar, Matei Zaharia, Ion Stoica University of California, Berkeley Intel ISTC KTH/Royal Institute of Technology {alig, matei, istoica}@cs.berkeley.edu,

### Job Scheduling for MapReduce

UC Berkeley Job Scheduling for MapReduce Matei Zaharia, Dhruba Borthakur *, Joydeep Sen Sarma *, Scott Shenker, Ion Stoica RAD Lab, * Facebook Inc 1 Motivation Hadoop was designed for large batch jobs

### Fairness in Routing and Load Balancing

Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria

### Long-Term Resource Fairness

Long-Term Resource Fairness Towards Economic Fairness on Pay-as-you-use Computing Systems Shanjiang Tang, u-sung Lee, ingsheng He, Haikun Liu School of Computer Engineering Nanyang Technological University

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Delay Scheduling. A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling

Delay Scheduling A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling Matei Zaharia, Dhruba Borthakur *, Joydeep Sen Sarma *, Khaled Elmeleegy +, Scott Shenker, Ion Stoica UC Berkeley,

### A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems

A Hybrid Scheduling Approach for Scalable Heterogeneous Hadoop Systems Aysan Rasooli Department of Computing and Software McMaster University Hamilton, Canada Email: rasooa@mcmaster.ca Douglas G. Down

### Heterogeneity-Aware Resource Allocation and Scheduling in the Cloud

Heterogeneity-Aware Resource Allocation and Scheduling in the Cloud Gunho Lee, Byung-Gon Chun, Randy H. Katz University of California, Berkeley, Yahoo! Research Abstract Data analytics are key applications

### Benchmarking Hadoop & HBase on Violin

Technical White Paper Report Technical Report Benchmarking Hadoop & HBase on Violin Harnessing Big Data Analytics at the Speed of Memory Version 1.0 Abstract The purpose of benchmarking is to show advantages

### arxiv:1112.0829v1 [math.pr] 5 Dec 2011

How Not to Win a Million Dollars: A Counterexample to a Conjecture of L. Breiman Thomas P. Hayes arxiv:1112.0829v1 [math.pr] 5 Dec 2011 Abstract Consider a gambling game in which we are allowed to repeatedly

### Windows Server Performance Monitoring

Spot server problems before they are noticed The system s really slow today! How often have you heard that? Finding the solution isn t so easy. The obvious questions to ask are why is it running slowly

### CPC/CPA Hybrid Bidding in a Second Price Auction

CPC/CPA Hybrid Bidding in a Second Price Auction Benjamin Edelman Hoan Soo Lee Working Paper 09-074 Copyright 2008 by Benjamin Edelman and Hoan Soo Lee Working papers are in draft form. This working paper

### Adaptive Tolerance Algorithm for Distributed Top-K Monitoring with Bandwidth Constraints

Adaptive Tolerance Algorithm for Distributed Top-K Monitoring with Bandwidth Constraints Michael Bauer, Srinivasan Ravichandran University of Wisconsin-Madison Department of Computer Sciences {bauer, srini}@cs.wisc.edu

### Guidelines for Selecting Hadoop Schedulers based on System Heterogeneity

Noname manuscript No. (will be inserted by the editor) Guidelines for Selecting Hadoop Schedulers based on System Heterogeneity Aysan Rasooli Douglas G. Down Received: date / Accepted: date Abstract Hadoop

### Algorithm Design for Performance Aware VM Consolidation

Algorithm Design for Performance Aware VM Consolidation Alan Roytman University of California, Los Angeles Sriram Govindan Microsoft Corporation Jie Liu Microsoft Research Aman Kansal Microsoft Research

### MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research

MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With

### Efficient and Robust Allocation Algorithms in Clouds under Memory Constraints

Efficient and Robust Allocation Algorithms in Clouds under Memory Constraints Olivier Beaumont,, Paul Renaud-Goud Inria & University of Bordeaux Bordeaux, France 9th Scheduling for Large Scale Systems

### Cloud Computing. Computational Tasks Have value for task completion Require resources (Cores, Memory, Bandwidth) Compete for resources

Peter Key, Cloud Computing Computational Tasks Have value for task completion Require resources (Cores, Memory, Bandwidth) Compete for resources How much is a task or resource worth Can we use to price

### 17.6.1 Introduction to Auction Design

CS787: Advanced Algorithms Topic: Sponsored Search Auction Design Presenter(s): Nilay, Srikrishna, Taedong 17.6.1 Introduction to Auction Design The Internet, which started of as a research project in

### Computing Load Aware and Long-View Load Balancing for Cluster Storage Systems

215 IEEE International Conference on Big Data (Big Data) Computing Load Aware and Long-View Load Balancing for Cluster Storage Systems Guoxin Liu and Haiying Shen and Haoyu Wang Department of Electrical

### Moral Hazard. Itay Goldstein. Wharton School, University of Pennsylvania

Moral Hazard Itay Goldstein Wharton School, University of Pennsylvania 1 Principal-Agent Problem Basic problem in corporate finance: separation of ownership and control: o The owners of the firm are typically

### A Simple Characterization for Truth-Revealing Single-Item Auctions

A Simple Characterization for Truth-Revealing Single-Item Auctions Kamal Jain 1, Aranyak Mehta 2, Kunal Talwar 3, and Vijay Vazirani 2 1 Microsoft Research, Redmond, WA 2 College of Computing, Georgia

### The Truth Behind IBM AIX LPAR Performance

The Truth Behind IBM AIX LPAR Performance Yann Guernion, VP Technology EMEA HEADQUARTERS AMERICAS HEADQUARTERS Tour Franklin 92042 Paris La Défense Cedex France +33 [0] 1 47 73 12 12 info@orsyp.com www.orsyp.com

### INFO5011. Cloud Computing Semester 2, 2011 Lecture 11, Cloud Scheduling

INFO5011 Cloud Computing Semester 2, 2011 Lecture 11, Cloud Scheduling COMMONWEALTH OF Copyright Regulations 1969 WARNING This material has been reproduced and communicated to you by or on behalf of the

### MROrchestrator: A Fine-Grained Resource Orchestration Framework for Hadoop MapReduce

MROrchestrator: A Fine-Grained Resource Orchestration Framework for Hadoop MapReduce Bikash Sharma, Ramya Prabhakar, Seung-Hwan Lim, Mahmut T. Kandemir and Chita R. Das Department of Computer Science and

### Persuasion by Cheap Talk - Online Appendix

Persuasion by Cheap Talk - Online Appendix By ARCHISHMAN CHAKRABORTY AND RICK HARBAUGH Online appendix to Persuasion by Cheap Talk, American Economic Review Our results in the main text concern the case

Characterizing Task Usage Shapes in Google s Compute Clusters Qi Zhang University of Waterloo qzhang@uwaterloo.ca Joseph L. Hellerstein Google Inc. jlh@google.com Raouf Boutaba University of Waterloo rboutaba@uwaterloo.ca

### Energy Efficient MapReduce

Energy Efficient MapReduce Motivation: Energy consumption is an important aspect of datacenters efficiency, the total power consumption in the united states has doubled from 2000 to 2005, representing

### Dynamic Resource Pricing on Federated Clouds

Dynamic Resource Pricing on Federated Clouds Marian Mihailescu and Yong Meng Teo Department of Computer Science National University of Singapore Computing 1, 13 Computing Drive, Singapore 117417 Email:

### Summary of Doctoral Dissertation: Voluntary Participation Games in Public Good Mechanisms: Coalitional Deviations and Efficiency

Summary of Doctoral Dissertation: Voluntary Participation Games in Public Good Mechanisms: Coalitional Deviations and Efficiency Ryusuke Shinohara 1. Motivation The purpose of this dissertation is to examine

### This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

IEEE/ACM TRANSACTIONS ON NETWORKING 1 A Greedy Link Scheduler for Wireless Networks With Gaussian Multiple-Access and Broadcast Channels Arun Sridharan, Student Member, IEEE, C Emre Koksal, Member, IEEE,

### On the Interaction and Competition among Internet Service Providers

On the Interaction and Competition among Internet Service Providers Sam C.M. Lee John C.S. Lui + Abstract The current Internet architecture comprises of different privately owned Internet service providers

### Analysis of an Artificial Hormone System (Extended abstract)

c 2013. This is the author s version of the work. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purpose or for creating

### Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center

Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, Ion Stoica University of

### Storage Systems Autumn 2009. Chapter 6: Distributed Hash Tables and their Applications André Brinkmann

Storage Systems Autumn 2009 Chapter 6: Distributed Hash Tables and their Applications André Brinkmann Scaling RAID architectures Using traditional RAID architecture does not scale Adding news disk implies

### By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

This material is posted here with permission of the IEEE Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services Internal

### A Virtual Machine Consolidation Framework for MapReduce Enabled Computing Clouds

A Virtual Machine Consolidation Framework for MapReduce Enabled Computing Clouds Zhe Huang, Danny H.K. Tsang, James She Department of Electronic & Computer Engineering The Hong Kong University of Science

### Scheduling Real-time Tasks: Algorithms and Complexity

Scheduling Real-time Tasks: Algorithms and Complexity Sanjoy Baruah The University of North Carolina at Chapel Hill Email: baruah@cs.unc.edu Joël Goossens Université Libre de Bruxelles Email: joel.goossens@ulb.ac.be

### An example of a computable

An example of a computable absolutely normal number Verónica Becher Santiago Figueira Abstract The first example of an absolutely normal number was given by Sierpinski in 96, twenty years before the concept

### Managing Capacity Using VMware vcenter CapacityIQ TECHNICAL WHITE PAPER

Managing Capacity Using VMware vcenter CapacityIQ TECHNICAL WHITE PAPER Table of Contents Capacity Management Overview.... 3 CapacityIQ Information Collection.... 3 CapacityIQ Performance Metrics.... 4

Optimizing a ëcontent-aware" Load Balancing Strategy for Shared Web Hosting Service Ludmila Cherkasova Hewlett-Packard Laboratories 1501 Page Mill Road, Palo Alto, CA 94303 cherkasova@hpl.hp.com Shankar

### Report: Google Data center Scheduling

1 Research Objective Report: Google Data center Scheduling Tony T. Tran and J. Christopher Beck Department of Mechanical and Industrial Engineering, University of Toronto {tran, jcb}@mie.utoronto.ca The

### Notes V General Equilibrium: Positive Theory. 1 Walrasian Equilibrium and Excess Demand

Notes V General Equilibrium: Positive Theory In this lecture we go on considering a general equilibrium model of a private ownership economy. In contrast to the Notes IV, we focus on positive issues such

### Subsets of Euclidean domains possessing a unique division algorithm

Subsets of Euclidean domains possessing a unique division algorithm Andrew D. Lewis 2009/03/16 Abstract Subsets of a Euclidean domain are characterised with the following objectives: (1) ensuring uniqueness

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### Technical Investigation of Computational Resource Interdependencies

Technical Investigation of Computational Resource Interdependencies By Lars-Eric Windhab Table of Contents 1. Introduction and Motivation... 2 2. Problem to be solved... 2 3. Discussion of design choices...

### Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay

Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding

### Gains from Trade. Christopher P. Chambers and Takashi Hayashi. March 25, 2013. Abstract

Gains from Trade Christopher P. Chambers Takashi Hayashi March 25, 2013 Abstract In a market design context, we ask whether there exists a system of transfers regulations whereby gains from trade can always

### Cost of Conciseness in Sponsored Search Auctions

Cost of Conciseness in Sponsored Search Auctions Zoë Abrams Yahoo!, Inc. Arpita Ghosh Yahoo! Research 2821 Mission College Blvd. Santa Clara, CA, USA {za,arpita,erikvee}@yahoo-inc.com Erik Vee Yahoo! Research

### Partitioned real-time scheduling on heterogeneous shared-memory multiprocessors

Partitioned real-time scheduling on heterogeneous shared-memory multiprocessors Martin Niemeier École Polytechnique Fédérale de Lausanne Discrete Optimization Group Lausanne, Switzerland martin.niemeier@epfl.ch

### Fixed-Effect Versus Random-Effects Models

CHAPTER 13 Fixed-Effect Versus Random-Effects Models Introduction Definition of a summary effect Estimating the summary effect Extreme effect size in a large study or a small study Confidence interval

### PEPPERDATA IN MULTI-TENANT ENVIRONMENTS

..................................... PEPPERDATA IN MULTI-TENANT ENVIRONMENTS technical whitepaper June 2015 SUMMARY OF WHAT S WRITTEN IN THIS DOCUMENT If you are short on time and don t want to read the

Mechanisms for Fair Attribution Eric Balkanski Yaron Singer Abstract We propose a new framework for optimization under fairness constraints. The problems we consider model procurement where the goal is

### Sharing Online Advertising Revenue with Consumers

Sharing Online Advertising Revenue with Consumers Yiling Chen 2,, Arpita Ghosh 1, Preston McAfee 1, and David Pennock 1 1 Yahoo! Research. Email: arpita, mcafee, pennockd@yahoo-inc.com 2 Harvard University.

### A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems

A Game Theoretic Formulation of the Service Provisioning Problem in Cloud Systems Danilo Ardagna 1, Barbara Panicucci 1, Mauro Passacantando 2 1 Politecnico di Milano,, Italy 2 Università di Pisa, Dipartimento

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### Introduction to Big Data! with Apache Spark" UC#BERKELEY#

Introduction to Big Data! with Apache Spark" UC#BERKELEY# This Lecture" The Big Data Problem" Hardware for Big Data" Distributing Work" Handling Failures and Slow Machines" Map Reduce and Complex Jobs"

### CHAPTER 2 Estimating Probabilities

CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a

### Cost Effective Automated Scaling of Web Applications for Multi Cloud Services

Cost Effective Automated Scaling of Web Applications for Multi Cloud Services SANTHOSH.A 1, D.VINOTHA 2, BOOPATHY.P 3 1,2,3 Computer Science and Engineering PRIST University India Abstract - Resource allocation

### Throughput Capacity Planning and Application Saturation

Throughput Capacity Planning and Application Saturation Alfred J. Barchi ajb@ajbinc.net http://www.ajbinc.net/ Introduction Applications have a tendency to be used more heavily by users over time, as the

### each college c i C has a capacity q i - the maximum number of students it will admit

n colleges in a set C, m applicants in a set A, where m is much larger than n. each college c i C has a capacity q i - the maximum number of students it will admit each college c i has a strict order i

### A Study on Workload Imbalance Issues in Data Intensive Distributed Computing

A Study on Workload Imbalance Issues in Data Intensive Distributed Computing Sven Groot 1, Kazuo Goda 1, and Masaru Kitsuregawa 1 University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan Abstract.

### Rackspace Cloud Databases and Container-based Virtualization

Rackspace Cloud Databases and Container-based Virtualization August 2012 J.R. Arredondo @jrarredondo Page 1 of 6 INTRODUCTION When Rackspace set out to build the Cloud Databases product, we asked many

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Module 6. Embedded System Software. Version 2 EE IIT, Kharagpur 1

Module 6 Embedded System Software Version 2 EE IIT, Kharagpur 1 Lesson 30 Real-Time Task Scheduling Part 2 Version 2 EE IIT, Kharagpur 2 Specific Instructional Objectives At the end of this lesson, the

### 1. Comments on reviews a. Need to avoid just summarizing web page asks you for:

1. Comments on reviews a. Need to avoid just summarizing web page asks you for: i. A one or two sentence summary of the paper ii. A description of the problem they were trying to solve iii. A summary of

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### THE FUNDAMENTAL THEOREM OF ARBITRAGE PRICING

THE FUNDAMENTAL THEOREM OF ARBITRAGE PRICING 1. Introduction The Black-Scholes theory, which is the main subject of this course and its sequel, is based on the Efficient Market Hypothesis, that arbitrages

### Load Balancing and Switch Scheduling

EE384Y Project Final Report Load Balancing and Switch Scheduling Xiangheng Liu Department of Electrical Engineering Stanford University, Stanford CA 94305 Email: liuxh@systems.stanford.edu Abstract Load

### Joint Optimization of Overlapping Phases in MapReduce

Joint Optimization of Overlapping Phases in MapReduce Minghong Lin, Li Zhang, Adam Wierman, Jian Tan Abstract MapReduce is a scalable parallel computing framework for big data processing. It exhibits multiple

### Bargaining Solutions in a Social Network

Bargaining Solutions in a Social Network Tanmoy Chakraborty and Michael Kearns Department of Computer and Information Science University of Pennsylvania Abstract. We study the concept of bargaining solutions,

### Supplement to Call Centers with Delay Information: Models and Insights

Supplement to Call Centers with Delay Information: Models and Insights Oualid Jouini 1 Zeynep Akşin 2 Yves Dallery 1 1 Laboratoire Genie Industriel, Ecole Centrale Paris, Grande Voie des Vignes, 92290

### Multi-unit auctions with budget-constrained bidders

Multi-unit auctions with budget-constrained bidders Christian Borgs Jennifer Chayes Nicole Immorlica Mohammad Mahdian Amin Saberi Abstract We study a multi-unit auction with multiple agents, each of whom

### MROrchestrator: A Fine-Grained Resource Orchestration Framework for MapReduce Clusters

MROrchestrator: A Fine-Grained Resource Orchestration Framework for MapReduce Clusters Bikash Sharma, Ramya Prabhakar, Seung-Hwan Lim, Mahmut T. Kandemir, Chita R. Das Department of Computer Science and

### Falloc: Fair Network Bandwidth Allocation in IaaS Datacenters via a Bargaining Game Approach

Falloc: Fair Network Bandwidth Allocation in IaaS Datacenters via a Bargaining Game Approach Fangming Liu 1,2 In collaboration with Jian Guo 1,2, Haowen Tang 1,2, Yingnan Lian 1,2, Hai Jin 2 and John C.S.

### The Design Trade-offs of BitTorrent-like File Sharing Protocols

1 The Design Trade-offs of BitTorrent-like File Sharing Protocols Bin Fan John C.S. Lui Dah-Ming Chiu Abstract The BitTorrent (BT) file sharing protocol is very popular due to its scalability property

### Survey on Job Schedulers in Hadoop Cluster

IOSR Journal of Computer Engineering (IOSR-JCE) e-issn: 2278-0661, p- ISSN: 2278-8727Volume 15, Issue 1 (Sep. - Oct. 2013), PP 46-50 Bincy P Andrews 1, Binu A 2 1 (Rajagiri School of Engineering and Technology,

### ECON 459 Game Theory. Lecture Notes Auctions. Luca Anderlini Spring 2015

ECON 459 Game Theory Lecture Notes Auctions Luca Anderlini Spring 2015 These notes have been used before. If you can still spot any errors or have any suggestions for improvement, please let me know. 1

### Resource-Aware Scheduling for Data Centers with Heterogenous Servers

MISTA 2015 Resource-Aware Scheduling for Data Centers with Heterogenous Servers Tony T. Tran + Peter Yun Zhang Heyse Li + Douglas G. Down J. Christopher Beck + Abstract This paper presents an algorithm

### Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling

Delay Scheduling: A Simple Technique for Achieving Locality and Fairness in Cluster Scheduling Matei Zaharia University of California, Berkeley matei@berkeley.edu Khaled Elmeleegy Yahoo! Research khaled@yahoo-inc.com

### Research on Job Scheduling Algorithm in Hadoop

Journal of Computational Information Systems 7: 6 () 5769-5775 Available at http://www.jofcis.com Research on Job Scheduling Algorithm in Hadoop Yang XIA, Lei WANG, Qiang ZHAO, Gongxuan ZHANG School of

### Towards an understanding of oversubscription in cloud

IBM Research Towards an understanding of oversubscription in cloud Salman A. Baset, Long Wang, Chunqiang Tang sabaset@us.ibm.com IBM T. J. Watson Research Center Hawthorne, NY Outline Oversubscription

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

Hadoop Scheduler w i t h Deadline Constraint Geetha J 1, N UdayBhaskar 2, P ChennaReddy 3,Neha Sniha 4 1,4 Department of Computer Science and Engineering, M S Ramaiah Institute of Technology, Bangalore,

### Evaluating HDFS I/O Performance on Virtualized Systems

Evaluating HDFS I/O Performance on Virtualized Systems Xin Tang xtang@cs.wisc.edu University of Wisconsin-Madison Department of Computer Sciences Abstract Hadoop as a Service (HaaS) has received increasing

### How to Price Shared Optimizations in the Cloud

How to Price Shared Optimizations in the Cloud Prasang Upadhyaya, Magdalena Balazinska, and Dan Suciu Department of Computer Science and Engineering, University of Washington, Seattle, WA, USA {prasang,