Measurement and Utilization of Customer-Provided Resources for Cloud Computing

Transcription

1 2012 Preedings IEEE INFOOM Measurement and Utilizatin f ustmer-prvided Resures fr lud mputing Haiyang Wang Simn Fraser University British lumbia anada hwa17@s.sfu.a Feng Wang Simn Fraser University British lumbia anada fwa1@s.sfu.a Jianghuan Liu Simn Fraser University British lumbia anada jliu@s.sfu.a Justin Gren Enmaly In. Ontari anada justin@enmaly.m Abstrat-Reent years have witnessed lud mputing as an effiient means fr prviding resures as a frm f utility. Driven by the strng demands suh industrial leaders as Amazn Ggle and Mirsft have all ffered pratial lud platfrms mstly dataenter-based. These platfrms are knwn t be pwerful and st-effetive. Yet as the lud ustmers are pure nsumers their lal resures thugh abundant have been largely ignred. In this paper we fr the first time investigate a nvel ustmer-prvided lud platfrm Sptlud thrugh extensive measurements. mplementing data enters Sptlud enables ustmers t ntribute/sell their private resures t lletively ffer lud servies. We find that althugh the apaity as well as the availability f this platfrm is nt yet mparable t enterprise dataenters Sptlud an prvide very flexible servies t ustmers in terms f bth perfrmane and priing. It is friendly t the ustmers wh ften seek t run shrt-term and ustmized tasks at minimum sts. Hwever different frm the standardized enterprise instanes Sptlud instanes are highly diverse whih greatly inrease the diffiulty f instane seletin. T slve this prblem we prpse an instane remmendatin mehanism fr lud servie prviders t remmend shrt-listed instanes t the ustmers. Our mdel analysis and the real-wrld experiments shw that it an help the ustmers t find the best trade ff between benefit and st. I. INTRODUTION lud mputing has reently attrated a substantial amunt f attentins frm bth industry and aademia [1][2][3][4][5]. The emergene f lud mputing as an effiient means f prviding mputing as a frm f utility an already be felt with the burgening f lud servie mpanies. The existing lud platfrms are knwn t be pwerful and st-effiient t run many servies. Hwever as the lud ustmers are pure nsumers their lal resures have been largely ignred espeially nsidering the fast grwing f persnal mputing apaities. In this paper we take a first step twards the ptential f a ustmer-prvided platfrm fr lud mputing: Enmaly's Sptlud [6]. This newbrn platfrm has already attrated an inreasing number f users wrld wide. Different frm mst f the existing enterprise luds (prvided by enterprise dataenters) Sptlud allws ustmers t ntribute/sell their wn idle mputing resures and build their wn lud platfrms. In this way the ustmers are n lnger pure nsumers but an als gain benefits while interating with lud servies. Unlike grid mputing [7] this platfrm is nt deplyed by servie prviders. Instead it is simply frmed/rganized by self-mtivated ustmer resures. The ustmers an earn prfit when their instanes are used by thers. This distinguishes the ustmer-prvided luds frm peer-t-peer netwrks whih have n lear enmi mdel. Hwever these features als raise many new hallenges suh as the availability seurity and instane seletin. Mre imprtantly it is nt lear that what kind f appliatins are suitable fr this platfrm and hw lud ustmers shuld selet the instanes t serve their appliatins. T answer these questins we lsely examine this ustmer-prvided lud platfrm in its mputing apaity server availability netwrk perfrmane and the priing mdel. We find that it is nt yet ready t serve lng-term appliatins (fr example the web servie) r sme PU sensitive tasks. Hwever it prvides very flexible hies (in terms f bth perfrmane and priing) and is very friendly t individual ustmers when they seek t run shrt-term tasks at minimum sts. The ustmer-prvided instanes are highly diverse in terms f their perfrmane and st. When the ustmers need t lease multiple lud instanes their perfrmane/st highly depends n their instane seletin. Unfrtunately mst buyers d nt have enugh knwledge t selet the right instanes r even deide hw many instanes they need t use. Therefre we develp an instane remmendatin mdel t failitate lud prviders t remmend a set f shrt-listed instanes t the ustmers. T understand its effetiveness we applied SeedBx servie [8] in ur real-wrld experiment. Suh a ntent delivery servie is nt the lassi servie that well mathes enterprise lud platfrms. Yet it ptentially wrks well n the Sptlud platfrm given the shrt servie duratin and the distributed resure demands. The experimental results shw that ur slutin an help the ustmers find the best trade ff between benefit and st; in partiular the ustmers an get better perfrmane while minimizing their st at the same time. It is als wrth nting that Sptlud is mpatible with the existing enterprise luds and it is reasnable t believe that it will beme an imprtant mplement t enterprise luds with hybrid resures. The rest f this paper is rganized as fllws: In Setin II we present the related wrks. Based n the bakgrunds that prvided by Setin III we examine the real-wrld perfrmane as well as the st f Sptlud in Setin /12/$ IEEE 442

2 IV. We develp the mdel f instane remmendatin in Setin V. Setin VI evaluates the benefit f this apprah and mpares the perfrmane/st f enterprise and ustmerprvided luds. Sme pratial issues are further disussed in Setin VII and Setin VIII nludes the paper. II. RELATED WORKS The salient features f lud mputing have entied a number f mpanies t enter this market [1][9][2][3][10] and have als attrated signifiant attentin frm aademia [4][5][11]. There have been a series f wrks measuring the perfrmane f publi lud servies frm diverse aspets inluding mputatin saling strage and netwrking servies [12] [13]. Ward et al. [14] have als mpared the perfrmane f a publi lud with that f a private lud. A reent wrk frm Li et al. [15] further examined the inter-dataenter netwrk transfer thrugh fine-grained measurement. Our wrk differ frm them in that we fus n the measurement f resures ntributed frm lud ustmers and the ptentials f utilizing suh resures as a mplement t data-enterbased lud. Many studies have als addressed appliatin designs that leverage lud platfrms [16] [17] r appliatin migratins t the lud. Fr example Wu et al. [18] explred the use f lud fr Vide-n-Demand appliatins; Kannan et al. [19] examined the ptimizatin f hme luds fr mbile devies. We have als identified ptential appliatins that best explre the ustmer ntributed resures. III. SPOT LOUD: BAKGROUNDS AND FRAME WORKS Sptlud is built n the Ggle App Engine [2] and the Enmaly EP platfrm [20]. It prvides a strutured lud apaity marketplae where servie prviders an sell their mputing apaity t a wide range f buyers and re-sellers. Different frm mst f the existing enterprise luds (the lud servies that are prvided by enterprise dataenters) Sptlud allws the ustmers t ntribute/sell their wn idle mputing resures and build their wn lud platfrms. As shwn in Figure 2 we an see that Sptlud is integrating the lud resures frm bth enterprise and ustmer sellers. There are tw kinds f users in this system: Sellers wh sell their idle lud mputing resures and buyers wh nsume/buy these apaities. The instane sellers an dynamially define hardware prfiles latin infrmatin duratin f available apaity and assiated resure sts. Nte that Sptlud als prvides a priing guide t help sellers estimate the revenue they an btain frm Sptlud. This priing guide is based n suh metris as the apaity and the availability f the instanes. An instane buyer n the ther hand needs t reate a VM applianel using the Enmaly Sptlud pakage builder r using the default appliane prvided by Sptlud [21]. After that the buyer an uplad its VM appliane using the Sptlud management interfae and the VMs will be autmatially delivered t sellers' lud 1 VM appliane is a virtual mahine image designed t run n a virtualizatin platfnn (e.g. VirtuaIBx Xen and VMware). 4% Fig. 1: Latins f Sptlud Instanes ustmer lud Resures (sellers) Fig. 2: Basi Framewrk f Sptlud infrastrutures where the VM pakages are pressed arding t the buyers' requirements. The Sptlud mnitrs will then debit buyers n an hurly utility basis with a ntifiatin sent when redits drp belw minimum threshld. Finally the sellers will be paid diretly fr any apaity utilized via the Sptlud marketplae. It is easy t see that this platfrm is different frm the existing enterprise luds that nsist f dediated dataenters. In partiular the ustmers' lal resures play an imprtant rle in Sptlud whih generally ntrl the perfrmane/priing f the instanes. Based n this new feature the existing studies n enterprise luds annt be diretly brrwed t understand suh a new platfrm. It is nt lear that what kind f appliatins are best suitable fr this platfrm and hw buyers shuld selet the instanes t serve their appliatins. T answer these questins we investigate the ustmer-prvided resures n Sptlud thrugh extensive measurements 2. This newbrn platfrm has already attrated attentins ver the Internet; hwever its sale is nt yet mparable t existing enterprise lud servies. Therefre we investigate 116 instanes as a first step t understand the basi features f this platfrm. The physial latins f these instanes are shwn in Figure ur data is btained frm the management servers that deplyed in the Sptlud platfnn. 443

3 # f PUs Fig. 3: # f PUs in the instanes 20 20% 40% 60% 80% 100% Perentage f nline availability Fig. 5: Online availability f Sptlud instanes l.j O L Memry (G8) Fig. 4: Memry in the instanes Initializatin Oelay(mins) Fig. 6: Instane initializatin delay 30 IV. PERFORMANE AND OST MEASUREMENT We first hek the number f PUs in the Sptlud instanes. As shwn in Figure 3 it is easy t see that mst Sptlud instanes (> 75%) pssess less than 4 virtual res. This is nt surprising sine mst f the ustmerprvided resures are nt as pwerful as thse frm enterprise dataenters. Yet there are als sme relatively pwerful instanes; fr example an instane has 16 virtual res with 2 mputatin units in eah re whih is apable f running ertain PU-intensive tasks. We als shw the memry sizes f the instanes in Figure 4. We an see that mst (80%) instanes in Sptlud have a memry less than 5GB whih is nt extra huge but is suitable t run mst f the real-wrld tasks. Different frm enterprise servers that are knwn t have very high availability the servie availability in Sptlud mstly depends n the instane sellers. Figure 5 shws the nline availability f the instanes fr ne mnth; 40% instanes have an nline availability belw 20% that is less than 6 days in the 30-day measurement perid. This availability is aeptable fr shrt-term tasks lasting fr a few hurs r days. Fr lnger tasks Sptlud needs t arefully assist its ustmers t hse prper instanes. It is als wrth nting that befre a buyer an really use a lud instane there is a delay due t a neessary initializatin press. Fr example the AW S Management nsle [22] shws that it generally needs 15 t 30 minutes t initialize a Windws instane n Amazn E2 befre a buyer an really nnet t it. Fr Sptlud as shwn in Figure 6 we an see that mst instanes (mre than 60%) an be initialized within 10 minutes and the maximum initializatin delay is less than 27 minutes. This is nsiderably lwer than that f Amazn E2. The reasn is that the system/user prfiles f Sptlud instanes are already inluded in buyers' VM applianes. Nte that the instanes' peratin systems an als be persnalized by the buyers in Sptlud. If a buyer dse nt want t deide the OS type Linux (Ubuntu 10.10) is set as a default whih indeed has even lwer initializatin delay than the average. We further investigate the instane thrughput. As shwn in Figure 7 we an see that the thrughput f ver 50% instanes are mre than 10 MB whih is gd enugh t deliver ustmers' ntents t the lud servers in nrmal ases. One imprtant feature f lud servies is that the us- 444

4 Server-user Thrughput Fig. 7: Thrughput between lud server and user Priing (USD) Fig. 8: Priing f Sptlud instanes tmers pay nly fr what they have used. In mst f the enterprise luds this st nsists f tw majr parts: the st f using instanes and the st f data transfer. Their priing mdel is mputed/deided arefully by the enterprise servie prviders. Hwever the prie f Sptlud instanes is ustmized by individual sellers wh prvide/sell their lud apaities. As shwn in Figure 8 we an see that the Sptlud instanes are mstly very heap. Mrever this urve is als quite smth indiating that the buyers have diverse ptins t selet instanes in this ustmer-prvided lud platfrm. 2.5 V. INSTANES REOMMENDATION: PERFORMANE OPTIMIZATION AROSS LOUD INSTANES Based n ur measurement it is easy t see that the main advantage f Sptlud mes frm its flexible and ustmized perfrmane/st that are enabled by diverse ustmers. Hwever this als brings new hallenges t the lud servie espeially when the ustmers need t apply fr multiple lud instanes at a time. Different frm enterprise luds where the instanes are mstly standardized int a few lasses the instanes in Sptlud are highly diverse; fr example we an hardly even find tw idential instanes in the Sptlud marketplae. In this ase when the buyers need t rent multiple lud instanes the perfrmane/st will highly depend n their instane seletin. Unfrtunately mst buyers d nt have enugh knwledge t selet the right instanes r even deide hw many instanes they need t use. A gd instane remmendatin mehanism is therefre ritial t the design f Sptlud. A. Mdeling f Instane Remmendatin T investigate instane remmendatin in Sptlud we apply the ppular Seed Bx servie as a ase study [8]. A seedbx is a private dediated server fr uplading and dwnlading files where a peer-t-peer prtl like BitTrrent is used fr data exhange. The file will be dwnladed frm BT swarms t seedbx servers via the BitTrrent prtl and finally be sent t the users via FTP. Suh a ntent delivery servie is nt the lassi servie that well mathes enterprise lud platfrms. Yet it ptentially wrks well n the Sptlud platfrm given the shrt servie duratin and the distributed resure demands. Fr the instane remmendatin in this ase we assume that the lud prviders an btain sme pre-knwledge f buyers' budget and task. Fr example the lud prviders an btain the trrent that the buyers want t dwnlad and the ttal amunt f mney that the buyers want t spend. Based n this infrmatin the lud prviders will help the buyers t find a set f instanes t maximize their ntent dwnlading perfrmane and make sure that the ttal st will nt exeed the buyers' budget. We use t dente the set f lud instanes; eah instane E has an uplading apaity U and a dwnlading apaity de. The prie f buying an instane is dented by Wins (USD per hur) and the traffi prie is dented by Wtra (USD per GB). Fr the BitTrrent swarm3 we use T t refer the set f peers. S refers t the set f seeders (the peers wh have a mplete py f the file and still stay in the system t help ther peers) and L refers t the set f leehers (peers wh d nt have a mplete py f the file; they are generally sharing what they have and dwnlading what they need) where T = S U L. Eah peer t E T has an uplading apaity Ut and a dwnlading apaity dt. This swarm serves a given file f size F. Giving this setting a buyer p* is trying t buy a set f lud instanes t failitate its dwnlading. The uplading and dwnlading apaity f this buyer is dented by up' and dp respetively. We assume the bttlenek is at the edge f netwrks and we use b t refer the end-t-end bandwidth. I) Pure Seeder ase: We first disuss the simplest ase when the BitTrrent swarm nsists f nly ne seeder. We fus n imprving the dwnlading mpletin time as the perfrmane gain. Therefre the gain f buying ne lud instane (fr example the instane E ) is given by 3 Nte that the BT prtl is supprted in Amazn S3 t aelerate the ntent delivery. This prtl als helps mst seedbx servers t btain Internet ntents. 445

5 Gain 1 (s ) = F F (1) bs p' mm(bs e' PI be p' ) s.t. min(bs e be p' ) :::; min(us dp.) (2) where PI is the prbability f ptimisti unhking (PI = II ITI)' In the pure seeder ase PI is equal t 1. The first part f Eq.1 refers t the dwnlading time withut lud assistane and the send part refers t the dwnlading time with the help f lud instane. Nte that Gain 1 (s ) :::; 0 means that we annt benefit frm using lud instane. Based n this equatin we an further get the perfrmane gain f renting multiple (fr example I 11) lud instanes as fllws (where 1 ): (a) Diret dwnlading frm BT swarm (b) lud assisted dwnlading Fig. 9: Example f perfrmane gain (b.<. 0) (4) Fig. 10: Flw netwrk transfrmed frm Figure 9(b) The send part f Eq.3 refers t the dwnlading time f btaining ntent F with the help f I11 lud instanes. Let Time(F 1) = { '(b b )}' the ttal st f L Ee! m'tn S p* renting I11 lud instanes is given by (8) sh(s 1) = Time(F 1) L Wins + F W tra (5) ee! where the first part f Eq.5 refers t the instane st and the remaining part refers t the traffi st. Nte that in the pure seeder ase the lud instanes nly need t uplad the ntents t the peer p *. Therefre the traffi st is equal t F W tra. 2) Pure Leeher ase: We nw disuss anther ase when the BitTrrent swarm nsists f nly leehers. The analysis f these tw ases will help us btain the general mdel f the instane remmendatin prblem. Assuming that there is nly ne leeher l in the BitTrrent swarm. A buyer p * wants t rent I 21 lud instanes t assist its dwnlading. In this ase the perfrmane gain f renting a lud instane is given by: F F Gain 2 (l ) = - - (6) blp' min(rz e be p' ) where rz e refers t the expeted dwnlading rate between BitTrrent swarm and the lud instanes: Tl e = bl e. H + bl e. P 2 (7) where PI is the prbability f ptimisti unhking (PI = II/ITI) and P2 is the prbability f regular unhking and ( ) ITI P L 2 it [23] where N is the number f piees f the served file F and ITI is the ttal number f peers. Therefre the perfrmane gain f renting I 21 lud instanes is given by s.t. min(rz e be p) :::; min(ul dp.) We use T) t refer the trade ff between uplading and dwnlading in the swarm where T) = 1- (Mdwn/Mu p +Mdwn); Mdwn is the dwnlading amunt and Mu p is the uplading amunt. T) E [0 1) where T) = 0 refers the free ridding ase. Let Time(F 2) = LeE2 min( rl A.)' p the ttal st f buying I21 lud instanes is given by (9) st 2 (l n 2 ) = Time(F 2) L Wins ee2 (10) + F. W tra + F. W tra. T) Nte that in the ustmer-prvided luds where the traffi is nt harged W tra will be set t O. 3) General ase With Multiple SeederslLeehers: Based n the analysis f abve tw ases we an merge them tgether and btain the mdel fr instane remmendatin arss all the peers in bth seeder set S and leeher set L aiming t maximize the ttal Benefit and t minimize the ttal st: Benefit = {L st = {L Gain 1 (s 1) + L Gain 1 (l 2)} (11) ses IEL sh(s 1) + L st 2 (l 2)} (12) ses IEL 446

6 r _- - -_-_-- 30 When using 2 Ir-Sptlud instanes 00 8! - ' i u f 'I!? : When using ' '- I'll f E Sptlud instanes 1- Users' ttal st 1 Espeted dwnlading ' mpletin time 1; ' L. 4-0.'- 5-0-' '0.7--0' L '1 8 Ttal Budget that nfigured by the users ($) Ui'.J 0):0 20 : 'tie ' g E. '0 g 10t5 a. a.e x 0 Wu Fig 11: Simulatin result f Sptlud instane remmendatin s.t. min(bs e be p' ) < min(us dp.) (13) min(rz e be p) < min(ul dp.) (14) z= be p' < dp (15) e in Benifit > 0 and st::; Q (16) Besides the existing nstraints in Gainl and Gain 2 (Eq.13 and Eq.14) Eq.15 and Eq.16 shw the extra nstraints after the merging; where Eq.15 is the bandwidth nstraint and Q is the buyer's ttal budget and this ttal st annt exeed Q; Benefit> 0 means that the use f lud instanes shuld at least aelerate the dwnlading f peer p*. B. Prblem Transfrmatin and Analysis Based n this mdel it is easy t see that the lud instanes are wrking tgether like an amplifier between peer-t-peer swarms and buyer p*. Figure 9 gives an illustrative example f the perfrmane gain. PI and P 2 refers t the prbability f ptimisti unhking and regular unhking respetively. If we nly mpare the end-t-end dwnlading rate between seeder S and buyer P* we an see that the buyer an ahive better dwnlading rate unless bs p' 3bs e. nsidering the fat that the lud instanes are mstly high perfrmane servers the ase f bs p' 3bs e an hardly happen in real-wrld4. T slve the instane remmendatin prblem we nvert it int a minimum st maximum flw prblem in a flw netwrk. The ndes' bandwidth nstraints are transferred int edge apaities suh as (uso) n seeder s. Withut lss f generality we als give edges diretins and add a virtual nde A as the sure f the flw netwrk. Figure 10 shws a nversin fr Figure 9(b )where the darker lines shw an example f the slutin. Eah edge has a apaity and a st marked as (apaity st). The bjetive is t maximize the flw (end-t-end dwnlading rate) and minimize the flw st whih an be 4 Withut lss f generality we assume that bep' = bse in this ase. addressed by the lassial Frd-Fulkersn algrithm. Nte that the main inputs f this mdel are tw matrixes: edge apaity matrix and edge st matrix. Different lud prviders an use their wn server apaities and st mdels t generate these tw matries. Fr example in Sptlud Wtra = 0 and the st f mst edges will equal t zer. Fr a given file f size F this is an equivalent prblem with Eq.II A slutin t this prblem gives the buyers lear guide n the instane seletin. Fr example in Figure 10 (where the paths f maximal flws are marked in dark lines) buyer P* shuld selet 2 lud instanes t assist its dwnlading. Figure 11 shws a simulated result fr Sptlud. In this simulatin we use ur measurement trae and the priing mdel t generate the edge apaity matrix and edge st matrix. T) is set t be and the file size F is 1 GB. In Figure 11 the slid line refers t buyers' ttal st( annt exeed the buyers' budget) and the dtted line refers t the expeted dwnlading mpletin time. We an see that in Sptlud when the buyer's budget is set t be less than 0.05 USD ur algrithm shws that the buyers an use 2 lud instanes t ahieve the ttal dwnlading time f 20 minutes. Hwever when the buyer is willing t pay mre say 0.1 USD it an then use 5 instanes at a time and the ttal dwnlading time will be sharply dereased t 2.8 minutes. We an als see that the dwnlading mpletin time annt be further dereased even when the buyers want t pay mre. This minimum dwnlading mpletin time is bunded by the maximum flw (maximum dwnlading rate) in the flw netwrk and using 5 instanes is the ptimal slutin in this ase. Based n these results Sptlud an remmend a list f instanes with 2 ptins fr the buyers: (1) spend 0.03 USD and 20 minutes t rent tw instanes; r (2) spend 0.1 USD and less than 3 minutes t rent five instanes. VI. EXPERIMENT AND EVALUATION In this setin we will further disuss the pssible gain f the instane remmendatin mdel via real-wrld experiments. In partiular we apply the SeedBx servie n bth E2 and Sptlud platfrms t understand whether Sptlud has the ptential t mplement the enterprise luds. The nfiguratin f ur experiment is as fllws: We deplyed 5 ndes in Planet Lab t simulate an unppular BT swarm6. This swarm nsists f 1 seeder and 4 leehers; these peers are serving a ntent f size 384MB (this ntent size is limited by the strage nstraints f the Planet Lab servers). We use a nrmal P in ur ampus t simulate buyer P*. We selet a sample set f 10 instanes frm Amazn E2 inluding 4 mir instanes with hurly prie f 0.02 USD 613M memry and 2 E2 mpute units; 3 small instanes with hurly prie f USD 1.7GB memry and 1 E2 mpute unit; 3 large instanes with hurly prie f 0.34 USD 7.5G memry and 4 E2 mpute units. We have tried 5 The prf an be fund in ur tehnial reprt[24] 6 This is based n the servie feature f SeedBx. If the peer an btain very high dwnlading speed frm a ppular swarm (with mre peers) it is nt neessary t apply SeedBx servie t aelerate the dwnlading 447

7 1000 '-----r====n f) a. e Nrmal Dwnlading ' With the help f 1 Sptlud instane Q) ::: l '6.Q :s: _With the help f 5 Sptlud instanes Time slts (mins) (a) Dwnlading perfnnane f using 0 I 5 instanes i) a. e Q) O 0::: l '0 :s: - Nrmal Dwnlading _With the help f 5 Sptlud instane With the help f 9 Sptlud instanes Time slts (mins) (b) Dwnlading perfnnane f using instanes Fig. 12: Real-wrld experiments f using Sptlud instane 1400 '--'----r====n - Nrmal Dwnlading f) a. e ; ::: l '6.Q :s: ' With the help f 1 E2 instane _With the help f 5 E2 instanes Time slts (mins) (a) Dwnlading perfnnane f using 0 I 5 instanes 1500 f) a. e ::: l '6.Q :s: Nrmal Dwnlading _With the help f 5 E2 instane With the help f 9 E2 instanes Time slts (mins) (b) Dwnlading perfnnane f using instanes Fig. 13: Real-wrld experiments f using Amazn E2 instane t use different types f E2 instanes in the experiment and we fund that using mre pwerful instanes will nt affet the results f ur instane remmendatin. The set f Sptlud instane als nsists f 10 randmly seleted instanes and the instane infrmatin is shwn in Table I. The buyers' ttal budget is set t be 0.5 USD n E2 and 0.1 USD n Sptlud. We als have tested these budgets in the experiment fr several times; these tw budgets an ahieve similar dwnlading perfrmane n bth platfrms and thus give us lear results fr mparisn. Based n this ntrlled envirnment we an suessfully btain the apaity f lud instanes in bth platfrms. After running ur instane remmendatin algrithm we have the mdeling results as fllws: Fr the E2 platfrm the buyers have tw ptins: (1) Use ne small instane fr 22 minutes whih will st USD; r (2) Use ne small and fur mir instanes at the same time fr 8 minutes whih will st USD. Fr Sptlud platfrm the buyers als have tw ptins: (1) Use instane #4 fr 26 minutes whih will st 0.02 USD; (2) Use instane #1 #2 #4 #5 and #6 at the same time fr 9 minutes whih will st USD. Based n these remmendatin results we further validate the real-wrld perfrmane/st n bth E2 and Sptlud platfrms by answering fllwing questins: First whether the instane remmendatin mdel an find gd trade ff between the benefit and the st? Send whether the buyers an experiene better perfrmane by using mre instanes beynd the remmendatin advies? and Third mparing t enterprise luds whether the ustmer-prvided instanes an give mre benefits t the buyers? T answer these questins we perfrm experiments n bth E2 and Sptlud platfrms t test the dwnlading perfrmane as well as the st fr the buyers. Figure 12 shws the dwnlading rate as well as the dwnlading mpletin time when the buyers are using different numbers f Sptlud instanes based n the remmendatin. As shwn in Figure 12a we an see that if the buyers d nt use any lud instane t help their dwnlading the dwnlading mpletin time will be minutes and the dwnlading rate is arund 80K Bps. When s/he fllws the first remmendatin frm Sptlud (using instane #4 448

8 Assisted by A Users ttal st (Sptlud) Sptlud Instanes.. Users' ttal st (E2) i Assisted by f m.' Q)... J1..'.: : 3 '.... :..... en (ij 0.8 : 0.7 R 0.6 W 0.5 :g Dwnlading mpletin time '5-- # f used instanes # f used instanes 60.Q OJ. E 8 01 i5 :g : (ij : Q; 0.5 :') Dwnlading mpletin time ' ' # f used instanes Fig. 14: mparisn f average dwn- Fig. 15: Trade ff between benefit and Fig. 16: Trade ff between benefit and lading rate st (Sptlud) st (Amazn E2) E 8 g> i5 ' 1 t help the dwnlading) the dwnlading mpletin time will be dereased t 46.9 minutes and the dwnlading rate will be inreased t arund 200K Bps. Mrever when the buyers fllw the send remmendatin (using 5 instanes t help the dwnlading). The dwnlading mpletin time will be dereased t 17.0 minutes and the dwnlading rate an smetimes reah t 900K Bps. Hwever if the buyers d nt want t fllw the remmendatin advies and try t use 9 instanes fr dwnlading as shwn in Figure l2b we an see that the dwnlading mpletin time is still slightly redued frm 17.0 minutes t 14.8 minutes. If we mpare this benefit with buyers' st as shwn in Figure 15. It is easy t see that this benefit is nt prprtinal t the buyers' st. In partiular the buyers an spend USD (using 5 instane) t redue the dwnlading mpletin time by 80% (frm minutes t 17.0 minutes); yet when the buyers further inrease their st by 0.14 (ttal st bemes 0.187) USD the dwnlading mpletin time an nly be dereased by 13% (frm 17 minutes t 14.8 minutes). Figure l2b als nfirms that the dwnlading mpletin time an n lnger be effetively redued when the buyers applied mre than 5 instanes. On the ther hand Figure 13 shws the dwnlading rate as well as the dwnlading mpletin time using different numbers f E2 instanes. We an see that when using a similar number f instanes the dwnlading mpletin times f using E2 instanes are slightly shrter than that f using Sptlud. This an be further nfirmed in Figure 14 whih shws that Sptlud an prvide similar perfrmane when mpared t the enterprise servers. Frm Figure 16 we an see that the st f using enterprise instanes is generally quite high. Fr example it will st the buyers 05 USD t mplete the dwnlading in 37.8 minutes and USD t further redue it t 14.3 minutes. mparing t Figure 15 (the Sptlud ase) the buyers nly need t spend 10% f this st t ahieve similar dwnlading mpletin time by using Sptlud instanes. Nte that the real-wrld sts are slightly higher than the mdeling results; this minr differene is due t the value f T) and will nt bias ur investigatin. TABLE I: Seleted Sptlud Instanes Index I Prie I # f PU I Memry I untry 1 $ MB Isle f Man 2 $ MB United States 3 $ GB United States 4 $ GB United States 5 $ GB Isle f Man 6 $ MB United States 7 $ MB Netherland 8 $ MB Ieland 9 $38 4 8GB Ieland 10 $ MB Pland VII. FURTHER DISUSSIONS This paper takes a first step twards the measurement and the utilizatin f ustmer-prvided lud platfrm: Spt lud. There are still many pen issues that an be further explred. Servie Availability: T enable enterprise-level servies in Sptlud we have t ensure high servie availability when integrating ustmers' resures. Different frm dataenters there is n guarantee that a partiular ustmer's lal resures will be always nline fr lud mputing. Thrugh trae-analysis and algrithm design we are nw wrking n a smart resure prvisining arss dynami ustmerprvided resures. Our initial results suggest that highly stable servie availability that is mparable with state-f-theart dataenters is pssible with the distributed and dynami resures. ustmer Inentive:Anther ritial hallenge is t ffer inentive fr a ustmer t ntribute herlhis resures r t utilize thers'. The prblem is further mpliated given that the ustmers are highly hetergeneus making a arse-grained priing mdel used by the existing lud 449

9 prviders hardly wrk. Althugh the Sptlud has already attrated many ustmers it remains unlear whether sme ustmers are selling the instanes with the pries that lwer than their running sts. Therefre the design f a better inentivelbusiness mdel is still neessary fr ur systems and remains an pen issue. Hybrid lud platfrm with bth enterprise and ustmer-prvided resures: It is wrth emphasizing that Sptlud serves as a mplement t data-enter-based enterprise lud servies. While it is flexible and inexpensive fr ertain servies we d nt expet that it an well serve all types f servies with mparable perfrmane and quality as enterprise lud. It wuld be interesting t frm a hybrid lud platfrm that leverages bth enterprise and ustmerprvided resures t ahieve the best flexibility salability and st-effetiveness. Hwever the instane seletin strategy and the priing mdel have t be substantially revised t best rdinate the tw distint types f resures nt t mentin the servie availability and seurity nerns. VIII. ONLUSIONS In this paper we fr the first time examined the ptentials as well as the hallenges in a ustmer-prvided lud platfrm Sptlud. We fund that althugh the apaity as well as the availability f this platfrm are nt yet mparable t the enterprise dataenters Sptlud an prvide very flexible hies t ustmers in terms f bth perfrmane and priing. T better utilize the stumer-prvided resures we als prpsed an instane remmendatin mehanism t address the instane seletin in suh a newbrn platfrm. Our mdel analysis and the real-wrld experiments shwed that it an help the instane buyers t find the best trade ff between benefit and st. The analysis als validated Sptlud as a mplement f great ptentials t dataenter-based luds. AKNOWLEDGEMENT This researh was supprted by a anadian NSER Disvery Grant a Disvery Aeleratr Supplements Award an NSER Engage Grant a MITAS Prjet Grant and a hina NSF Majr Prgram f Internatinal peratin Grant ( ). REFERENES [1] Amazn Web Servie. [Online]. Available: [2] Ggle AppEngine. [Online]. Available: [3] Mirsft Windws Azure. [Online]. Available: [4] M. Armbrust R. G. A. Fx A. D. Jseph R. H. Katz A. Knwinski G. Lee D. A. Pattersn A. Rabkin 1. Stia and M. Zabaria Abve the luds: A Berkeley View f lud mputing University f alifrnia Berkeley Teh. Rep [5] K. Sripanidkulhai S. Sahu Y. Ruan A. Shaikh and. Drai Are luds Ready fr Large Distributed Appliatins? in Pr. SOSP LADIS Wrkshp [6] Sptlud. [Online]. Available: [7] F. Berman G. Fx and A. J. Hey Grid mputing: Making The Glbal Infrastruture a Reality Jhn Wiley & Sns April ISBN [8] SeedBx. [Online]. Available: [9] GGrid lud Hstin. [Online]. Available: [10] Rakspae lud. [Online]. Available: [11] M. Hajjat X. Sun Y.-w. E. Sung D. Maltz S. Ra K. Sripanidku\hai and M. Tawarmalani ludward Bund: Planning fr Benefiial Migratin f Enterprise Appliatins t the lud in Pr. AM SIGOMM [12] S. Garfinkel An Evaluatin f Amazn s Grid mputing Servies : E2 S3 and SQS Harvard University Teh Rep [13] E. Walker Benhmarking amazn E2 fr high-perfrmane sientifi mputing Pr. USENIX Lgin [14] J. S. Ward A Perfrmane mparisn f luds: Amazn E2 and Ubuntu Enterprise lud Pr. SISA DemFEST [15] A. Li and X. Yang ludmp: mparing Publi lud Prviders Pr. AMlUSENIX IM [16] Y. Seung T. Lam L. E. Li and T. W Seamless Saling f Enterprise Appliatins int The lud in Pr. IEEE INFOOM [17] U. Sharma P. Sheny S. Sabu and A. Shaikh Kingfisher: st-aware Elastiity in the lud in Pr. IEEE 1DS [18] Y. Wu. Wu B. Li X. Qiu and F. Lau ludmedia: When lud On Demand Meets Vide On Demand in Pr. IEEE IDS [19] S. Kannan A. Gavrilvska and K. Shwan lud4hme - Enhaning Data Servies with Hme luds in Pr. IEEE IDS [20] Enmaly EP. [Online]. Available: [21] Sptlud Appliane reatin Guide (Fr Buyers). [Online]. Available: [22] AWS Management nsle. [Online]. Available: [23] D. Qiu and R. Srikant Mdeling and Perfrmane Analysis f Bit Trrent-Like Peer-t-Peer Netwrks in Pr. AM SIGOMM [24] H. Wang F. Wang and J. Liu Instane Remmandatin fr Userassisted luds: Design and Optimizatin Simn Fraser University Teh Rep. 20ll. 450