MODELING SERVER USAGE FOR ONLINE TICKET SALES

Proceedigs of the 2011 Witer Simulatio Coferece S. Jai, R.R. Creasey, J. Himmelspach, K.P. White, ad M. Fu, eds. MODELING SERVER USAGE FOR ONLINE TICKET SALES Christie S.M. Currie Uiversity of Southampto Highfield, Southampto SO17 1BJ, UK Latig Lu Peisula College of Medicie & Detistry PeTAG/PeCLAHRC Salmo Pool Lae, EXETER EX2 4SG, UK ABSTRACT This article describes the use of a discrete evet simulatio model to estimate the server power required for the olie sale of cocert tickets to a required service stadard. Data are available o the umber of purchases made per hour ad the percetage of tickets booked olie for previous cocerts ad we describe how these are used to estimate the umber of users i the system. We use bootstrappig to allow us to take accout of the variability i this estimate whe calculatig the cofidece itervals for the simulatio model outputs. A queuig model is also itroduced, which is useful to provide a quick calculatio of how busy the server is before ruig the more computatioally-itesive simulatio model. A umerical example is used to describe the model ad the methodology. 1 INTRODUCTION We describe a project that was carried out to estimate the computig power required to serve a web page sellig a popular product with a limited shelf life, where the iitial demad for the stock is large, e.g. the sale of cocert tickets for a high profile act. The vedor has a miimum service rate based o the time take to complete the purchase ad the proportio of buyers who are uable to complete the buyig process due to server error. There is cosiderable ucertaity about the umber of customers who will try to buy tickets i the first few hours that the sale is ope for, which ted to be the peak hours for ticket sales. We are icorporatig ucertaity from a umber of differet data sets ad wish to iclude a accurate measure of this ucertaity i our fial estimate of the computig power required. Therefore, we use bootstrappig to give a more complete estimate of the variability i the umber of users accessig the website at ay oe time, takig ito accout all of the differet sources of variability. See Cheg ad Currie (2010) for more iformatio about bootstrappig. We cosider both a simulatio model ad a queuig model to describe how customers are progressig through the purchasig process ad to obtai the distributio of the umber of requests beig set to the server per uit time. Kowledge of this distributio allows us to determie the ecessary server capacity for achievig differet service levels for the web-based ticketig system. We begi with a brief literature review of previous work modelig the performace of web systems i Sectio 2 before describig the iput modelig ad associate bootstrappig i Sectio 3. The two models used to describe the purchasig process are the described i Sectios 4 ad 5 ad we give some results of the aalysis, before cocludig i Sectio 6. 978-1-4577-2109-0/11/$26.00 2011 IEEE 752

2 LITERATURE REVIEW Currie ad Lu The focus of our study is to determie how much server power is required to meet performace targets o server respose time for customers ad we do ot cosider the iteral workigs of the computer system that is processig the ticket sales, i.e. we model performace at a system level rather tha a compoet level. Meascé ad Almeida (2002) gives a detailed descriptio of all aspects of capacity plaig for web services, withi which the models developed ca be categorized as system level performace models or compoet-level performace models. The basic structure of models of web server performace at a system level is give i Figure 1. Both Cherkasova ad Phaal (2002) ad Aderso, et al. (2005) use a model of a similar structure to this to model web server performace. The differeces betwee these two models are that 1) the former makes the assumptio of determiistic service times ad sessio-based workload, while a arbitrary service time distributio ad request-based workload are assumed i the latter; 2) the former places a limit o the umber of users i the system while the latter allows a limited umber of jobs at oe time; 3) the former uses oe arrival rate while i the latter a two-state Markov modulated Poisso process is assumed for the arrival process, i.e. two arrival rates are used: oe for low traffic ad the other for busy traffic; ad 4) the former set the service time ad maximum umber of users as give, while i the latter, the maximum umber of jobs that ca be processed by the server at the same time ad the average service time were obtaied through a maximum likelihood estimatio usig the likelihood fuctio of the observed average respose time as discussed i Cao, et al. (2003). Queue for Service Processig by the web server Arrival Process Figure 1: The basic structure of a model of web server performace Exit the system Whe cosiderig a web server system from a compoet level, more emphasis is placed o modelig the iteractios betwee differet compoets of a web server system. I the majority of studies queuig etworks are used, e.g. va der Weij, et al (2009) describe a layered queuig model that is used to fid the optimal assigmet of threads withi the server to miimize server respose times; Dilley, et al (1998) use layered queuig models i their evaluatio studies of the performace of web servers. Va Der Mei, et al (2001) proposed to model the performace of web servers usig a tadem queuig model that cosisted of three sub-models. I their study web server performace metrics were predicted usig a simulatio model, which was validated usig measuremets obtaied i a test lab eviromet. I Elleithy ad Komaraligham (2002), a aalytical model of web servers was proposed, but with a assumptio that the respose time of the queue costatly grows alog with the utilizatio of the server. Wells, et al (2001) aalyze the performace of a Colored Petri Nest model of a web server. There are several ukow parameters i their model, which are determied usig simulatio. Chapters 8 ad 9 of Meascé ad Almeida (2002) provide a overview of the area of performace modelig of web systems. 3 ESTIMATING RESPONSE IN THE PEAK HOUR Data are available o the umber of ticket purchases made per hour i the first 8 hours of the sellig period for a similar cocert. The peak hour for ticket sales is usually the first hour ad always falls withi the first 8 hours ad so we do ot eed to cosider data from later i the sellig period. We also have data available o the proportio of users who have completed purchases over the phoe versus those who have bought their tickets over the Iteret for the previous 4 cocerts. The real data used i the aalysis is cofidetial; therefore we here use simulated data of a similar form to demostrate our methodology. 753

3.1 Resposes per Hour Currie ad Lu The umber of resposes per hour for the first 8 hours after tickets go o sale for a previous cocert are give i Table 1. Table 1: Purchases per Hour for First 8 Hours of the Sellig Period Hour Number of Purchases 1 45893 2 23358 3 8143 4 3258 5 826 6 912 7 519 8 463 We assume that the umber of tickets sold i each hour follows a trucated multivariate ormal distributio, with the trucatio prevetig more tickets beig sold tha the capacity of the cocert veue (assumed to be 90,000). We assume that the covariace structure of the multivariate ormal distributio matches the covariace structure of a multiomial distributio fitted to these data, i.e. the variace of the umber of purchases i hour i is equal to ( i) Var ( X i) ( i) 1, 8 ( k) k1 where (i) is the umber of tickets sold i hour i. The covariace betwee the umbers of purchases i hours i ad j is equal to ( i) ( j) Cov ( X i, X j ). 8 ( k) 3.2 Proportio of Tickets Sold Olie The proportio of tickets sold olie versus o the phoe for the previous 4 cocerts is give i Table 2. k1 754

Currie ad Lu Table 2: The percetage of tickets sold olie for the previous 4 cocerts. Cocert Percetage Sold Olie 1 73.7 2 71.4 3 81.8 4 90.2 The data exhibit a upwards tred, which appears to be liear give the few data poits available. We ca therefore fit a liear regressio lie to the data to give us a predictio of the olie usage for cocert 5. We assume ormal errors ad obtai a result suggestig that 94% (95% predictio iterval: 64% - 100%) will purchase tickets olie for cocert 5. The use of a straight lie predictor for this value is a little aïve as at some poit i time the liear relatioship is likely to break dow ad the percetage applyig olie will level off. However, with few data available it seems a reasoable first approximatio. 3.3 Combied Probability Distributio for the Olie Respose Durig the Peak Hour We use Bootstrap resamplig to combie the ucertaities from the two data sets ad obtai a estimate of the umber of olie purchase requests arrivig at the server i the peak hour of the sellig period. The bootstrappig allows us to obtai a full probability distributio for the umber of olie purchases, which will be useful i the modelig stages of the project. We use 1000 iteratios of the bootstrappig. I each iteratio, we carry out the followig procedure: 1. Sample purchase umbers i each of the sellig periods from a multivariate ormal distributio. 2. Sample the percetage sold olie for the previous 4 cocerts from uivariate ormal distributios with mea equal to the predictio from the fitted regressio lie ad variace calculated from the sum of squares of the errors betwee the data poits ad the fitted lie 3. Fid the best fit regressio lie for the percetage sold olie ad use this to predict the percetage sold olie for the 5 th cocert, settig ay predicted percetages greater tha 100% to be equal to 100% 4. Calculate the umber of people purchasig tickets olie durig the peak hour The bootstrap results ca the be used to fid a distributio for the umber of people purchasig tickets olie. For these data we fid that the expected umber of people purchasig tickets olie durig the peak hour is 42,300 (95% cofidece iterval: 32,600-46,100). 4 SIMULATION MODELING OF THE ONLINE PURCHASING SYSTEM The trasitio diagram for the simulatio model of the olie purchasig system is give i Figure 2. 755

Currie ad Lu Customer starts purchasig process Wait for a thread to become available Purchase is complete Customer completes oe sectio of the bookig form Server processes iformatio More iformatio is required Figure 2: Trasitio Diagram for the Simulatio Model of the Olie Purchasig System. We cosider oly the peak hour of sales ad for each of 25 rus of the simulatio, draw the umber of customers who wish to purchase a ticket olie from the empirical distributio fuctio of the bootstrappig results, obtaied as described i the previous sectio. We use the Trials Calculator i Simul8 (www.simul8.com) to determie the appropriate umber of rus such that the variability i the average waitig time is equal to 5% of the mea. This is foud to be approximately 25. The Trials Calculator uses AutoSimOA (http://www2.warwick.ac.uk/fac/soc/wbs/projects/autosimoa/), which automates the output aalysis. The methodology used by AutoSimOA is described i Hoad, et al (2011). The aim of the modelig is to determie the umber of threads eeded i the server to best satisfy the performace requiremets that o request should wait loger tha 2 secods ad that fewer tha 10% of requests should wait loger tha 0.5 secods. More detailed assumptios of the simulatio model are give i the followig sectio ad the results are give i Table 3. 4.1 Assumptios The rate at which oe thread respods to oe request is idepedet of the umber of threads that are i use (i.e. performace is ot depedet o how busy the server is) Each piece of iformatio is submitted idividually The state i which all threads are beig used costitutes a state i which some customers are waitig for threads to be available The time a user speds workig o a request (e.g. completig credit card details) follows a expoetial probability distributio with a mea of 30 secods The time the server speds processig each questio follows a expoetial distributio with a mea of 70 millisecods (based o expert opiio) There are o average 10 pieces of iformatio to be submitted by the user but some users may have to resubmit iformatio because of errors; therefore the probability of exitig the system after server processig is 0.1 ad the probability of eedig to submit more iformatio is 0.9 4.2 Results The results give i Table 3 suggest that the optimal umber of threads required to meet the required service stadard is 10. Table 3: Results of the Web System Simulatio Model (95% Cofidece Itervals Give i Paretheses) Number of Threads Percetage of Requests Waitig Loger tha 0.5s Maximum Waitig Time for a Request (mis) 8 85.3 [84.6, 86.0] 0.57 [0.55, 0.59] 9 33.0 [26.5,39.4] 0.04 [0.03, 0.05] 10 0.02 [0.00,0.03] 0.01 [0.01, 0.01] 756

Number of Customers Currie ad Lu 5 SIMULATION MODELING OF A STEADY-STATE WEB SYSTEM Observatios of the simulatio model output suggest that the system approaches a steady state approximately 30 miutes after the start of the sales period. For example Figure 3 shows how the umber of customers i the process of completig the olie form chages sice the start of the sellig period, demostratig a levelig off after approximately 25 to 30 miutes. For the steady-state situatio, a queuig model ca provide us with some simple aalytical results much more quickly tha the simulatio model, which takes some time to ru give the large umbers of idividuals i the system. 4500 4000 3500 3000 2500 2000 1500 1000 500 0 0 10 20 30 40 50 60 70 Time Figure 3: the variatio i the umber of customers completig the olie form over the sellig period, where zero time is whe the ticketig system is opeed. 5.1 The Model With the assumptios give above i Sectio 4.1, we ca use a queuig model to describe the usage of the web system. We defie our system i terms of, the umber pieces of iformatio eedig processig by the system. Whe there are zero threads i use, = 0 ad M users will be thikig about submittig iformatio; whe there are 10 threads i use, = 10 ad M 10 users will be thikig about submittig iformatio. Whe the system is i a state that is greater tha N, there are -N users waitig for threads to become available. Figure 4 gives the trasitio diagram for the system. Figure 4: Trasitio Diagram for the Markov Chai Model of the Web-Based System. 757

Currie ad Lu The system ca therefore be described as a birth-death process, with M + 1 states ( = 0, 1, 2,, M). The rate at which we move from state to state + 1 is equal to ( M ) 0,1,..., M 1 0 M ad the rate at which we move from state to state 1 is equal to 0 0 1,2,..., N N N 1, N 2,..., M Therefore, the probability that we fid the system i state whe we observe it, or alteratively, the proportio of time that the system speds i state is equal to M! 0 0,1,2,..., N ( M )!!, M! 1 N 1, N 2,..., M N 0 ( M )! N! N where π 0 is determied by esurig that the sum of the probabilities of beig i each of the states is oe. For this problem, π 0 does ot have a ice, closed form expressio. 5.2 Results If the steady-state assumptio is valid for the system, we ca use the queuig model to obtai some simple statistics. Whe the system is i state N + 1 or greater, there will be iformatio waitig to be processed by the server. Therefore, the probability that there are users waitig for the server to process oe of their resposes will be equal to Pr( there are users waitig). M N 1 The expected umber of users waitig i a queue ca be estimated by Expected umber i queue M N 1 ( N). Below i Table 4 we give the values of these statistics for three differet values of N, the umber of threads available. Usig the output of the bootstrappig described i Sectio 3, we are also able to give a idicatio of the variability i these statistics that reflects the variability i the umber of cocurret users, M. The olie form takes approximately 5 miutes to complete; therefore the average umber of cocurret users is likely to be closer to 3500 (oe twelfth of 42,300) ad we divide our estimates of the total users i the first hour by 12 to obtai our estimates of M. Table 4: Results of the Queue Model (95% Cofidece Itervals Give i Paretheses) Number of Threads Probability there are Users Waitig Expected Number i the Queue 8 1.00 [0.342, 1.00] 170 [1.61, 404] 9 0.718 [0.173, 0.927] 9.70 [0.576, 38.5] 10 0.419 [0.0835,0.577] 2.56 [0.226, 5.24] 758

Currie ad Lu The results suggest that the system is likely to be too busy with 8 threads to provide a reasoable service but should be able to cope with the demad if there were 10 threads. 6 CONCLUSION The results preseted i the previous sectio give a suggestio of the umber of threads required for the server to meet the suggested performace requiremets. The results of the simulatio model lead to the same coclusios as those calculated aalytically from the queue model, which provides a good check of their accuracy. It also justifies the use of the aalytical model to provide a iitial, very quick calculatio of the state of the system before ruig the computatioally-itesive simulatio model. We have described two models of a web-based purchasig system that allow us to draw out key performace statistics for the sales process. The discrete evet simulatio model is particularly useful i this applicatio as it allows us to iclude the iitial trasiet effects o the performace of the web system. Noetheless, the system appears to eter a steady-state after approximately half a hour ad so a steadystate queuig model is justified to give us a quick, rough ad ready guide to how busy the server is likely to be before ruig the more computatioally itesive simulatio model. The use of bootstrappig i this applicatio allows us to very easily draw out statistics for our fial results that take ito accout the full variability i our estimate of the umber of cocurret users durig the peak hour of sales. I this example, our estimate of this key simulatio parameter draws o oly two differet data sets but the methodology would hold i other situatios where the process of estimatig the simulatio iput parameters is much more complex. REFERENCES Adersso, M., J. Cao, M. Kihl ad C. Nyberg. 2005. "Performace modelig of a Apache web server with bursty arrival traffic", IC'03 : Proceedigs of the Iteratioal Coferece o Iteret Computig. ISBN: 1-932415-02-5. Publisher: CSREA Press. Box, G. E. P., W. Huter ad S. Huter. 1978. Statistics for Experimeters: A Itroductio to Desig, Data Aalysis, ad Model Buildig. Wiley. Cao, J., M. Adersso, C. Nyberg, ad M. Kihl. 2003. Web Server Performace Modelig Usig a M/G/1/K*PS Queue, i 10th Iteratioal Coferece o Telecommuicatios, Papeete, Tahiti. Cheg, R.C.H. ad C.S.M. Currie. 2009. Resamplig methods of aalysis i simulatio studies I Proceedigs of the 2009 Witer Simulatio Coferece, Edited by M.D. Rossetti, R.R. Hill, B. Johasso, A. Duki ad R.G. Igalls. Piscataway, New Jersey: Istitute of Electrical ad Electroics Egieer, Ic. Cherkasova, L. ad P. Phaal. 2002. Sessio-based admissio cotrol: A mechaism for peak load maagemet of commercial web sites. IEEE Trasactios o computers, vol. 51, o. 6, pp. 669-685, Jue 2002. Dilley, J., R. Friedrich, T. Ji ad J. Rolia. 1998. Web server performace measuremet ad modelig techiques, Performace Evaluatio, vol. 33, pp. 5-26, 1998. Elleithy, K. M. ad A. Komaraligam. 2002. Usig a Queuig Model to Aalyze the Performace of Web Servers. Available at http://citeseerx.ist.psu.edu/viewdoc/summary?doi=?doi=10.1.1.19.3667. Hoad, K. A., S. Robiso ad R. Davies. 2011. AutoSimOA: A framework for automated aalysis of simulatio output. Joural of Simulatio, vol. 5, pp. 9-24. Meascé, D. A. ad V.A. Almeida. 2002. Capacity Plaig for Web Services. Pretice Hall, Upper Saddle River, NJ. Va Der Mei, R. D., R. Harihara ad P.K. Reeser. 2001. Web server performace modelig. Telecommuicatio Systems, vol. 16, o. 3, 4, pp. 361-378. Va der Weij, W., S. Bhulai ad R. va der Mei. 2009. Dyamic thread assigmet i web server performace optimizatio, Performace Evaluatio, vol. 66, pp. 301-310. 759

Currie ad Lu Wells, L., S. Christese, L.M. Kristese ad K.H. Mortese. 2001. Simulatio based performace aalysis of web servers. I Proceedigs of the 9th Iteratioal Workshop o Petri Nets ad Performace Models (PNPM 2001). IEEE Computer Society, 2001, pp. 59-68. AUTHOR BIOGRAPHIES CHRISTINE CURRIE is a lecturer of Operatioal Research i the School of Mathematics i the Uiversity of Southampto, UK, where she also obtaied her Ph.D. She is ow Maagig Editor for the Joural of Simulatio ad previously the Book Review editor. Christie has bee co-chair of the Simulatio Special Iterest Group i the UK Operatioal Research Society for a umber of years ad ivolved i the orgaizatio of the UK Simulatio Workshop. Her research iterests iclude mathematical modelig of epidemics, Bayesia statistics, reveue maagemet, variace reductio methods ad optimizatio of simulatio models. LANTING LU is a research fellow at the Peisula College of Medicie ad Detistry, affiliated to Exeter Uiversity, UK. She has a PhD ad MSc i Operatioal Research from the Uiversity of Southampto, UK. Her research iterests iclude simulatio modelig of maufacturig processes, health care ad classificatio aalyses. 760