PLANNING THE CAPACITY OF A WEB SERVER: AN EXPERIENCE REPORT

Transcription

1 PLANNING THE CAPACITY OF A WEB SERVER: AN EXPERIENCE REPORT Daniel A. Menascé Robet Peaino Depatment of Compute Science, MS 4A5 Univesity Computing Geoge Mason Univesity Geoge Mason Univesity Faifax, VA Faifax, VA Menasce@cs.gmu.edu Peaino@gmu.edu Nikki Dinh Quan T. Dinh SRA Intenational, Inc. Lockheed Matin Fedeal Systems, Inc Fai Lakes Cout 9500 Godwin D. Faifax, VA Manassas, VA nikki_dinh@sa.com Quan.t.dinh@lmco.com Web seves have become a key aspect of most mission-citical applications. This pape descibes ou expeience in planning the capacity of a poduction seve at ou oganization. The methodology used hee is model-based and allows fo pefomance pediction. The pape descibes how each step of the methodology was caied out with special emphasis in wokload chaacteization, pefomance model development, validation, and calibation. 1. Intoduction At pesent, Geoge Mason Univesity does not have a good undestanding of its web sevice wokload, o the subsequent esouce equiements ceated by this wokload. As a esult, the univesity can neithe quantify cuent esouce utilization, no pedict futue esouce needs. Though the use of a model-based capacity planning methodology [MA98], we seek to paint an analytical pictue of the cuent web sevice wokload, and the esulting esouce utilization. We will also pedict maximum thoughput, and suggest changes that will incease maximum thoughput. This pape will step though the stages of the model-based capacity planning methodology, to include envionmental desciption, wokload chaacteization, wokload model development and validation, pefomance model development, pefomance model validation and calibation, and pefomance pediction. As the Wold Wide Web has become an integal pat of ou society, has become one of the most impotant, and most visible epesentatives of the univesity. The seve (whose fomal name is jiju) is a 24-by-7 opeation. Because it is a highly visible poduction seve, if jiju wee to be down fo any length of time, phones would stat inging almost immediately. This need fo constant availability seveely limited ou ability to pefom benchmaks and othe activities that would nomally occu on off-line equipment. Anothe challenge to ou analysis is the constantly changing natue of jiju. Just pio to the stat of ou analysis, the administato wanted to upgade the seve softwae. Since this might change the pefomance behavio of the seve, we opted to wait fo the upgade to be completed befoe stating ou analysis. And as we wee finishing ou analysis, we wee pessued to do so quickly, since anothe softwae upgade was planned, as well as memoy and RAID disk upgades. This is the volatile natue of mission-citical components. In this pape, we use the capacity planning methodology descibed in [MA98] to plan the capacity of a public Web seve. The methodology consists of the following steps: undestanding the envionment, wokload chaacteization, pefomance model development, model validation, pefomance pediction, wokload foecasting, and costpefomance analysis. We descibe in the following sections ou expeience with applying the methodology. 2. Undestanding the Envionment This fist step in the methodology is concened with descibing the hadwae, the softwae, the connectivity of the seve, and the peak usage peiods. We gatheed ou initial infomation by inteviewing the administation to get an oveview of the hadwae and softwae configuation, and the sevices povided.

2 Figue 1 details the hadwae of the seve which consists of a dual 168 MHz pocesso Sun Seve with 256 MB of RAM unning Solais 2.6. The HTTP seve is Apache vesion The stoage subsystem is composed of thee disks. The opeating system and the logs it geneates ae stoed on disk 1, the HTTP access and eo logs ae stoed on disk 2, and the document tee is stoed on disk 3. The machine is dedicated to being a Web seve. The only pesons who would log diectly into the system ae the administatos. CPU MHz Disk 1 Opeating System epesents appoximately 40,000 samples. Figue 3 is a gaph of the tps ate fo evey minute of the month. The X-axis is labeled by hou of day. We can clealy see that the peak hous ae fom 9 AM to 5 PM. The mean tps ate ove the entie month is 5.49 tps. The maximum ate, aveaged ove one minute, is 28.4 tps. The minimum ate is tps. This means that the seve was neve idle, ove any one-minute inteval. To futhe undestand the data, we added a thid dimension to the data, which is Day of the Week (0 to 6 in Figue 4; 0 is Monday and 6 is Sunday). We could then see how the usage fluctuates thoughout the week. Fom the figue it becomes clea that weekends epesent peiods of lowe activity than weekdays. The peak peiod was detemined to be weekdays fom 9 AM to 5PM CPU MHz Disk 2 HTTP Logs 256 MB RAM Disk 3 Document Tee Figue 1 - Hadwae Configuation of the Seve Figue 2 illustates the connectivity. Jiju is connected to a local netwok via a 100Mbps Ethenet inteface using the TCP/IP potocols. This inteface is connected diectly to a oute, and then out to the Intenet via a T3 line. Jiju is a publicly accessible seve; 50 pecent of all equests oiginate off-campus. Ou analysis concentates on the seve plus the link that connects the seve to the Intenet. Figue 3 - Wokload Intensity (in tps) fo the Month of Febuay Web Seve LAN Route Intenet Client Client Figue 2 - Seve Connectivity In ode to detemine the peak usage peiods, we analyzed the web seve access log fo the month of Febuay, which consists of 13.2 million equests. Fo each minute of the month, an aveage aival ate was calculated, in tansactions pe second (tps). This Figue 4 - Wokload Intensity by Hou of the Day and Day of the Week. 3. Wokload Chaacteization In this step of the capacity planning methodology, we descibe the main components of the global wokload, the decomposition into classes, and how we aived at these esults.

3 3.1 Identifying the Basic Component The sevice povided by jiju suppots the HTTP Potocol. This potocol defines a seies of equest types, with ancillay data. The basic component of this system is the HTTP GET equest. Analysis of one month of the Apache web seve access logs shows that pecent of all activity is of this type. 3.2 Choice of the Chaacteizing Paametes The choice of the chaacteizing paametes is staightfowad. The wokload intensity of equests is chaacteized by the aival ate, measued in tansactions pe second (tps). Sevice demands ae computed fo the thee disks, and the two CPUs. 3.3 Patitioning the Wokload Fo the pupose of making the wokload model moe epesentative, we patitioned the wokload accoding to file sizes. Utilizing the web seve access log fo the peak peiods as defined in section 2, we wee able to obtain file sizes and fequencies. We wanted to see if thee wee aeas of file size concentations that would make the patitions obvious. The data was so dispaate as to make any visual display useless. We then implemented the minimum spanning tee (MST) [MAD94, MA98] clusteing algoithm so that we could pefom a one-dimensional clusteing on file sizes. We initially an the clusteing algoithm with 150 clustes. Upon inspection, we noticed that the top 4 clustes epesented 98 pecent of all equests. Additionally, zeo-length tansfes wee the esult of pecent of all equests. A equest may epot a zeo-length tansfe fo seveal easons including: The vesion in the client cache is up to date. The equested file was not found. An access violation pevented file tansfe. In each of these cases, the equest must be pocessed, which ceates CPU demand, and a seach must be done fo the file, which ceates disk demand. So, the zeo-length tansfes ae a vey eal pat of the system activity. Using the clusteing esults as a stating point, along with the above obsevations, we decided on fou classes of equests, as shown in Table 1. Class Range (Bytes) % Requests % Data Deived fom Fom cluste Remainde of cluste Clustes 1, 2, and 3 4 > Remainde of clustes Table 1 - Class Chaacteization. Note that 10.77% of the equests (classes 3 and 4) ae esponsible fo etieving about 77% of the data. This is typical of Web seves [AW96,CB96]. 3.4 Data Collection This section discusses the tools used fo data collection and descibes how they wee used to obtain the sevice demands at the CPU and disks Tools Used fo Data Collection Fom the pespective of the system, we used iostat and pocess accounting. iostat povided us with the necessay device utilization fo both the CPU and disks. Pocess accounting povided us with a vey detailed look at the system activity. This allowed us to ensue that we accounted fo all majo esouce usage. Fom the pespective of the Apache seve, we elied on the seve access log. In detemining what ou data equiements would be, we decided that the existing access log did not povide sufficient infomation about seve activity. Since the souce code fo Apache is available, we could instument the seve, to wite additional infomation into the log. By mid-mach, the following additional fields wee being ecoded in the poduction log: PID, sevice time, CPU time, and system date/time. We explain each of these additional fields in what follows. PID: the Pocess ID. The Apache seve woks in a maste/slave aangement. A maste pocess spawns slave tasks to sevice equests. When a slave seve has handled a cetain pe-configued numbe of equests, the maste kills the slave and stats a new one in its place. The intent behind this design is to avoid memoy leaks in key system functions, on cetain OS platfoms. Ou intention in ecoding the PID of the slave on each log ecod was to be able to find all equests handled by a single slave, and then match that slave to a pocess accounting ecod. Sevice Time. Also defined as the seve-side esponse time, this is the elapsed time in micoseconds, fom the stat of pocessing of a equest, until the data is completely sent. The intent of ecoding this was to have a diect measue of esponse time fo validation of the pefomance model, and fo benchmaking puposes. CPU Time. This is the accumulated CPU time of the equest, in milliseconds. This field allows us to have a diect measue of CPU sevice demand fo each equest. Ou plan was to find the aveage CPU sevice demand fo each class of equest. It tuns out that this field poved useless as explained in section System Date/Time in Seconds. Although the date and time is ecoded in the HTTP log, it is not ecoded in a fomat useful fo time manipulation. This field ecods the system intenal clock in accumulated

4 seconds, since Time of Epoch. This allows us to find the elapsed time between any two log ecods by simply subtacting the time values, and was used to compute equest inte-aival times. 3.5 Computing Sevice Demands The sevice demand D i, of equests of class at device i is defined as the total sevice time of equests of that class at that device [MAD94, MA98] and can be computed as D i U i, T C, (1) whee U i, is the utilization of device i due to equests of class, T is the inteval duing which the utilization was measued and C is the numbe of equests of class pocessed in the inteval of duation T. While it is easy to obtain the oveall utilization of a device, it is not always easy to obtain the utilization of that device due to a specific type of equest, especially when the seve is a poduction seve that cannot be taken offline fo unning contolled benchmaks. Also, we did not have access to any simila machine on which we could un equivalent benchmaks. This placed a temendous limitation on ou options fo computing sevice demands fo the esouces and classes. We decided to ely on a ule of popotionality fo cetain esouces. In othe wods, we assumed, that fo cetain devices, the utilization of a specific class at that device is popotional to the faction of data etieved by that class fom the total numbe of bytes etieved. We also decided to distibute system ovehead popotionally among the classes. The utilization of system esouces was ecoded fo one inteval of the peak peiod time window. The sevice demands fo each esouce and class wee deived fom these utilization values based on the fomula: D i U i f T C, (2) whee, D i, is the sevice demand of equests of class at esouce i, U i is the total utilization of esouce i (obtained fom iostat), T the measuement peiod, C is the numbe of equests of class pocessed duing the inteval (obtained fom the HTTP log), and f, is a popotionality facto that indicates the faction of the busy peiod of esouce i that can be attibuted to class CPU Sevice Demand At fist, we planned to deive the CPU sevice demands diectly fom the instumented seve log. We wee disappointed to discove that this would not be possible due to the ganulaity with which the system ecods CPU time accumulation. The system can only update the CPU time at the end of a pocess timeslice. On jiju, this time peiod is 10 msec. Testing veified that CPU time would always be epoted in incements of 10 msec. Unfotunately, the sevice demand of most equests is less than 10 msec. As a esult, the time epoted in the log fo small files is eithe 0 msec o 10 msec, neithe of which is coect. We decided to wok with the pemise that the CPU sevice demand is popotional to the file size. To test this supposition, we benchmaked the CPU time equied to equest a set of files of inceasing size. The files had to be of sufficient size such that the ganulaity of the CPU time epoted by the system would not have a significant impact on the measuements. We benchmaked files of size 1 MB to 10MB in incements of 1MB. Each file size was benchmaked 10 times. We then pefomed a linea egession on the esulting data. Figue 5 shows the esults of the benchmaks, and the linea egession. Assuming that CPU demand is popotional to file size implies that class 1 (zeolength files) equies no time. But those equests must use some time, so we ely on the linea egession and set the sevice demand to be the y-intecept, which would be the expected value fo a file of zeo bytes. To calculate the sevice demands fo the othe classes, we use Eq. (2), but we fist must subtact fom the CPU busy time (2 x U CPU x T) the activity epesented by class 1, which was calculated independently. Also, the facto f is the pecentage of bytes b etieved by class equests. So the CPU sevice demand is given by D cpu, 2 U T D 1 C1 b cpu cpu, whee C 1 is the numbe of class 1 equests executed duing time T. Note that the facto 2 in the total CPU time (2 x U CPU x T) is due to the fact that the seve has two CPUs and the CPU utilization epoted by iostat is the aveage utilization of the two pocessos. C Disk 1 Sevice Demand Disk 1 contains the opeating system and its suppot files such as the pocess accounting log. As mentioned in section 3.4.1, slave seve pocesses ae constantly being ceated and killed. When a pocess ends, a pocess accounting log ecod is witten. Theefoe, the highe the aival ate of equests, moe pocess accounting ecods ae geneated. Instead of a popotional distibution of activity, each class has the same sevice demand since all accounting ecods (3)

5 ae the same size egadless of class. The sevice demand fo disk 1 is then computed as D disk, U disk1 4 1 T C 1 (4) Figue 5 - CPU time (in msec) vesus file size (in Mbytes) Disk 2 Sevice Demand Disk 2 holds the web seve log. Just like disk1, the sevice demand necessay to wite a log ecod is the same fo all equests, egadless of class. So the sevice demand is computed in the same manne as disk Disk 3 Sevice Demand Disk 3 is the seve file epositoy. Sevice demands must be deived in a manne simila to the CPU demand. As was the case with the CPU sevice demand, the class 1 sevice demand could not be based on the class 1 popotion of data etieved, since the class 1 file size is zeo. And as mentioned befoe, it was not possible to take the machine off-line fo a contolled benchmak. To get a ough estimate of the sevice demand of disk 3 fo class 1, we pefomed an on-line benchmak. Consideing the easons why a equest would fall into this class (mentioned in section 3.3), the benchmak meely needed to seach fo files, and not open them. The benchmak accessed the I- node infomation fo all files in seveal file systems, while the disk utilization was measued with iostat. This was pefomed by issuing the UNIX command 'ls -lr' ove entie file systems, monitoing the utilization of the disk. We used entie file systems to avoid picking up anything in the cache. We used the long listing (-l) option, to foce the system to look at the file i-node, which should be equivalent to the amount of wok pefomed by 0-length tansfes. Afte the un, we counted the numbe of files in the file system. This would become C 1. We then computed the sevice demand at disk 3 fo class 0 as D=U*(T/C 1 ). We did this fo thee diffeent file systems on jiju and aveaged the esults. The emaining class sevice demands wee computed using the same method as the CPU classes. The esulting sevice demands ae then given in Table 2. Note that we have added the incoming link and outgoing links as additional devices. The sevice demand at these devices was obtained fom the file size fo each class and fom the potocol (TCP/IP + HTTP) ovehead incued in tansmitting files of that size CPU CPU Disk Disk Disk In-link Out-link Table 2 - Sevice demands (msec) Wokload Model Validation Wokload model validation would entail the development of a synthetic wokload based on the wokload chaacteization, which would be un on the system so that pefomance esults could be compaed to poduction pefomance esults. Because jiju is a poduction seve, we wee not pemitted to bing down the seve so that we could un a synthetic wokload. Had we been able to do this, we would have built a scipt that would submit synthetic equests of fou types based on the fou classes. The file sizes would be based on the aveage file sizes fo each class. The distibution of the equests would be based on the popotion of equests that each class epesents. 5. Pefomance Model Development and Pefomance Pediction The pefomance model used to epesent the seve and the incoming and outgoing links is a multiclass open queuing netwok (QN) model (see Fig. 6). To model the dual pocesso CPU we used the appoximation suggested by Seidman et al [SSS87]. Figue 7 shows the aveage esponse time fo the fou classes as a function of the aggegate aival ate of equests. The popotion of equests in each class is the same shown in Table 1. The model shows that afte 120 tps, the utilization of the CPU eaches 100%. Class 4 is the one with the lagest esponse time and with the lagest gowth ate in esponse time.

6 Avg. Response Time (sec) Outgoing link Incoming link Disk 1 Disk 2 Disk 3 CPUs Figue 6- QN model fo the Web seve Total Aival Rate (tps) Class 1 Class 2 Class 3 Class 4 Figue 7 Response Time vs. Aival Rate 6. Pefomance Model Validation and Calibation In this step, we compaed the model pedictions with the actual epoted system activity. We descibe the validation poblems and how we fixed them. When we tested the model fo the fist time, the esults wee wildly inaccuate, and inconsistent. Each set of data we an fom diffeent days povided diffeing amounts of eo. So we tuned back to ou envionmental eview, to see what we had missed. To get a close look at the system activity, we wee pemitted by the system administato to eview the pocess accounting logs and the con configuation, to get a much moe pecise look at the system activity. Fom the pocess accounting log eview, we identified a pocess that used a temendous amount of esouces, and an evey hou. We taced the pocess to a job that was scheduled in the con configuation. The job tabulated usage statistics by eading the entie access log evey hou. As the log gew in size, so did the esouce equiements of this pocess. Thus, the esouce equiements wee diffeent fo evey hou of evey day of the month. Since the access log is estated only at the end of the month, it will gow to about 1.5 gigabytes in size befoe being emoved. As a esult, by the end of the month, this statistics job equied 45 minutes of CPU time each hou, which is much moe than even the seve equies! We estimated that the statistics job accounts fo appoximately 37 pecent of the epoted aggegate CPU utilization, and this is consistent with the activity we wee seeing with iostat. Befoe we decided how to incopoate this activity into the model, we bought ou findings to the attention of the administation. They did not know that the job was unning, and detemined that it only needed to un once pe week, duing offhous. The escheduling of this job pecluded any need to include it in the model. We felt that we achieved much success aleady, since we saved the univesity almost an entie pocesso! Once the statistics usage job was addessed, ou model accuacy in tems of utilization became quite acceptable. The maximum pecent absolute eo in the utilization was 6.5% and that happened fo class 2. We planned to validate the model esponse time by compaing it to the ecoded seve-side esponse time as defined in section We wee teibly disappointed to discove that although the esponse was being accuately ecoded, it was not stictly seve-side esponse time. This effect was a chaacteistic of being on an open netwok. When we andomly sampled the log, we found that esponse times could vay fom 0.5 seconds to 60 seconds fo the same file size. Causes fo this behavio include netwok sluggishness, timeouts, and slow modem speed at the client. Of couse, the esponse times epoted by the model do not conside these issues. Slow modem speed is pobably the single geatest cause of esponse time vaiability. It should be noted that esponse time cannot be ecoded in the HTTP log until all of the data fo a equest is sent. If the client equest is coming ove low speed modem connections, this will limit the effective bandwidth of the TCP connection used to send the esults back to the client and will theefoe incease esponse time. To study the effects on esponse time of the client access speed, we developed a mixed QN model with one closed class epesenting a client making equests and fou open classes identical to the ones in the pevious model (see Fig. 8). Figue 8 shows thee delay devices, epesented by cicles, coesponding to the bowse, client access link, and the Intenet. The sevice demand at the bowse is the client think time.

7 The sevice demand at the client access link is the total time to tansmit equests and eceive eplies. It is assumed that client access link is dedicated, as is the case with modem access. When this is not tue, as in the case of LAN-based access, we used the effective bandwidth seen by the client to compute the sevice demand. Using the solution methods fo mixed QNs given in [MAD94], we obtained the esponse time as a function of the client-side access speed. This is depicted in Fig. 9 fo class 4 fo an aggegate aival ate fo the open classes equal to 120 tps and fo an Intenet ound tip delay of 200 msec. Response times vay fom 9.51 sec to 6.70 sec depending on the client-side access speed. Note that fo the same aival ate of 120 tps, the seve-side esponse time obtained fom the open QN model is 6.66 sec. This model explains the lage vaiability in esponse time obseved in the log. The esponse times fo equests coming fom campus ove a high speed ATM backbone and 100 Mbps LANs only wee moe in line with those pedicted by the open QN model. Bowse Client access link Intenet Outgoing link Incoming link fom/to othe souces Disk 1 CPUs utilized. We chose this tool because it povided two key featues; we could use actual log activity to dive the benchmak, and we could mimic the aival ate chaacteistics, which would maintain the bustiness of ou wokload. In fact, if we look at a plot of the aival ate ove time we see clealy the busty natue of the wokload. Figue 10 illustates the effects of bustiness. This gaph is a 191-second inteval andomly taken fom the access log on Apil 26, 1999 at 15:37:05, showing the aival ate in equests pe second. The bustiness is clealy visible. While the aveage aival ate ove this inteval is only equests pe second, the busts ae consideably highe. We geneated an InetLoad scipt by pocessing the poduction log to obtain the file efeences and the equest inte-aival times. Busts of equests ae handled in the following manne: if 10 equests aived within one second, the scipt will ensue that the benchmak will equest those 10 files within one second. This is how we maintain the exact natue of the wokload. The aival ate can be inceased without compomising the aival ate chaacteistics, by compessing the eal time inteval duing which an inteval of log activity will be pefomed. This can be accomplished by educing the inte-aival time. Disk 2 Disk 3 Figue 8 Mixed QN Model Response Time (sec) ,600 56,000 1,250, ,000,000 Client access speed (bps) Figue 9 Response Time vs. Client-side Access Speed. Ou analytic model showed a satuation aound 120 tps. We wanted to veify though measuements if this was indeed the case. To that end, we an a benchmak using the InetLoad benchmak tool [InetLoad] to dive the seve to its maximum capacity, detemined by a CPU, ou bottleneck, close to 100% Figue 10- Busty Natue of the Wokload. We an ou benchmak seveal times and aveaged the thoughput obtained. Simultaneously with each un we had iostat active so that we could veify that both CPUs wee close to 100% utilized. The aveage thoughput obseved was 131 tps, 9.2% above the maximum thoughput pedicted by ou model. To detemine the maximum aival ate unde nonbusty conditions, we used InetLoad to pefom a benchmak consisting of equests submitted at a constant ate. The non-busty benchmak was able to dive the seve at 202 tps, well above the values obtained with the busty wokload. These findings cooboate the ones given in [Banga97].

8 Additional discussion of load testing can be found in [Michalsky98]. 7. Answeing What-If Questions The univesity is consideing moving the Web seve to a machine with RAID disks fo inceased fault toleance and a diffeent pocesso. One of the machines being consideed by the administation had two slightly slowe pocessos. We advised against it knowing that the CPU was the bottleneck. The othe option had to 250 MHz pocessos. To obtain a vey ough estimate of the gains to be achieved by using the faste pocessos we an ou model again. We an ou model again by multiplying the CPU sevice demand by 168/250 = To account fo the incease in pocesso speed. We could also have used the atio of SPECint95 atings. Fo an aival ate value of 120 ts, class 4 now has a esponse time of sec as opposed to 6.6 sec with the slowe pocessos. Also the maximum thoughput suppoted by the Web seve inceases to 183 tps, an incease of about 52% with espect to the pevious situation. So, ou ecommendation to the administation was to upgade to the configuation with two 250 MHz pocessos. [MENA94] Menascé, D. A., V. A. F. Almeida, and L. W. Dowdy, Capacity Planning and Pefomance Modeling: fom mainfames to client-seve systems, Pentice Hall, Uppe Saddle Rive, NJ, [MENA98] Menascé, D. A. and V. A. F. Almeida, Capacity Planning fo Web Pefomance: metics, models, and methods, Pentice Hall, Uppe Saddle Rive, NJ, [Michalsky98] Michalsky, R. J., Load Testing in an Intenet Wold, Poc CMG Confeence, Dec. 6-11, Anaheim, CA, pp [SSS87] Seidman, A., P. Schweitze, and S. Shalev- Oen, Computeized closed queueing netwok models of flexible manufactuing systems, Lage Scale Syst. J., Noth Hollland, vol. 12, pp , Concluding Remaks The pupose of this pape was to pesent a vey pactical view of how we applied a model-based capacity planning methodology to a poduction webseve. We also discussed whee things went appaently wong and what steps had to be taken to validate and calibate the model. One of the biggest challenges in any capacity planning and pefomance modeling study is making sue one has the ight data, both fo the pupose of computing input paametes to analytic models, as well as fo validating model outputs such thoughput, utilization, and esponse times. Refeences [AW96] Alitt, M. and C. Williamson, Web Seve Wokload Chaacteization: the seach fo invaiants, Poc ACM Sigmetics Confeence, Philadelphia, May [Banga97] Banga, G. and P. Duschel, Measuing the Capacity of a Web Seve, Usenix Symposium on Intenet Technologies and Systems, Decembe [CB96] Covella, M. and A. Bestavos, Self-Similaity in Wold Wide Web Taffic: Evidence and Causes, Poc ACM Sigmetics Confeence, Philadelphia, May [InetLoad] d.htm