Model-Based Monitoring in Large-Scale Distributed Systems

Transcription

1 Model-Based Monioring in Large-Scale Disribued Sysems Diploma Thesis Carsen Reimann Chemniz Universiy of Technology Faculy of Compuer Science Operaing Sysem Group Advisors: Prof. Dr. Winfried Kalfa Dr. Sven Graupner

2 Model-Based Monioring in Large-Scale Disribued Sysems Reimann, Carsen p. Chemniz Universiy of Technology, Faculy of Compuer Science, Operaion Sysem Group Diploma Thesis By: Carsen Reimann Born 05/07/1978 in Zwickau, Germany Issued: 03/01/2002 Published: 05/31/2002

3 Task The goal of his work is o define a model descripion of a chosen disribued applicaion running in he CLIC environmen, represen his model wih is elemens and parameers in he daabase buil in he previous projec and build a monioring infrasrucure which allows o consanly calibrae model parameers such ha a any ime a realisic picure of he applicaion s behavior can be obained. Anoher goal is o collec saisics (races) of how model parameers have evolved over ime. Experimens conclude he work demonsraing he concep and implemenaion. Following seps are proposed: 1. Choosing a disribued applicaion environmen 2. Idenifying componens and links wih associaed parameers represening he behavior of ha applicaion 3. Idenifying appropriae merics o express behavioral characerisics 4. Mapping he model ino a daabase schema 5. Exernal presenaion of model informaion 6. Sensor infrasrucure 7. Experimenaion in he CLIC

4

5 Absrac Monioring remains an imporan problem in compuer science. This hesis describes which monior informaion is needed o analyze disribued service environmens. This hesis also describes how o ge hese informaion and how o sore hem in a monioring daabase. The resuling model is used o describe a disribued media conen environmen and a simulaion sysem ha runs on he CLIC helps o generae measuremens as in real sysems.

6

7 7 Conens 1 Inroducion Scenario Goal Precursory Work Daa Srucure Inerface Browser Adaper Overview Modeling Services Inroducion Basic Services Characerisic Parameers of Basic Services Disribued Services Characerisic Parameers of Disribued Services Summary Geing Characerisic Parameers Measuremens Evaluaing Service Parameers Geing Measuremens Measuremen Failure Sensors Locaion Aciviy Daabase Sorage XML Represenaion Curren Sae Trace Disribued Media Conen Overview Sysem Model Sysem Characerisics Cusomer Behavior Sysem Srucure Typical Parameers of Media Conens...31

8 8 Conens Typical Cusomer Behavior Service Models Sensor Summary Daabase Sorage XML Represenaion Simulaion wih Service Models Inroducion Model Srucure Policy Requiremens Mehods Synchronizaion Implemenaion Simulaion Processes Configuraion Browser Adaper Overview XML-Message Examples A Cause for Media Ceners Service Model Configuraion Resuls Conclusion Preloading Media Conen Service Model Configuraion Resuls Conclusion Conclusion Possibiliies for he Fuure Relaed Work Acronyms Figures Bibliography...71

9 9 1 Inroducion 1.1 Scenario Monioring remains an imporan problem in compuer sysems. Research a HP Labs anicipaes ha compuer sysems and heir managemen sysems will become much larger in fuure. Exising monioring echniques such as hose used in producs like HP OpenView are assumed o reach heir limis for several reasons. They collec oo fine-grained daa such as uilizaion of neworks, machines, sorage and oher hardware componens. The goal of a projec a HP Labs called self-organizing services is o beer undersand sysem behavior a higher-order sofware and service layers. Since hardware infrasrucure is shared among differen applicaions and services, i is hard o derive applicaion- and service-level behavior from he colleced monioring daa. Though echniques have been developed for deriving and condensing applicaion shares from aggregaed load in a boom-up fashion, hese approaches are complicaed and only applicable in cerain areas. 1.2 Goal The goal of his work is o define a model a applicaion layer of disribued sysems. I aemps o find ou which merics can be idenified for individual services ha characerize heir behavior and how hese merics can be obained from monioring a real sysem. These monioring daa should be sored ino he daabase buil in he previous projec [XMDB]. Therefore he model mus be mapped ino a daabase schema o enable adjusmen of model parameers by consanly monioring hem laer in he real applicaion sysem. The measured values need o be analyzed in wo ways. Firs, he saisical curren model descripion wih a se of model parameer obained by measures in he real sysem. Second, i conains saisical races showing how model parameers have changed over ime. The daabase schemaa for he model and for he hisory races should be exposed in form of separae downloadable XML [XML] documens ouside he daabase allowing he discoverabiliy of he daa srucure. The model informaion mus be made exernally accessible by defining queries over hese daa ses. The exising XML and browser inerface of he daabase should be used for his purpose and possibly be exended.

10 10 1 Inroducion Anoher goal is o use he model by choosing a disribued applicaion for monioring. The applicaion should use muliple componens ha communicae among each oher.

11 11 2 Precursory Work The precursory work was a daabase build by Koenig, R. and Reimann, C. I allows o sore monioring daa ino hierarchically srucured measuremen ables. The daabase operaes wih an XML-inerface for daa access and daa manipulaion. I uses HTTP for communicaion and is accessible by sandard browsers like Inerne-Explorer and Nescape Communicaor. The previous work is compleely described in [XMDB]. 2.1 Daa Srucure All monioring daa can be srucured hierarchically for sorage in he daabase. This srucure is comparable o he organizaion of file sysems. There are folders o organize he srucure and here are parameer ables as conainers for measured values. A simple example of heir use is shown in Figure 2-1. I conains he organizaion of he CLIC ( Chemnizer Linux Cluser ): There are differen machines (e.g. clic2f23 ) and hey each have daa ables for measuring values, in his example memory and cpu. CLIC folder +clic2f23 folder +memory daa able +cpu daa able +clic3i31 folder +memory daa able +cpu daa able Figure 2-1: Simple srucure in daabase Every iem in he ree srucure has a parameer ha shows is acual sae. There are 3 differen possible saes: - Acive The daa able is ready for measuremen values - Inacive The daa able is emporarily no ready for measuremen values - Closed Measuremen is complee and only reading access is possible Measuremen daa o be sored ino a daa able consis of wo values. One is he imesamp and he oher is he dedicaed daa value. The daabase is able o sore differen daa ypes ino he daa ables. Possible ypes are Ineger-, Floa- and Sring-Values. A few ypical examples are shown in Figure 2-2.

12 12 2 Precursory Work Real-values: Time Value :21: :21: :21: :21: :21: :21: Sring-values: Time Value :32:41 WAIT :32:43 RECEIVE :32:44 COMPUTE :32:45 WAIT :32:46 WAIT :32:48 SEND Figure 2-2: Parameer able examples 2.2 Inerface All communicaions wih he daabase (e.g. inser records or manipulae ree srucure) are hough XML. Tha means ha all commands o he daabase are in he form of XMLdocumens. The exchange of hese documens is realized wih he servle echnology and uses HTTP as ranspor proocol. HP Labs offered his messaging sysem. There are 3 ypes of command messages: - Queries Used for geing sored daa values - Manipulaions Used for inserion, changing or deleion of daa values - Managemen Used o manage he hierarchical srucure (e.g. creae ables) Figure 2-3 shows a sample message for a daa query from he cpu -measuremens of he clic2f23 machine in he CLIC. All oher command messages are buil similarly. <?xml version="1.0" encoding="utf-8"?> <body> <daaquery> <ableselec> <sub name="roo" /> <sub name="clic" /> <sub name="clic2f23" /> <sub name="cpu" /> </ableselec> <inerval> <begin daetime=" :00:00" /> <end daetime=" :30:00" /> </inerval> </daaquery> </body Figure 2-3: Message for daa query

13 2.3 Browser Adaper Browser Adaper The daabase also offers a user inerface, which can be used wih sandard browsers. This inerface, a so-called Browser Adaper, communicaes hrough he XML inerface described in secion 2.2 and is also based on servle echnology. This user inerface offers a comforable use of he daabase and presenaion of monioring daa. The presenaion of monioring daa can be exual and also graphical by using SVG, a language o describe wo-dimensional graphics in XML. 2.4 Overview Figure 2-4 shows he main componens of he daabase and how hey cooperae. Tomca 3.3 is used as a HTTP-server and servle engine. And PosgreSQL is used as a relaional daabase. HTTP- Server Browser- Adaper XML-Inerface DB Figure 2-4: Main componens of he daabase

14

15 15 3 Modeling Services 3.1 Inroducion Typical disribued sysems consis of differen aciviies, which are communicaing wih each oher [MVS]. For he analysis of disribued sysems we need a model ha represens he whole sysem. The model should suppor performance analysis of he sysem. Tha means o find ou: Where are he bolenecks? And, which componens are rarely used? Quasi: Analyze a sysem for an opimizaion. A curren performance model of disribued sysems is he queuing nework model. I suppors he needed analysis of disribued sysems. Therefore his model is he fundamen of he following service model. 3.2 Basic Services Services are aciviies, which process cerain jobs. They are hosed on an infrasrucure called server, his may be a simple compuer, a muli processor machine, a cluser sysem and he like. Several services can be hosed on such a server: Server Services Figure 3-1: Services on a server The general procedure of a service process is: Waiing for a job, receiving job informaion from a clien, processing he job, sending he resul of he job back o he clien and waiing for nex job. The clien is anoher process, which is hosed on he same server or on a remoe sysem (see chaper 3.4.) Jobs can be e.g. ransacions on a daabase, HTML-requess o a web-server or media requess o a media conen server in a VoD sysem.

16 16 3 Modeling Services Waiing for jobs Complee (send resuls) Receive job Processing Job Figure 3-2: The general saes of a service process If we look a a web-server on a Linux machine, he server will be he compuer on which he web-server process (e.g. Apache [AWS]) runs. The service process will be he web-server process. The saes of his service process are shown in Figure 3-3. The web-server sleeps unil a clien requess a HTML-page. Afer he reques he server provides he requesed HTML-page and he web-server sends i hen o he clien. Waiing for HTTP-requess Complee (send HTML-page) Receive HTTP-requess Geing HTML-page Figure 3-3: Saes of a web-server service process 3.3 Characerisic Parameers of Basic Services A basic service can be reaed like a single queuing saion in a queuing nework [PCCS], [MLR]. Imporan quesions are: How many jobs can he service process? How many jobs ges he service? These are parameers, which depends on a period of ime: ime We can define he following parameers: Figure 3-4: Period of ime for he parameers λ (, ) : The number of incoming jobs per second (reques rae) in a paricular ime period

17 3.4 Disribued Services 17 µ (, ) : The mean number of jobs per second, which he service is able o process 1 : The mean ime o process one job µ (, ) λ(, ) ρ (, ) = : The uilizaion of a service a a special ime period µ (, ) λ = 5 1 s µ = s λ p = = 0,05 µ Figure 3-5: A simple example of characerisic parameers 3.4 Disribued Services In he previous chaper we analyzed basic services. In pracice, many sysems consis of muliple, disribued services [MDS]. Disribued services means ha here are differen services on differen servers. These services depend on each oher; one service needs oher services for he compleion of a job. Services are in relaion wih each oher, hey form a kind of nework. Disribuion also means ha several services, on differen servers, offer he same scope of funcion. Which service is used can be chosen by he clien or a disribuing aciviy. This needs a sligh change in he service sae model: Waiing for jobs Complee (send resuls) Receive job Processing Job Receive resuls Reques remoe job Waiing for remoe job Figure 3-6: Saes of disribued service processes

18 18 3 Modeling Services 3.5 Characerisic Parameers of Disribued Services Le us ake a look a a simple scenario on Figure 3-7, which consiss of hree service processes on hree servers. A LAN connecs hem. Service process 1 is a web-server, which ges jobs (HTTP-requess) from web-browsers. Service processes 2 and 3 are daabase processes he webserver accesses. The sequence may be: The web-server ges a reques from a web-browser, reads values from a daabase and sends he resul back o he web-browser. The web-server ges requess wih rae λ 0, 1 and needs daa from service process 2 wih rae λ 1, 2 and from service process 3 wih rae λ 1, 3. I mus be poined ou ha all raes depend on a special ime period (, ) λ 0,1 λ 1, λ 1,2 2 Figure 3-7: Scenario for disribued services The nework creaed by disribued services can be analyzed wih queuing neworks [ALVS], [ICPE], [MLR], [PCCS]. Thus we can generally define he following parameers for disribued services: λ (, ) : Reques rae o service process n from an exernal clien 0, n λ (, ) : Reques rae from service process n o service process m n, m µ (, ) : Capaciy of a service process n n 1 : Mean ime o process one job a service process n µ n (, ) ρ (, ) = n λm, n (, ) : Uilizaion of service process n µ (, ) m n

19 3.6 Summary 19 Addiionally, i mus be noed ha requess o remoe services ake ime and generae nework load. This ime resuls from he summaed ime for communicaion and he ime waiing for he resul. Thus we can addiionally define he following parameers: ω (, ) : Mean ime one reques akes from service process n o service process m n, m η (, ) : Mean nework load in KB caused by one reques from service process n n, m o service process m 3.6 Summary We use a kind of queuing neworks for he analysis of disribued service environmens. Wih he queuing neworks we have a model o describe he behavior of disribued services wih sochasic parameers. Queuing neworks do no conain any parameers of communicaion expanse. Tha is why we added he parameers ω (, ) and η (, ). How all named n, m parameers of his model can be measured is described in chaper 3.7. n, m 3.7 Geing Characerisic Parameers Measuremens In his secion he measuremens are defined, which can be derived from real services o evaluae he service parameers [MLR]: : Period of ime he measuremen runs r n ( ) : Number of requess o service process n in period u n ( ) : Time used for processing all requess a service process n in period his includes he waiing ime for oher service processes ( ) w n, m n, ( ) : Number of requess from service process n o service process m in r m period n, ( ) : Time service process n wais for service process m in period w m

20 20 3 Modeling Services ) (, n m n : Summaed nework load produced by requess from service process n o service process m in period Evaluaing Service Parameers For evaluaing characerisic service parameers, he following formulas can be used: : Time when measuremen period ends r n n = ) ( ), ( λ ) ( ) ( ), ( 1 r u n n n = µ r m n m n = ) ( ), (,, λ ) ( ) ( ), (,,, r w m n m n m n = ω ) ( ) ( ), (,,, r n m n m n m n = η Geing Measuremens In order o ge he measuremens we need a kind of sensor ha collecs he daa. This sensor could be embedded ino he service process or in he environmen of he service process. In any even, he daa he sensor collecs are he same. Le us review he behavior of a service (Figure 3-6): The service is waiing for jobs, processes jobs and sends jobs o oher services. The sensor could ake noe of every sae ransiion and couns he received and sen byes.

21 3.8 Sensors 21 Waiing for jobs Complee (Send resuls) Receive job Noed by sensor Receive resuls Processing Job Reques remoe job Sensor Waiing for remoe job Figure 3-8: evens logged by he sensor For jobs he service receives by iself, he sensor does no have o coun he received and sen byes, because he sensor of he service, which sends he job, will do ha already Measuremen Failure A sensor is an aciviy, hus i needs resources like processor ime or nework capaciy for communicaing wih he daabase. So he sensor falsifies he measuremens, e.g. he processing ime of a reques or somehing else. We assume ha his influence is very sligh, hus we will no include i in our consideraions. The oher kind of measuremen failures is ha some requess are sill in process when a measuremen period is already finished. Thus here will be a failure in he processing ime measures. This influence is also very lowly, because he measuremen period is usually much longer han a reques processing ime. Therefore, hose influences will no be apar of our consideraions, eiher. 3.8 Sensors Locaion The locaion of he sensor may vary, i can be: - Inegraed in he service process - Anoher process on he same server - Anoher process on a server nex o he service process

22 22 3 Modeling Services The firs locaion, inegraed in he service process, is very suiable, because you direc have access o all informaion you will need. You can do his by changing he source code of he service process or by changing libraries used by he service. If he locaion is anoher process hosed by he same server, i mus be a process ha has access o he process conrol in order o ge needed informaion. Tha means he process needs sysem privileges provided he hos operaing sysem suppors process privileges. Alernaively, he sensor could read log-files from he service process o ge he informaion. For oher processes on a differen server he only way o ge he informaion i needs is o ap he communicaion beween service processes from he communicaion media. This creaes many problems in idenifying service processes if here are more han one hosed by he server. Bu you do no need o change any source code and you can have measures wihou influence of an inernal sensor (see 3.7.4) Aciviy The sensor ges he informaion of he ransiion evens. This includes he ime of he ransiion and he new sae of he service. The sensor ges also informaion abou he expense of communicaion. The informaion sequence will be: Sae<Waiing for jobs> Even<Incoming job> Even<Job complee> Sae<processing job> Even<Remoe job> Even<Remoe job complee> Sae<Waiing for compleion> Even<Receive/Send n byes> Figure 3-9: Informaion sequence of he sensors The colleced informaion could be compleely sored or summarized. By soring every single even he sorage space of he sensor will overflow quickly provided he even rae is very high. So i is absoluely necessary ha he informaion mus be summarized. This can be easily

23 3.9 Daabase Sorage 23 realized by couning every even ype using a separae variable. The number of remoe job evens mus be separaed for each remoe service, which is used. Now we have all needed measuremens: = Time since he couning variables were zeroed r n ( ) = Number of Even<Incoming job> u n ( ) = Summaed ime beween Even<Incoming job> and Even<Job complee> r m n, ( ) = Number of Evens<Remoe job> by using service m w m n, ( ) = Summaed ime beween Even<Remoe job> and Even<Remoe job complee> by using service m n m n, ( ) = Number of byes ransmied o and from service m The measured daa should be sored in he daabase. This can be realized by insering all colleced values ino he daabase afer a ime period. Afer ha, all variables will be zeroed for he nex period. Every sensor is direcly conneced o he daabase and works independenly from all oher sensors. So we do no need a cenral insance for daabase access. This disribuion of sensors has an imporan disadvanage: Every measuremen value depends on a poin of ime. Bu he ime in such a sysem may differ from uni o uni. So here is a need for synchronizaion. Time synchronizaion is a known problem in disribued sysems and here are some possible soluions like Crisian s Algorihm or he Berkeley-Algorihm hey are described in [MBS]. 3.9 Daabase Sorage The daabase is able o sore srucures and also measuremen values. So we can sore he srucure of he disribued service environmen wih all measured daa a once. To sore he srucure of he service environmen you mus know heir componens. I consiss of differen servers wih a link for communicaion and every server is a hos for muliple services. Firsly, we sar wih idenifying he whole environmen by a folder in he daabase. This should be he opfolder. The nex sep is o presen every server by a folder wih he environmen-folder as

24 24 3 Modeling Services faher. The same procedure needs o be done wih all services. The resul of hese seps may be as shown in Figure Bu here are also oher hierarchies possible, e.g. which conain subnes or he like Measuremen +Server1 +Service1 +Server2 +Service2 +Server3 +Service3 Figure 3-10: Possible daabase srucure Now we can inegrae he measuremen value ables. There are wo measuremens ha only belong o he service iself and should be direcly sored in he service folder: r n ( ) and u n ( ). The oher values are relaed o oher (remoe) services and should be associaed wih hem. The resuling able srucure is shown in Figure Measuremen +Server1 +Service1 +reques +process +Remoe +Service2 +reques +wai +nework +Service3 +reques +wai +nework +Server2 +Service2 +reques +process +Server3 +Service3 +reques +process Figure 3-11: Possible represenaion of he hree-server sysem 3.10 XML Represenaion The XML forma should be used for exernal represenaion (see 1.2). I should be possible o exrac he curren sae and also a sae race. There are differen XML srucures possible. You can describe all componens separaely and han describe all connecion wih a lis of links. One possible srucure is shown in Figure 3-12 (noe: I is he srucure scenario picured in Figure 3-11). The measuremen daa can be sored ino he ags of services or links.

25 3.10 XML Represenaion 25 <Server1> <Service1 /> </Server1> <Server2> <Service2 /> </Server2> <Server3> <Service3 /> </Server3> <Links> <Link from= Service1 o= Service2 /> <Link from= Service1 o= Service3 /> </Links> Figure 3-12: XML represenaion 1 Anoher possibiliy is o use he able srucure of he daabase for soring he informaion ino an XML-file. The scenario shown in Figure 3-11 may look like he following: <Server1> <Service1> <Remoe> <Service2 /> <Service3 /> </Remoe> </Service1> </Server1> <Server2> <Service2 /> </Server2> <Server3> <Service3 /> </Server3> Figure 3-13: XML represenaion 2 Of course here are more possibiliies bu he second represenaion is used in his work because i reflecs he used daabase srucure and his is useful when searching for paricular measuremens or he like Curren Sae The curren sae depends on he measuremen period (, ), hus here is ime informaion needed in he XML represenaion. This could be a simple ag, which conains he poin of ime, when he period ends and he duraion of he period in seconds. If we use global ime informaion we do no need o sore he ime wih any value. Tha way sorage space for he XML-file can be reduced. The nex sep is o sore he measuremens in he XML-file. This can

26 26 3 Modeling Services be realized by using he able srucure described in chaper Thus he resuling srucure will be: <Time value= :23:20 duraion= 300 /> <Server1> <Service1> <reques> </reques> <process> </process> <Remoe> <Service2> <reques> </reques> <process> </process> </Service2> </Remoe> </Service1> </Server1> Figure 3-14: Curren sae XML represenaion The values of he measuremens can be sored ino ag <Value>. The resuling XML-file will have he following srucure: <Time value= :23:20 duraion= 300 /> <Server1> <Service1> <reques> <Value>12</Value> </reques> <process> <Value>0.412</Value> </process> <Remoe> <Service2> <reques> <Value>3</Value> </reques> <process> <Value>1.04</Value> </process> </Service2> </Remoe> </Service1> </Server1> Figure 3-15: Curren sae

27 3.10 XML Represenaion Trace The srucure of curren sae represenaion can be used for he race represenaion. The difference is ha here are more values of measuremen and he duraion propery is no needed any more. A simple way o realize his is o use a sequence of <Time> -ags by giving hem a sequenial number. The same should be done wih he <Value> -ag. Thus we have global ime informaion abou he race and all values in one XML-file. <Time number= 1 value= :23:20 /> <Time number= 2 value= :23:25 /> <Time number= 3 value= :23:30 /> <Server1> <Service1> <reques> <Value number= 1 >12</Value> <Value number= 2 >2</Value> <Value number= 3 >4</Value> </reques> </Service1> </Server1> Figure 3-16: Trace represenaion

28

29 29 4 Disribued Media Conen 4.1 Overview Media-on-demand has become popular, especially video-on-demand bu also music-on-demand. There are differen sysem srucures possible, bu here we use he following: A media-ondemand sysem has hree componens: Cliens (Cusomer), a nework and media archive servers. Differen media archive server will be disribued worldwide o provide cliens wih media conen. Media archive servers are classified o conen provider and media cener. Media cener are minor media archives, which are placed in each housing area. Cliens will send all requess direcly o hese media ceners and are direcly conneced o hem. If he conen a clien asks for is no sored on he media cener, he media cener will ask a neighboring media cener or a conen provider. They funcion as a local cache for media conen. Conen providers are cenralized servers, which disribue he media conen o he media ceners. The conen on providers may vary beween differen regions. Conen provider Nework Media cener Cusomer Figure 4-1: Disribued media conen scenario 4.2 Sysem Model To describe he scenario here are wo caegories of parameers: Sysem characerisics and cusomer behavior [LTRA], [TFIC], [IMA] Sysem Characerisics Characerisic parameers, which describe he sysem capaciy, are for example: - Number of simulaneous media sreams a media server can suppor

30 30 4 Disribued Media Conen - Sorage capaciy of a media server - Available media conen - Sysem srucure - Nework bandwidh of he connecions beween separae media servers - Number of cusomers conneced o a media server - Policy of geing media conen - Processing ime of a reques Cusomer Behavior For cusomer we can define he following parameers in order o describe heir behavior: - The reques arrival paern or workload a he server - Media conen selecion paerns Sysem Srucure The disribued media conen sysem is srucured hierarchically. The upper hierarchy consiss of he conen provider, he nex hierarchy consiss of he media cener and he boom hierarchy consiss of he cusomers. Cusomers are conneced o one media cener and media ceners are conneced o several conen providers and also o media ceners in heir neighborhood. The sysem srucure is shown in Figure 4-2. We can assume ha abou 100 o 1000 cusomers are conneced o a media cener and basically every media cener o every conen provider. Media ceners are also conneced o media ceners in heir neighborhood o compensae failures on single media ceners by using he neighboring media cener s services.

31 4.2 Sysem Model 31 Conen provider Media cener Cusomer Figure 4-2: Sysem srucure Typical Parameers of Media Conens If we assume ha movies are compressed wih he MPEG-1 echnology [MPG] a movie ransmission will have a rae of 1.5 Mbps [OVD]. Wih a ypical lengh of abou 105 o 135 minues a movie needs beween 1.1 GB and 1.4 GB of sorage capaciy. Music daa wihou compression has a rae of 1.4 Mbps (44.1 khz, 2 channels wih 16 Bi), by using compression like MP3 or similar here are raes of 128 kbps possible. Thus a whole CD of audio daa needs approximaely 60 MB of sorage capaciy Typical Cusomer Behavior Cusomers have a ypical reques paern by using media conen: There is a prime-ime during he day, which is characerized by high workload on he server because several cusomers wan o see a movie. The res of he day cusomers may only lisen o music or he workload is low because people work or sleep. This prime ime depends on he region of counry (e.g. Germany: 9:00 p.m. or Spain: 10:30 p.m. [TSOV]). Finally we can classify hree phases of cusomer behavior: Sleeping ime, prime ime and he remaining ime. A he prime ime everybody wans o see a movie, and so here are high raes of movie requess a he media ceners. The sleeping ime is a period when nobody wans o see a movie or lisen o any music. The reques rae on media ceners is very low a his ime. A he remaining ime people only lisen o music and someimes wach a movie. The simplified behavior of a cusomer is shown in Figure 4-3.

32 32 4 Disribued Media Conen Requess rae Music Movies Remaining ime Prime ime Day ime Figure 4-3: Simplified cusomer behavior Cusomers do no choose he conen compleely randomly. There are some movies or music conens cusomers ofen wan o see or lisen o i. These are curren conens like new movies or new songs. 4.3 Service Models The goal of monioring a disribued media conen environmen and is service model is o ge informaion abou he uilizaion of special media server, he relaions beween differen media ceners and he used nework capaciies. The model described in chaper 3 delivers all hese informaion, hus we can compleely use his model for he monioring of disribued media conen environmens. Services are he paricular media server (media cener and conen provider) and cusomers are creaing requess o media servers Sensor The sensors are measuring he parameers of disribued services as described in Excepions are he parameer u n ( ) and w n, m ( ). Normally hey would represen he ime in which a service complees is work; here his would nearly (processing delays) be he movie lengh or he music duraion because he service is complee if he sream is ransmied. More imporan for his scenario is o measure he processing ime, which is needed before a sream begins. So u n ( ) is he summaed ime beween he incoming reques and he sar of ransmission. Wih w n, m ( ) i is similar. Afer he measuring period he sensor sends he parameer o he daabase. The daabase srucure is described in The adjused sensor noaion is shown in Figure 4-4. Basically, i is he same as in Figure 3-8, bu for beer undersanding he process saes are renamed.

33 4.3 Service Models 33 Transmission sars Waiing for media requess Receive reques Noed by sensor Transmission sars Processing reques Reques remoe media Sensor Waiing for remoe media Figure 4-4: Noaions by he sensor Summary For he service model of he disribued media conen environmen we can summarize: Sysem Configuraion Μ : Available media conen divided ino music conen and movie conen a number represens a media ile. 1 1 µ n : Reques processing ime of media cener n (ime beween incoming reques and sar of ransmission) m n : Sorage capaciy of media cener n c n : Number of cusomers conneced o media cener n η n,m : Nework capaciy in KB from media cener n o conen provider m Μ m Μ : Media conen on conen provider m 1 2 µ m : Reques processing ime of conen provider m χ m : Available ransmission channels on conen provider m

34 34 4 Disribued Media Conen Model Parameer In comparison o chaper 3.5 we can define he following parameers, which reflec he curren sae of he model: Media Cener ), ( 1, 0 n λ : Rae of cusomer reques rae o media cener n ), ( 1 n µ : Reques processing capaciy of media cener n ), ( 1 1 µ n : Mean ime o process one reques a media cener n ), ( ), ( ), ( 1 1 0, 1 n n n = µ λ ρ : Uilizaion of media cener n ), ( 1, 0 m ω : Mean ime one cusomer reques akes o media cener n ), (, m n η : Nework load in KB from media cener n o conen provider m Conen Provider ), ( 2, m n λ : Reques rae from media cener n o conen provider m ), ( 2 m µ : Reques processing capaciy of conen provider m ), ( 1 2 µ m : Mean ime o process one reques a conen provider m ), ( ), ( ), ( 2 2, 2 m n m n m = µ λ ρ : Uilizaion of conen provider m ), ( 2, m n ω : Mean ime one reques akes from media cener n o conen provider m Measures The following measures are needed o ge he model parameer: : Period of ime he measuremen runs

35 4.3 Service Models 35 Media Cener r 1 n ( ) : Number of requess o media cener n in period u 1 n ( ) : Cusomer reques processing ime on media cener n in period r m n, ( ) : Number of requess from media cener n o conen provider m in period w m n, ( ) : Time he media cener n wais for conen provider m in period n m n, ( ) : Summaed nework load of requess from media cener n o conen provider m in period Conen Provider r 2 m ( ) : Number of requess o conen provider m in period 2 u m ( ) : Media cener reques-processing ime on conen provider m in period Daabase Sorage The daabase sorage is realized as described in 3.9, hus he resuling able srucure is: Conen Providers: DisribuedMediaConen +ConenProvider1Hos +ConenProvider1 +reques +process +ConenProvider2Hos +ConenProvider2 +reques +process +ConenProvider3Hos +ConenProvider3 +reques +process Media Ceners: +MediaCener1Hos + MediaCener1 +reques +process +Remoe +ConenProvider1 +reques +wai +nework +ConenProvider2 +reques +wai +nework Figure 4-5: Table srucure in daabase

36 36 4 Disribued Media Conen XML Represenaion The XML-represenaion of he disribued media conen environmen s sae is realized as described in chaper The only differences are he names of he servers and services. Curren Sae <Time value= duraion= /> <MediaCener1Hos> <MediaCener1> <reques> <Value></Value> </reques> <process> <Value></Value> </process> <Remoe> <ConenProvider2> <reques> <Value></Value> </reques> <process> <Value></Value> </process> </Service2> </Remoe> </MediaCener1> </MediaCener1Hos> Figure 4-6: Curren sae Trace <Time number= 1 value= /> <Time number= 2 value= /> <Time number= 3 value= /> <ConenProvider1Hos> <ConenProvider1> <reques> <Value number= 1 ></Value> <Value number= 2 ></Value> <Value number= 3 ></Value> </reques> </ ConenProvider1> </ ConenProvider1Hos> Figure 4-7: Trace represenaion

37 37 5 Simulaion wih Service Models 5.1 Inroducion The goal of he simulaion is o sore monior measuremens in a daabase ha are similar o he measuremens of real sysems. Thus you can idenify he ypical cusomer behavior (reques paern) and he media ransmission hrough he nework infrasrucure Model Srucure The simulaion model includes all componens of he disribued media conen scenario: Cusomers are conneced o a media cener and he media ceners are conneced o differen conen providers. The resuling srucure is as shown in Figure Policy The policy gives he working sraegy of he paricular sysem componens. I is imporan ha we are no concerned wih any kind of failures, such as a down conen provider or media cener. We assume ha all componens are working during he simulaion. Conen Provider A conen provider is waiing for a reques from a media cener. If a media cener requess media conen, he conen provider reserves a ransmission channel for his ransmission. If he conen provider has no channel available, he reques is refused. To see how many requess are refused we add a couning variable o he sensor, which is used like he sensor s reques couner. Afer a ransmission he conen provider de-allocaes he used resources. Media Cener Media ceners are waiing for requess from a cusomer. Afer a cusomer s reques for a media ile he media cener searches he conen in is local cache. If i is available, he media cener sreams he conen coninuously o he cusomer. If i is no available, he media cener asks he conen providers, which are conneced o he media cener, for his conen. When a conen provider sars he ransmission of he conen o he media cener, he sream will be handed over o he cusomer and sored in he media cener cache synchronously. For media replacemen in he cache an LRU-algorihm is used. The connecion o a neighboring media cener will no be reaed wih.

38 38 5 Simulaion wih Service Models Cusomer Cusomers are he producer of workload in he simulaion. They are requesing media conen as described in chaper I is imporan ha cusomers do no use any VCR-funcions such as fas-forward or sop and replay. This simplifies he simulaion so ha a cusomer only requess media conen one afer he oher Requiremens I should be simple o configure he simulaion. Tha means o conrol he cusomer s behavior and which componens (e.g. cusomer) are conneced o which oher componen (e.g. media server). And i is imporan ha he simulaion is independen from he compuing power of he hoss, which process he simulaion. Therefore we need a virual ime for simulaion. 5.2 Mehods In pracice here are various simulaion mehods [MSIM] for differen asks. The simulaion sysem consiss ypically of various ineracing objecs. Each objec changes is sae in ime. The differen simulaion mehods differ in way of he simulaion ime goes by: Coninuous Simulaion The simulaion ime goes by coninuously. This is useful for simulaing models of differenial equaions. Time-Based Simulaion The simulaion ime goes by in discree seps. Some poins of ime perform evens, which were processed. The disadvanage is ha here are poins of ime a which no even (deah ime) is performed, bu hey were considered oo. Tha means a wase of compuing ime. Even-Based Simulaion The simulaing ime coninues o he respecive nex even. Tha is a grea advanage because we are no concerned wih poins of ime he sysem sae is no changing. Thus we have no deah ime. For his simulaion we use he even-based simulaion, because i is a mehod, which compues only he imporan evens. This simulaion mehod will be he fases.

39 5.3 Synchronizaion Synchronizaion There are differen objecs in he environmen of disribued media conens (conen provider, media cener and cusomer). Thus i is possible o separae hem in he simulaion as differen processes. The disribuion has he posiive effec of speedup if all objecs are independen as much as possible so ha he synchronizaion of even processing is as idle as possible. For synchronizaion we use a conservaive simulaion algorihm because of he easier handling and implemenaion. In his simulaion algorihm every simulaion process handles is own even lis unil here is an even, which may depend on evens of oher processes. This form of dependence is considered o be causal. Two evens are causal (noed: e1 e2 ) if: - The ime of e1 is before he ime of e 2 and: Even e 1 changes a sae variable ha even e 2 reads Even e 1 reads a sae variable ha even e 2 changes Even e 1 changes a sae variable ha even e 2 changes oo - There is an even e x wih e 1 ex and e2 e x (ransiive) A simple example is he simulaion of wo inersecions wih wo processes. A car drives ou of inersecion 1 ino inersecion 2. The even "incoming car" on inersecion 2 mus be afer he even "ougoing car" on inersecion 1.

40

41 41 6 Implemenaion 6.1 Simulaion The simulaion is implemened wih he help of he programming language C, runs on Linux sysems and uses he Message Passing Inerface [MPI] for disribued compuing. Thus he simulaion sysem is able o run on he CLIC. For parsing he XML configuraion-file he free XML-library libxml [LIBX] is used Processes The simulaion is divided ino 2 process ypes. The firs is he process, which conains he funcionaliy of all conen providers. The second conains he funcionaliy of a media cener wih is conneced cusomers. For a scenario of 5 conen providers and 10 media ceners we need 11 processes (one for he 5 conen providers, en for he 10 media ceners). The processes use he MPI for iner-process communicaion (IPC). The resuling scenario is shown in Figure 6-1. Media cener / cusomer processes Conen provider process Figure 6-1: Processes of simulaion General Behavior Each process (conen provider and media cener) has is own even lis. An even lis is a imeordered lis of evens. Time means he poin of ime he even happens. For virual ime we use a resoluion of one second. Imporan is ha every even needs no ime for happening. An even may conain special informaion, which depends on he even ype.

42 42 6 Implemenaion A process akes he firs even (nex poin of ime of he simulaion) from he lis and handles his even. The following opions are possible: Creaing new evens or changing a process sae variable. If he even is processed i will be removed from he lis. The iniial even liss of every process were generaed a he sar of he processes (simulaion sar). An iniial even, which is generaed on every process, is he even Simulaion-End. If a process akes his even from he lis, i will finish is work. So we have a conrolled behavior for ending he simulaion. Conen Provider All conen providers were simulaed in one process. Thus he media cener processes communicae o one process only. This reduces he iner-process ime-synchronizaion. A conen provider has he following saus informaion: - Movie conens sored on he conen provider - Music conens sored on he conen provider - Number of free ransmission channels Globally for all conen providers, here is an array ha is used o sore he curren ime of all media ceners. This is used o avoid he violaion of causal evens. The conen provider handles only evens whose poin of ime is earlier or equal o he minimum ime of all media ceners. So he conen provider process handles no even unil here is no causal even a each media ceners process any more The conen provider process communicaes wih media cener processes hrough MPImessages. These messages conain a ime samp, which presens he poin of ime he message was send. Thus he conen provider process can updae he curren ime array of he media ceners. There are hree ypes of messages: - Media requess: Afer receiving his message, a Media Reques even is insered ino he lis. - Time synchronizaion: This message is only used for updaing he curren ime array. - Reply: Afer a media reques, his message is send back o he media cener process.

43 6.1 Simulaion 43 The conen provider process knows he following evens: - Simulaion-End : This even is used for a conrolled program erminaion. - Media Reques : This even is used o handle he media reques of a media cener o a conen provider. If he conen provider has sored he requesed media conen and a free channel is available, he conen provider reserves a channel and insers a Transmission End -even ino he lis. A las he conen provider process sends a message o he media cener process, which has requesed he conen, o indicae he success of he reques. - Transmission End : This even simulaes he end of a media ransmission; he used channel is de-allocaed afer his even. - Sensor Transmission : This cyclic even is used o perform he ransmission of he daa o he daabase, which was colleced by he sensor. Every conen provider has is own sensor daa, hus by handling his even he daa of all conen providers is sen. The policy of processing he conen providers is: 1. Ge he nex even from he even-lis 2. If he poin of ime of his even is afer he minimum ime of all media cener sep o 4 3. Process he even and go o sep Wai for messages from a media cener. 5. Process he message and go o sep 1. Media Cener A single process represens every media cener. Media cener processes simulae he behavior of he media cener and he behavior of conneced cusomers, oo. A media cener has he following saus informaion: - Cached movie conens sored on he media cener - Cached music conens sored on he media cener

44 44 6 Implemenaion - Free nework resources o each conen provider The MPI-messages are he same as he messages from he conen provider because he messages are ransmied beween hese componens. The following evens are used in media cener processes: - Simulaion-End : This even is also used for a conrolled program erminaion. - Media Reques : This even is used for simulaing a cusomer reques o he media cener. The media cener searches for he requesed media conen in is local cache. If he conen is no found, he media cener will send a Media Reques -message o he conen providers, whose connecion has free nework capaciy, and wai for a replymessage. The cached media conen will be replaced (incase he cache is full) by using he LRU algorihm. Once he conen is available a Transmission Begin -even is insered ino he even-lis and he nework capaciy for ransmission from he conen provider is reserved (if in use). If he conen is no available, a new Media Reques - even is creaed. - Transmission Begin : This even simulaes he sar of a media ransmission o a cusomer. Wih his even a Transmission End -even or a Transmission -even is creaed, depending on he source of he conen (conen can be sored local or be provided by he conen provider). - Transmission : This even simulaes he nework raffic beween a media cener and a conen provider, by adding he number of ransmied byes o he sensors daa. This even will occur every second unil he conen is ransmied. Afer ha a Transmission End -even will be creaed. - Transmission End : This even simulaes he compleion of a media ransmission. Thus he used nework capaciy will be de-allocaed. Following his, a new Media Reques -even will be insered ino he even-lis o simulae he nex reques of his cusomer. - Time Synchronizaion : This even is used for updaing he curren ime of he media cener on he array of he conen provider process. I is creaed cyclical o guaranee a coninuous synchronizaion.

45 6.1 Simulaion 45 - Sensor Transmission : This cyclic even is used o perform he daa ransmission from he sensor o he daabase. The policy of a media cener process consiss only of processing he even-lis. If a media cener sends a media reques o a conen provider, he media cener wais for a reply This reply is only send by he conen provider if all media ceners have he same curren ime or laer. Therefore we have a global synchronizaion hrough he conen provider process. Media requess from cusomers are depending o paricular dayimes (see 4.2.5). Every dayime period has is own probabiliy for he choice beween movie and music and a special probabilisic delay beween wo cusomer requess. Cusomers of a media cener have heir own preferred op-movies and heir own preferred op-music, he media conen, which is liked requesed (see 4.2.5). This is realized by a lis of op-medias and he probabiliy of choosing one of hem. Example For a beer undersanding of he behavior of he processes, we will provide a simple example of one conen provider and one media cener. There is one cusomer who requess a paricular movie (a ime = 10). The media cener s cache is empy and has o send a reques o he conen provider. Afer ha he simulaion will end (a ime = 10000). To simplify his example we are no deal wih sensor specific evens Transmission or he like: Sep 1: The iniial even liss are creaed. Conen provider: Simulaion-End Media cener: 10 Media Reques Simulaion-End Figure 6-2: Example (sep 1) Sep 2: The conen provider wais for messages from he media cener since he Simulaion- End -even is causal o he Media Reques -even from he media cener. The media cener process handles he firs even ( Media Reques ) and sends a reques-message o he conen provider and wais for a reply.

46 46 6 Implemenaion Conen provider: Simulaion-End Media cener: 10 Media Reques Simulaion-End Figure 6-3: Example (sep 2) Sep 3: The conen provider receives he message and insers a Media Reques -even ino is lis (wih he curren ime of he message). This even can be processed because he media cener is a he same curren ime as he even. The conen provider has he media conen and reserves a channel for ransmission and sends a reply-message o he media cener. Conen provider: 10 Media Reques Simulaion-End Media cener: 10 Media Reques Simulaion-End Figure 6-4: Example (sep 3) Sep 4: The Media Reques -even is deleed from he conen provider s lis; nework capaciy is reserved for ransmission and a new even Transmission End is added o he lis (he movie needs 36 seconds). Afer ha he conen provider has o wai for messages from he media cener (causaliy). The media cener removes he Media Reques -even and insers a Transmission End -even, oo. Conen provider: 46 Trans. End Simulaion-End Media cener: 46 Trans. End Simulaion-End Figure 6-5: Example (sep 4) Sep 5: The media cener handles he Transmission End -even (de-allocae used nework capaciy) and removes i from he lis. Conen provider: 46 Trans. End Simulaion-End Media cener: Simulaion-End Figure 6-6: Example (sep 5)

47 6.1 Simulaion 47 Sep 6: The media cener processes he Simulaion-End -even and sends a ime synchronizaion message o he conen provider. Thus he conen provider can handle he Transmission End -even (de-allocaes used channel) and he Simulaion-End -even. Afer ha he simulaion is finished. Conen provider: 46 Trans. End Simulaion-End Media cener: Simulaion-End Conen provider: 46 Trans. End Simulaion-End Media cener: finished Conen provider: Simulaion-End Media cener: finished Configuraion Figure 6-7: Example (final seps) The simulaion should be configurable in order o run differen simulaion-configuraions wih he same simulaion sysem. The simulaion can be configured hrough an XML-file, which conains he informaion abou he sysem srucure and he cusomer s behavior. Global Informaion A firs here are several global informaion, which are valid for all processes: - Time when simulaion sars: Represened by ag <Sar>, conains he full dae and ime of he poin of ime he simulaion sars (e.g.: :00:00). - Time when simulaion ends: Represened by ag <End>, conains he full dae and he poin of ime he simulaion ends (see above). - Lengh of measuremen period of he sensors: Represened by ag <SensorPeriod>, conains he number of seconds a sensor measuremen period akes.

48 48 6 Implemenaion - Name of he daabase server: Represened by ag <DaabaseServer>, conains he hosname of he daabase server. - Folder in daabase ables o sore measuremens: Represened by ag <DaabaseFolder>, conains he name of he folder, which is used for soring he measuremens. - Available media conens: All available media conens are colleced under he ag <Medias>, separaed in movie conen ( <Movie> ) and music conen ( <Music> ). Media conen is represened by ag <Media> and conains he duraion, daa rae and a coninuous number for idenificaion. Duraion gives he lengh of media conen in seconds and he daa rae is given in Kbps. Because of he possibiliy ha here are more han 1000 available media conens we need he funcionaliy of generaing media conen. This funcionaliy is also implemened: Wih he ag <Generae> we can generae media conens, which have he same daa-rae specified by he propery daarae and a specific duraion configurable by he properies minduraion and maxduraion. The number of he generaed media conens can be configured wih he help of he propery coun. <Sar> :00:00</Sar> <End> :59:59</End> <SensorPeriod>60</SensorPeriod> <DaabaseServer>domiian.informaik.u-chemniz.de</DaabaseServer> <DaabaseFolder>CLICSim</DaabaseFolder> <Medias> <Movie> <Media number= 1 duraion= 7380 daarae= 1500 /> <Generae coun= 2000 minduraion= 7000 maxduraion= 8000 daarae= 1500 /> </Movie> <Music> <Media number= 1 duraion= 320 daarae= 128 /> <Generae coun= 2000 minduraion= 180 maxduraion= 300 daarae= 128 /> </Music> </Medias> Figure 6-8: Example of global configuraion

49 6.1 Simulaion 49 Conen Provider All conen providers are colleced by ag <ConenProviders> and separaed by ag <ConenProvider>. Every conen provider has a coninuous idenificaion number, which is represened by he propery number. For he conen provider we need he following informaion: - The sored media conen on he conen provider: The media conen, sored on he conen provider is represened like he global media conen. Bu we do no need he duraion and he daa rae, because hose are global parameers. Thus we only need he number, which refers o he number of he global media conen. I is also possible o generae a se of sored media conens by using ag <Generae>. The ag has he properies begin and end which means he firs number and he las number of he media conens ha are generaed. - The number of available ransmission channels: Represened by ag <Channels>. - The processing delay for a reques: Represened by ag <Delay>, conains he processing ime in seconds (floaing poin values are possible). Have a look a his shor example: <ConenProviders> <ConenProvider number= 1 > <Channels>100</Channels> <Delay>0.123</Delay> <Medias> <Movie> <Media number= 5 /> <Media number= 8 /> </Movie> <Music> <Generae begin= 12 end= 104 /> </Music> </Medias> </ConenProvider> </ConenProviders > Figure 6-9: Example of conen provider configuraion

50 50 6 Implemenaion Media Cener All media ceners are colleced by he ag <MediaCeners> and separaed by he ag <MediaCener>. Every media cener has a coninuous idenificaion number, which is represened by he propery number. For he media cener we need he following configuraion: - Size of he local cache: Represened by he ag <Capaciy>, conains he number of media conens, which can be sored on he media cener, divided ino music conen and movie conen (properies: movie, music ). - Processing delay for a reques: Represened by he ag <Delay> similar o he delay from a conen provider. - Conneced conen provider and he nework bandwidh of he link: Colleced by he ag <Conneced>, conains he conen provider (ag <ConenProvider> wih propery number, which refers o he conen provider idenificaion number) and he bandwidh of he connecion (propery nework ) in Kbps. - Number of conneced cusomers: Represened by he ag <Cusomer>. - Preload media conen: Similar o media conen on a conen provider bu wihou generaing funcionaliy. - Top-movies and Top-music of he cusomers: Colleced by he ag <TopMedias>, which conains he probabiliy of choosing a op media in percen represened by he propery probabiliy. The media conens are represened as he sored medias on a conen provider. - Time periods of cusomer behavior: Colleced by he ag <Periods>. A single period is represened by he ag <Period>, which conains he beginning of he period (propery begin ), he end of he period (propery end ), he movie-selecion-rae in percen (propery movieprob ) and he mean ime beween a cusomer requess wo media iles (propery mime ). A simple example is shown in Figure 6-10.

51 6.1 Simulaion 51 <MediaCeners> <MediaCener number= 1 > <Capaciy movie= 100 music= 100 /> <Delay>0.123</Delay> <Conneced> <ConenProvider number="3" nework="100000" /> <ConenProvider number="5" nework="100000" /> </Conneced > <Cusomer>1000</Cusomer> <TopMedias> <Movie> <Media number= 4 /> <Media number= 23 /> </Movie> <Music> <Media number= 5 /> <Media number= 7 /> </Music> </TopMedias> <Periods> <Period begin= 8:00 end= 12:00 movieprob= 20 mime= 100 /> <Period begin= 12:00 end= 20:00 movieprob= 40 mime= 50 /> </Periods> </MediaCener> </MediaCeners > Figure 6-10: Example for media cener configuraion Sequence The simulaion sequence sars wih he disribuion of processes, jus as normal MPIapplicaions. The nex sep is ha every process reads he configuraion from he XML-file. Afer he configuraion, i is possible ha he configuraion differs from process o process because of he random generaion of media conen (see 6.1.2). Transmiing all media conens from process 0 o all oher media cener processes, hrough MPI-messages compensaes his. Then he process wih he MPI-idenificaion number 0 creaes all needed ables in he daabase for soring monior daa.

52 52 6 Implemenaion CLIC Clic1b11 0 Clic1a22 1 Clic2a13 2 Clic1c32 3 Clic2j21 4 Compuer Creae daabase ables Figure 6-11: Sequence (sep 1) Afer ha every process becomes is own ask: One process simulaes he conen provider, he oher processes will simulae media ceners wih conneced cusomers. Process 0 simulaes all conen providers. CLIC Clic1b11 0 Clic1a22 1 Clic2a13 2 Clic1c32 3 Clic2j21 4 Compuer Task: Conen providers Task: Media cener Figure 6-12: Sequence (sep 2) Now every process creaes is iniial even lis. The conen provider only insers he Simulaion-End -even and he firs Sensor Transmission -even ino is lis. Media ceners addiionally inser he nex cusomer requess of all cusomers and he firs Time synchronizaion -even o heir even-lis. 6.2 Browser Adaper The browser adaper should ge he daa sored in daabase, no only he daa of one service, bu he daa of he whole sysem. Wha he represenaion looks like is described in 3.10.

53 6.2 Browser Adaper Overview For geing he model measuremens we mus upgrade he browser adaper of he precursory work. We need o modify he XML-Inerface and he Browser-Adaper module shown in Figure I is also possible o only modify he Browser-Adaper module, bu we use his way because i is he more efficien soluion (slow XML-messaging). The Browser-Adaper module needs an exension o query a model measuremen series. Moreover i needs an exension o generae an XML-message for he XML-Inerface module o ge all needed daa. The XML-Inerface module needs an exension o process hese messages and should deliver he measuremens hrough an XML-file. HTTP- Server Browser- Adaper XML-Inerface Modificaion DB Figure 6-13: Modificaion of he Browser Adaper XML-Message The message for geing he model measuremen series is similar o ordinary query messages. A subsanial difference is ha he able-selec pah is no a value able, bu he roo-folder (see 3.9) of he model measuremens. The inerval of he query also conains an enry for he sensormeasuring period. <?xml version="1.0" encoding="utf-8"?> <body> <modelquery> <ableselec> <sub name="roo" /> <sub name="clicsim" /> </ableselec> <inerval> <begin daetime=" :00:00" /> <end daetime=" :30:00" /> <period second="120" /> </inerval> </modelquery> </body Figure 6-14: XML-message for model measuremens query

54

55 55 7 Examples In his chaper we invesigae wo example scenarios in order o demonsrae how service models and measuremens looks like and which resuls can be derived. Measuremens are generaed by he simulaion sysem. 7.1 A Cause for Media Ceners The quesion considered here is: Is here an advanage o use media ceners as a local cache for disribued media conen? To analyze his quesion we compare wo scenarios: One wih media ceners and he oher wihou media ceners wih cusomers direcly conneced o he conen providers. To simulae direc conneced cusomers (o conen provider) we assume he local sorage and he processing ime of a media cener o be zero. Scenario 1: 1 3 Conen provider Cusomer (100) (100) (100) Scenario 2: 1 3 Conen provider Media cener (100) (100) (100) Cusomer Figure 7-1: Two scenarios for simulaion Service Model Configuraion In his example we use a scenario of 3 conen providers and 10 media ceners wih 2000 available media iles (1000 music and 1000 movie), which are disribued across he 3 conen providers. Each conen provider sores 600 iles (300 music and 300 movie) exclusively and

56 56 7 Examples 200 medias (100 music and 100 movie) are available on all conen providers, hese may are news or somehing else. Movie conen uses a daa rae of 1,500 Mbps and music conen uses a daa rae of 128 Kbps. The duraion of a movie is beween 1,5 hours and 2 hours. The 100 movies sored on all conen providers have duraion imes beween 15 and 30 minues. The duraion of music conen is beween 3 and 5 minues. 1 2 Available on all conen providers (400) Available media conen (2000) 3 Figure 7-2: Media conen disribuion on conen providers Each media cener is conneced o every conen provider hrough a 100Mbi link and has 100 cusomers online. There are wo groups of cusomers: The firs group accesses conen in a prime ime beween 08:00 p.m. and 10:30 p.m., he second in a prime ime beween 05:00 p.m. and 07:30 p.m. Each group consiss of 5 media ceners. These wo groups simulaes cusomers in wo ime zones: Remaining ime Prime ime 05:00 05:00 p.m. 07:30 p.m. Remaining ime Prime ime 08:00 08:00 p.m. 10:30 p.m. 00:00 12:00 Day ime Figure 7-3: Used ime zones in he simulaion The op medias are equal for each cusomer: One movie from every conen provider and hree movies, which are available everywhere (sored on each conen provider). A conen provider has 100 ransmission channels and a processing ime of 0,5 seconds per reques. A media cener has a processing ime of 0,25 seconds per reques. These values are invened bu sufficien for his simulaion.

57 7.1 A Cause for Media Ceners 57 We can summarize he values for he service model: Μ = { 1,..,1000},{ 1,.., } } movie 1000 music : 1000 movies and 1000 music conens. (Movie and music iles wih number beween 901 and 1000 are sored on every conen provider) 1 1 = µ n 0,25sec c = 100 Cusomers conneced o a media cener n η n m 100Mbis, = Μ 1 = { 1,..,300,901,..,1000},{ 1,..,300,901,.., } } movie 1000 music : 300 movie iles (number from 1 o 300) exclusively sored and 200 movie iles (number from 901 o 1000) sored on all conen providers. Wih music iles i is he same. 2 = { 301,..,600,901,..,1000},{ 301,..,600,901,.., } } Μ : 300 movie iles movie 1000 (number from 301 o 600) exclusively sored and 200 movie iles (number from 901 o 1000) sored on all conen providers. Wih music iles i is he same. 3 = { 601,..,1000},{ 601,.., } } Μ : 300 movie iles (number from 601 o 300) movie 1000 music exclusively sored and 200 movie iles (number from 901 o 1000) sored on all conen providers. Wih music iles i is he same. music 1 2 = µ m 0,5sec χ = 100 Transmission channels of a conen provider m The wo scenarios differ in one parameer: The sorage capaciy of media ceners. m ; m 0 : Sorage capaciy of media cener n in scenario 1 n, movie = 0 n, music = m n, movie = 120 ; mn, music = 120: Sorage capaciy of media cener n in scenario 2 For simulaion we use ime beween 00:00 a.m. and 11:59 p.m. We simulae a complee day wih all ime periods of cusomer behavior. We use a measuring inerval of = 2min.

58 58 7 Examples Resuls Boh simulaions for eiher scenario have been run hree imes and were compared again each oher. Since simulaion resuls were nearly be he same we use he resuls of he firs simulaion run for analysis. Figure 7-4 and Figure 7-5 show he reques race on conen provider 1 2 ( λ (,60sec); n Ν,1 n n n,1 10 ). The dashed line shows he rae of refused requess: 100 requess / minue :00 09:00 13:00 17:00 21:00 Figure 7-4: Reques race on a conen provider wihou media cener 100 requess / minue :00 09:00 13:00 17:00 21:00 Figure 7-5: Reques race on a conen provider wih media cener

59 7.1 A Cause for Media Ceners 59 In Figure 7-4 wo overload siuaions occur beween 05:00 p.m. and 00:00 a.m. The reques rae increases up o 60 requess per minue. For he conen provider i is impossible o perform hese requess because he conen provider has only 100 ransmission channels and a channel is used for duraion of a leas 3 minues. The reason for his high reques rae is ha mos of he requess are refused. In his case cusomers generae new requess afer a shor ime and so here are so much reques. In Figure 7-5 we see a race wih using media ceners as local caches. Overload condiions have disappeared: The reques rae is fla. This demonsraes he advanage of using media ceners as local caches. In Figure 7-6 we see an example of a media cener reques race wih a small peak a 08:15 p.m. (prime ime). In he remaining ime he reques rae is consanly fla. Afer 09:00 p.m. he reques rae decreases since cusomers are waching movies making no furher requess. 1 Figure 7-6 shows he reques race on media cener 1 ( λ (,60sec) ): 0,1 100 requess / minue :00 09:00 13:00 17:00 21:00 Figure 7-6: Reques race on media cener 1

60 60 7 Examples Figure 7-7 and Figure 7-8 show he nework load beween media cener 1 and conen provider 1 ( η,1sec) ): 1,1( MB / sec :00 09:00 13:00 17:00 21:00 Figure 7-7: Nework-load wihou media ceners MB / sec :00 09:00 13:00 17:00 21:00 Figure 7-8: Nework-load wih media ceners Figure 7-7 and Figure 7-8 show, ha using media ceners reduces nework load.

61 7.2 Preloading Media Conen 61 from Wihou media ceners (KB) Wih media ceners (KB) Conen provider Conen provider Conen provider oal Figure 7-9: Toal nework ransmissions produced by media cener 1 In Figure 7-9 you can see ha media ceners reduces significanly nework load. In his case by nearly 48% compared o he scenario wihou media ceners Conclusion We have seen ha media cener reduces nework load. Media ceners also avoid overload siuaions on conen providers. The conclusion can be drawn, ha media ceners are provide an advanage in order o guaranee ha all cusomers can wach a movie or lisen o music simulaneously. 7.2 Preloading Media Conen In he nex scenario we examine he case of a new movie being released and launched a he same ime naion-wide. The quesion is: Is i reasonable o preload he new movie o media ceners before launch dae Service Model Configuraion In order o invesigae his scenario, we look a a simple example of one conen provider and one media cener. The configuraion parameers are alike he parameers of he firs example, bu now wih a single conen provider ha provides he conen: Μ = { 1,..,1000},{ 1,.., } } movie 1000 music : 1000 movies and 1000 music iles. 1 µ 1 1 = 0,25sec c 1 = 100 Cusomers conneced o media cener η 1,1 = 100Mbis { 1,..,1000},{ 1,.., } } Μ : All media iles are sored on he conen provider 1 1 = movie 1000 music

62 62 7 Examples 1 µ 2 1 = 0,5sec χ 1 = 100 Transmission channels of conen provider 1 m 1, movie = 120 ; m1, music = 120 : Sorage capaciy of media cener 1 For simulaion we also use ime beween 00:00 a.m. and 11:59 p.m. We use a measuring period of = 2min. All new movies are pre-loaded o he media cener Resuls Figure 7-10 and Figure 7-11 show ha a his scenario preloading media conen has no influence o he reques race a he conen provider. Boh races are similar and consanly fla and so i is guaraneed ha all cusomers can wach he released movie. Figure 7-12 and Figure 7-13 show ha preloading media conen has no influence o nework raffic ( η ( 1,1,1sec) ) oo. Through he day almos he same nework raffic is produced. 20 requess / minue :00 12:00 16:00 20:00 Figure 7-10: Reques race wihou preloading media conen

63 7.2 Preloading Media Conen requess / minue :00 12:00 16:00 20:00 Figure 7-11: Reques race wih preloading media conen MB / sec 9 7,5 6 4,5 3 1,5 0 8:00 12:00 16:00 20:00 Figure 7-12: Nework load wihou preloading media conen

64 64 7 Examples MB / sec 9 7,5 6 4,5 3 1,5 0 08:00 12:00 16:00 20:00 Figure 7-13: Nework load wih preloading media conen Conclusion We have seen ha preloading media conen has no influence a his scenario. The sraegy of caching media conen on he media cener may explain hese resuls: Afer cusomers reques a media ile i will be sored on he local cache. If any cusomers wan o wach he same movie, he conneced media cener reques he media ile from a conen provider one imes only (a he firs reques by a cusomer). The following reques can be served from he local cache and here is no need o reques a conen provider again.

65 65 8 Conclusion In his work we discussed a model of disribued service environmens. This model describes he behavior of such environmens, relaions beween is componens and is relaed o queuing nework models, which describes sysems of communicaing componens. We showed which measuremens are needed o ge he model parameers, how o ge hese measuremens and how o sore hem in he daabase developed in he precursory work. We also described a mehod o sore model measuremens in an XML-documen. The model is used o describe a disribued media conen environmen, and a simulaion sysem ha runs on he CLIC helps o generae measuremens as in real sysems. Two examples illusrae how he model can be used o analyze specific scenarios. The simulaion sysem is even-based and disribued. According o relevan lieraure, his simulaion mehod uses approximaely 30% of processor ime for managing he even lis. Bu he big advanage is o simulae imporan poins in ime only. 8.1 Possibiliies for he Fuure The simulaion sysem is implemened o sore only one daa value in daabase by generaing an XML-message. Experimens show ha daabase and XML-messaging are bolenecks of he simulaion environmen because every daabase access needs approximaely one second. Thus simulaing a whole day of cusomer behavior may ake more han one hour of simulaion ime (depending on he configuraion). Wihou soring values ino he daabase i akes less han one minue. This effec may be reduced by caching measuring values in he sensor because soring more han one value in he daabase by using one XML-message only is faser han soring one by one. The sysem model does no conain informaion abou he hardware layer. You canno describe he physical connecion beween (logical only) and he physical load on he componens. We would need a possibiliy o combine applicaion-level measuremens wih hardware-level measuremens. For example: Three communicaing service processes use he same physical nework connecion. Thus he connecion capaciy is shared. The sensors assume ha service processes are communicaing synchronously. Tha means a service process wais for he compleion of remoe services, which he service process needs. This may deliver measuring failures by measuring asynchronous processes.

66 66 8 Conclusion The connecion beween media ceners is no been considered. Thus a nex sep o upgrade he simulaion sysem is o implemen his funcionaliy. Anoher fuure goal is he graphical represenaion of he sysem model. This may be a graph wih parameers (e.g. nework load or reques rae). The cause is: Graphical represenaion beer shows he sae of he sysem han an XML-file. 8.2 Relaed Work There are companies ha pracice conen delivering already. One of hem is Akamai [ACO]. This company s plaform ( EdgeSuie ) for conen, sreaming media and applicaion deliveries comprises more han servers in more han 1000 neworks in 63 counries. These servers deliver HTML documens, saic images and sreaming media for over 1300 conen providers, including many of he mos popular sies on he web. The used service hierarchy is similar o he hierarchy of disribued media conen deliveries in his work: There are some servers, which deliver he conen and here are some servers (so called Edge ) ha cache his conen for cusomer-delivery: Server of a conen provider Edges Inerne Figure 8-1: Scenario of Akamai s EdgeSuie Tha way conen can be delivered by accessing he Edges only. This happens wihou using he whole Inerne wih is bolenecks such as firs mile bandwidh consrains and rouer/swich limiaions.

67 67 Acronyms CD CLIC HTML HTTP IPC LAN LRU MPEG MPI SVG VCR VoD XML Compac Disc (German) Chemnizer Linux Cluser Hyperex Markup Language Hyperex Transpor Proocol Iner Process Communicaion Local Area Nework Las Recenly Used Moion Picure Expers Group Message Passing Inerface Scalable Vecor Graphics Video Cassee Recorder Video on Demand Exensible Markup Language

68

69 69 Figures Figure 2-1: Simple srucure in daabase...11 Figure 2-2: Parameer able examples...12 Figure 2-3: Message for daa query...12 Figure 2-4: Main componens of he daabase...13 Figure 3-1: Services on a server...15 Figure 3-2: The general saes of a service process...16 Figure 3-3: Saes of a web-server service process...16 Figure 3-4: Period of ime for he parameers...16 Figure 3-5: A simple example of characerisic parameers...17 Figure 3-6: Saes of disribued service processes...17 Figure 3-7: Scenario for disribued services...18 Figure 3-8: evens logged by he sensor...21 Figure 3-9: Informaion sequence of he sensors...22 Figure 3-10: Possible daabase srucure...24 Figure 3-11: Possible represenaion of he hree-server sysem...24 Figure 3-12: XML represenaion Figure 3-13: XML represenaion Figure 3-14: Curren sae XML represenaion...26 Figure 3-15: Curren sae...26 Figure 3-16: Trace represenaion...27 Figure 4-1: Disribued media conen scenario...29 Figure 4-2: Sysem srucure...31 Figure 4-3: Simplified cusomer behavior...32 Figure 4-4: Noaions by he sensor...33 Figure 4-5: Table srucure in daabase...35 Figure 4-6: Curren sae...36 Figure 4-7: Trace represenaion...36 Figure 6-1: Processes of simulaion...41 Figure 6-2: Example (sep 1)...45 Figure 6-3: Example (sep 2)...46 Figure 6-4: Example (sep 3)...46 Figure 6-5: Example (sep 4)...46 Figure 6-6: Example (sep 5)...46

70 70 Figures Figure 6-7: Example (final seps) Figure 6-8: Example of global configuraion Figure 6-9: Example of conen provider configuraion Figure 6-10: Example for media cener configuraion Figure 6-11: Sequence (sep 1) Figure 6-12: Sequence (sep 2) Figure 6-13: Modificaion of he Browser Adaper Figure 6-14: XML-message for model measuremens query Figure 7-1: Two scenarios for simulaion Figure 7-2: Media conen disribuion on conen providers Figure 7-3: Used ime zones in he simulaion Figure 7-4: Reques race on a conen provider wihou media cener Figure 7-5: Reques race on a conen provider wih media cener Figure 7-6: Reques race on media cener Figure 7-7: Nework-load wihou media ceners Figure 7-8: Nework-load wih media ceners Figure 7-9: Toal nework ransmissions produced by media cener Figure 7-10: Reques race wihou preloading media conen Figure 7-11: Reques race wih preloading media conen Figure 7-12: Nework load wihou preloading media conen Figure 7-13: Nework load wih preloading media conen Figure 8-1: Scenario of Akamai s EdgeSuie... 66

71 71 Bibliography [ACO] Akamai Technologies Corporae hp:// [ALVS] Tran-Gia, P. Analyische Leisungsbewerung vereiler Syseme Springer., 1996 [AWS] The Apache HTTPD Projec hp://hpd.apache.org/ [ICPE] Kan, K. Inroducion o Compuer Sysem Performance Evaluaion McGraw-Hill, Inc., 1992 [IMA] Wiig, H. Inelligen Media Agens Vieweg, 1999 [LIBX] The XML C library for Gnome hp://xmlsof.org/ [LTRA] Almeroh, K. C. Long Term Resource Allocaion in Video Delivery Sysems IBM Research Repor (RC 20249), 1996 [MBS] Tanenbaum, A. S. Moderne Beriebssyseme Hanser, 1995 [MDS] Mansouri-Masani, M. Monioring of Disribued Sysems Universiy of London, 1995 [MLR] Haas, M., Zorn, W. Mehodische Leisungsanalyse von Rechensysemen Oldenbourg, 1995 [MPI] [MPG] The Message Passing Inerface hp://www-unix.mcs.anl.gov/ MPEG.ORG hp:// [MSIM] Mehl, H. Mehoden vereiler Simulaion Vieweg, 1994 [MVS] Kurschl, W. Monioring von vereilen Sysemen Johannes Kepler Universiä, Linz, 1995

72 72 Bibliography [OVD] Rauenberg, M. Organisaion eines Servers für Video on Demand VDI Verlag, 1998 [PCCS] Haverkor, B.R. Performance of Compuer Communicaion Sysems John Wiley & Sons, Ld, 1999 [TFIC] Klava, H., Furh, B. Techniques for improving he capaciy of video-on-demand sysems Columbia Universiy [TSOV] Tagesspiegel Online Diense Verlag GmbH hp:// [XMDB] Koenig, R., Reimann, C. XML-based Monioring Daabase Chemniz Universiy of Technology, 2001 [XML] Exensible Markup Language hp://

73 Saemen of Auonomy Wih my signaure I affirm ha his work was wrien by myself wihou any help of srangers. I have no used any sources or aids, which are no denoed. This work was no used wih anoher esing office before. Chemniz, 05/31/2002 Carsen Reimann