Auto-tuning and Self-optimization of 3G and Beyond 3G Mobile Networks

Transcription

1 Auo-unng and Self-opmzaon of 3G and Beyond 3G Moble Nework Rdha Nar To ce h veron: Rdha Nar. Auo-unng and Self-opmzaon of 3G and Beyond 3G Moble Nework. Neworkng and Inerne Archecure. Unveré Perre e Mare Cure - Par VI, Englh. <el > HAL Id: el hp://el.archve-ouvere.fr/el Submed on 22 Jun 2010 HAL a mul-dcplnary open acce archve for he depo and demnaon of cenfc reearch documen, wheher hey are publhed or no. The documen may come from eachng and reearch nuon n France or abroad, or from publc or prvae reearch cener. L archve ouvere plurdcplnare HAL, e denée au dépô e à la dffuon de documen cenfque de nveau recherche, publé ou non, émanan de éablemen d enegnemen e de recherche frança ou éranger, de laboraore publc ou prvé.

2 THESE DE DOCTORAT DE L UNIVERSITE PIERRE ET MARIE CURIE Spécalé Informaque, Télécommuncaon e Élecronque Préenée par M. Rdha Nar Pour obenr le grade de DOCTEUR de l UNIVERSITÉ PIERRE ET MARIE CURIE Suje de la hèe : Paramérage Dynamque e Opmaon Auomaque de Réeaux Moble 3G e 3G+ Souenue le 23 Janver 2009 devan le jury compoé de : Prof. Guy Pujolle Dr. Zw Alman Dr. Mong Marzoug Prof. Nazm Agoulmne Prof. Tjan Chahed Dr. Salaheddne Maza Dreceur de hèe Encadreur Rapporeur Rapporeur Examnaeur membre Invé

3

4 À mon père Hacen, À ma mère Ramdhana, À ou me frère e me œur À ma fancée Mounra À ou me proche e me am

5 v

6 Remercemen Tou d abord, je en à remercer Mr. Zw Alman, ngéneur de recherche e chef de proje à France Télécom R&D, pour on encadremen. Sa grande expérence, a von e a connaance approfonde m on achemné oujour ur la bonne drecon. J a apprécé le dcuon echnque que nou avon eue enemble. Il m a perm d élargr le pecre de me connaance e de me conrbuon. Me plu vf remercemen von égalemen à Mr. Guy Pujolle, profeeur à l unveré UPMC pour a drecon de ma hèe e pour e répone préceue à me demande. J exprme ma graude à Mr. Mong Marzoug, dreceur du "pole modélaon de réeaux" à Orange, e Mr. Nazm Agoulmne, profeeur à l unveré d Evry, qu conenren à en êre le rapporeur de hèe. D égale façon, je remerce Mr. Tjan Chahed, profeeur à TELECOM SudPar, pour e nombreux conel préceux e pour a parcpaon en an qu examnaeur de ma hèe. Ma préceue graude va égalemen à Mr. Salaheddne Maza, Ingéneur exper en qualé réeau à SFR, d avor accepé mon nvaon pour parcper à ma ouenance. La hèe a éé effecuée au en de l uné de recherche REM du laboraore NET du FTR&D. Me graude von a oue le peronne de l uné qu par leur dée e ouen m on adé à ben fnr ce raval, en parculer Hervé Dubrel, Abed Samha, Aruro Orega Molna, Zakara Nour e Salah-Eddne Elayoub. Une grande pare de la hèe éa fae dan le cadre d un proje européen Eureka-Celc Gandalf. Je remerce vvemen ou le parenare du proje pour leur dée e expere enrchane. Enfn, ma reconnaance e mon affecon von à me cher paren qu m on ouenu ou au long de me éude. v

7 v

8 Réumé La élécommuncaon rado moble conna acuellemen une évoluon mporane en erme de dveré de echnologe e de ervce fourn à l ulaeur fnal. Il apparaî que cee dveré complexfe le réeaux cellulare e le opéraon d'opmaon manuelle du paramérage devennen de plu en plu complquée e coûeue. Par conéquen, le coû d exploaon du réeau augmenen corrélavemen pour le opéraeur. Il e donc eenel de mplfer e d'auomaer ce âche, ce qu permera de rédure le moyen conacré à l'opmaon manuelle de réeaux. De plu, en opman an de manère auomaque le réeaux moble déployé, l era poble de rearder le opéraon de denfcaon du réeau e l'acquon de nouveaux e. Le paramérage auomaque e opmal permera donc au d'éaler vore même de rédure le nveemen e le coû de manenance du réeau. Cee hèe nrodu de nouvelle méhode de paramérage auomaque (auo-unng) de algorhme RRM (Rado Reource Managemen) dan le réeaux moble 3G e au delà du 3G. L auo-unng e un proceu ulan de oul de conrôle comme le conrôleur de logque floue e d apprenage par renforcemen. Il ajue le paramère de algorhme RRM afn d adaper le réeau aux flucuaon du rafc. Le fonconnemen de l auo-unng e baé ur une boucle de régulaon opmale ploée par un conrôleur qu e almené par le ndcaeur de qualé du réeau. Afn de rouver le paramérage opmal du réeau, le conrôleur maxme une foncon d ulé, appelée au foncon de renforcemen. Quare ca d éude on décr dan cee hèe. Dan un premer emp, l auo-unng de l algorhme d allocaon de reource rado e préené. Afn de prvléger le ulaeur du ervce emp réel (vox), une bande de garde e réervée pour eux. Cependan dan le ca où le rafc emp réel e fable, l e mporan d exploer cee reource pour d aure ervce. L auo-unng perme donc de fare un comprom opmal de la qualé perçue dan chaque ervce en adapan le reource réervée en foncon du rafc de chaque clae du ervce. Le econd ca e l opmaon auomaque e dynamque de paramère de l algorhme du of handover en UMTS. Pour l auo-unng du of handover, un conrôleur e mplémené logquemen au nveau du RNC e règle auomaquemen le eul de handover en foncon de la charge rado de chaque cellule an que de e vone. Cee approche perme d équlbrer la charge rado enre le cellule e an augmener mplcemen la capacé du réeau. Le mulaon monren que l adapaon de eul du of handover en UMTS augmene la capacé de 30% par rappor au paramérage fxe. L approche de l auo-unng de la moblé en UMTS e éendue pour le yème LTE (3GPP Long Term Evoluon) ma dan ce ca l auo-unng e fondé ur une foncon d auo-unng préconrue. L adapaon de marge de handover en LTE perme de ler le nerférence nercellulare e an augmener le déb perçu pour chaque ulaeur du réeau. Fnalemen, un algorhme de moblé adapave enre le deux echnologe UMTS e WLAN e propoé. L algorhme e orcheré par deux eul, le premer e reponable du handover de l UMTS ver le WLAN e l aure du handover dan le en nvere. L adapaon de ce deux eul perme une exploaon opmale e conjone de reource dponble dan le deux

9 echnologe. Le réula de mulaon d un réeau mul-yème expoen égalemen un gan mporan en capacé.

10 Abrac Wh he wrele moble communcaon boom, auo-unng and elf-opmzaon of nework parameer are more han ever key ue o provde hgh-qualy ervce for he end-uer and o decreae he operaonal expendure of he nework operaon. The pecal aenon drawn o he elf-opmzaon of rado reource managemen (RRM) parameer movaed by he uer need for ubquou communcaon and by he ncreang complexy of nework reulng from he cooperaon of rado acce echnologe. Formerly, RRM ha been baed on ome algorhm (admon conrol, reource allocaon, handover ) governed by a e of fxed hrehold. Today, RRM procedure have undergone a conderable change and he paradgm wll hf oward a compleely auomac nework managemen. Opmal auo-unng mechanm, performed by conrol mehod uch a fuzzy logc opmzed by renforcemen learnng, could conderably mprove nework managemen funcon wh repec o radonal RRM algorhm wh fxed parameer. Th he nroduce new reul n auo-unng and elf-opmzaon of RRM parameer n 3G and beyond 3G nework. Auo-unng ak are organzed n he conrol plane where dfferen nformaon exchange nvolved beween he nework node. Auo-unng ung fuzzy logc conrol performed n a local loop, namely he conroller n connuou neracon wh he nework. The conroller feed he nework wh new parameer eng and converely he nework reurn feedback by delverng new qualy ndcaor ndcang operang ae. Dfferen ue-cae are nvegaed. Fr, an auo-unng of reource allocaon algorhm n UMTS uded a an alernave o he exng ac reource allocaon. The auo-unng proce dynamcally adap a guard band ha reerved for uer ung real me ervce. A be rade-off beween real me and non real me ervce acheved n he ene ha he qualy of ervce become comparable n he wo raffc clae epecally n a hgh load uaon. The econd ue-cae concern he elf-opmzaon of of handover parameer n UMTS nework. For each cell he conroller receve a npu he flered downlnk load and ha of neghbourng cell. The conroller connually learn he be parameer value n each nework uaon. The learnng proce governed by a uly funcon. Smulaon reul reveal gnfcan mprovemen n erm of nework performance. The propoed auo-unng algorhm balance he rado load beween bae aon and mprove he yem capacy by up o 30% compared o a UMTS nework wh fxed of handover parameer. However, he auounng ncreae he gnallng meage load n he rado nerface a well a n he core nework. Th negave effec mnmzed by reducng he reacvy of he auo-unng conroller. The hrd cae deal wh he auo-unng of LTE (3GPP Long Term Evoluon) mobly algorhm. The auo-unng carred ou by adapng handover margn nvolvng each couple of cell accordng o he dfference beween her load. The auo-unng allevae cell congeon and balance he raffc and he load beween cell by handng off moble cloe o he cell border from he congeed cell o neghbourng cell. Smulaon reul, baed on mplfed yem and nerference model, how ha he auo-unng proce brng abou an mporan gan n boh call admon rae and uer hroughpu.

11 Fnally, an algorhm of neryem mobly beween UMTS and WLAN acce echnologe propoed. The algorhm coverage and load baed, and governed by wo hrehold: he fr reponble for handover from UMTS o WLAN and he econd - for he nvere drecon. The elf-opmzaon of he wo hrehold jonly performed. The reul obaned ung he adapve neryem mobly algorhm how hgh rackng capacy gan, and llurae he mporance of nellgen cooperaon beween echnologe. The reul provded n h he are uppored by heorecal analy and exenve dynamc yem level mulaon wh mul-cell cenaro, ncludng he effec of many relevan mechanm ha have an mpac on he rado acce. However, mulaon do no exacly reflec he realy n he nework operaon. So auo-unng hould be eed n a real expermenal nework (e-bed). v

12 Table of conen Réumé... Abrac... Table of conen...v L of fgure and able...v Acronym...x Chap. 1 Inroducon Background and problem defnon Scope and objecve of he he Orgnal conrbuon The rucure...4 Chap. 2 Auo-unng n moble communcaon: Relaed work Inroducon Auo-unng of GSM man funconale Overvew of GSM Nework and her evoluon "GPRS and EDGE" RRM algorhm and arge auo-uned parameer Leraure revew of GSM auo-unng UMTS nework Overvew of UMTS Nework and evoluon UMTS RRM algorhm and relaed parameer Relaed work on he auo-unng of UMTS parameer Mul-yem nework GERAN UTRAN ner-workng GPP-WLAN ner-workng Leraure revew of auo-unng n mul-yem nework Concluon...27 Chap. 3 Auo-unng archecure and ool Inroducon Auo-unng archecure Gandalf managemen archecure Gandalf auo-unng archecure Auo-unng nformaon flow Auo-unng and opmzaon engne Fuzzy logc conroller Mahemacal framework of FLC Example: ue-cae of FLC Renforcemen learnng General vew of machne learnng Mahemacal framework of RL Fuzzy Q-learnng conroller Q-learnng algorhm Adapaon of Q-learnng o fuzzy nference yem Concluon...46 Chap. 4 Applcaon of auo-unng o he UMTS nework...47 v

13 4.1 Inroducon Correlaon beween qualy ndcaor Preenaon of ued qualy ndcaor Correlaon beween qualy ndcaor Auo-unng of reource allocaon n UMTS Admon conrol raegy Qualy ndcaor, acon and renforcemen funcon Performance evaluaon Auo-unng of UMTS of handover parameer SHO algorhm FQLC-baed auo-unng of SHO parameer Performance evaluaon Sgnallng overload due o auo-unng Concluon...66 Chap. 5 Self opmzaon of mobly algorhm n LTE nework Inroducon Overvew of LTE yem Syem requremen Syem archecure Phycal layer Self opmzng nework funconale Inerference n e-utran yem Syem model and aumpon Inerference model Auo-unng of e-utran handover algorhm E-UTRAN handover algorhm Handover adapaon and load balancng Auo-unng of handover margn Smulaon and reul Concluon...82 Chap. 6 UMTS-WLAN load balancng by auo-unng ner-yem mobly Inroducon Aumpon UMTS-WLAN ner-workng mode Technology elecon and admon conrol UMTS-WLAN vercal handover and auo-unng Vercal handover decrpon Auo-unng of vercal handover parameer Smulaon and performance evaluaon Concluon...91 Chap. 7 Concluon and perpecve Concluon Lmaon and perpecve...93 Reference...94 Appendx A: Convergence proof of he renforcemen learnng algorhm Appendx B: LTE nerference model Appendx C: Nework yem level mulaor v

14 L of fgure and able Fgure 1.1. Fully heerogeneou acce nework...1 Fgure 1.2. Targe of he he....3 Fgure 2.1. GSM yem archecure....7 Fgure 2.2. Dynamc reource allocaon beween CS and PS ervce....9 Fgure 2.3. UMTS archecure...14 Fgure 2.4. Inra-Frequency Sof Handover...18 Fgure 2.5. Cenralzed CRRM eny...23 Fgure 2.6. Decenralzed CRRM no every RNC/BSC Fgure 3.1. Nework managemen ak wh he correpondng me cale...29 Fgure 3.2. Auo-unng archecure n uer, conrol and managemen plane...30 Fgure 3.3. Sgnallng meage beween AOE and JRRM/RRM module Fgure 3.4. Auo-unng and Opmzaon Engne...33 Fgure 3.5. The concep of fuzzy logc conroller Fgure 3.6. Fuzzy logc conroller baed on Takag-Sugeno approach Fgure 3.7. Fuzzy e and menberhp funcon of he droppng and blockng rae Fgure 3.8. Example of Markovan Decon Proce Fgure 3.9. Fuzzy Q-learnng algorhm Fgure 4.1. Capacy model for a UMTS bae aon Fgure 4.2. Call ucce rae of each ervce a a funcon of he RT guard band for wo raffc uaon: (1) RT3 and NRT5; and (2) RT5 and NRT Fgure 4.3. Fuzzy Q-learnng conroller for auo-unng he RT guard band n each BS Fgure 4.4. Call ucce rae a a funcon of RT call arrval rae (RT_D mean RT CSR n he dynamc veron, RT_F25% mean RT CSR for he fxed guard band of 25%)...55 Fgure 4.5. Hogram of RT CSR for all BS wh raffc arrval rae of RT2 and NRT6 moble/...56 Fgure 4.6. Hogram of NRT CSR for all BS wh raffc arrval rae RT2 and NRT6 moble/...56 Fgure 4.7. UMTS SHO algorhm (even 1A, 1B and 1C)...58 Fgure 4.8. Evoluon of he convergence crera of he fuzzy-q-learnng conroller Fgure 4.9. Evoluon of he qualy of rule-acon par Fgure Call ucce rae veru ncommng arffc for he opmzed nework wh auonomc managemen compared o a clacal nework...62 Fgure Cumulave drbuon funcon of he cell call ucce rae n he opmzed nework compared o he clacal nework...63 Fgure Drbuon of cell load for he opmzed nework compared o a clac nework wh fxed confguraon Fgure Percenage of moble n SHO uaon a a funcon of arrval rae for he nework whou and wh auo-unng...64 Fgure Cumulave drbuon funcon of he acve e updae frequency...65 Fgure 5.1. LTE archecure...69 Fgure 5.2. E-UTRAN (enb) and EPC (MME and S-GW) Fgure 5.3. Iner-cell nerference coordnaon cheme...72 Fgure 5.4. Typcal paern of geographcal drbuon of HO procedure Fgure 5.5. Example of geographcal drbuon of HO procedure wh raffc balancng...77 Fgure 5.6. The nework layou ncludng coverage of each enb Fgure 5.7. Admon probably a a funcon of he raffc neny for auo-uned handover compared wh fxed handover margn nework (6dB)...79 v

15 Fgure 5.8. Connecon holdng probably a a funcon of he raffc neny for auo-uned handover compared wh fxed handover margn nework (6dB)...80 Fgure 5.9. Average hroughpu per uer veru he raffc neny for auo-uned handover compared wh fxed handover margn nework (6dB)...81 Fgure Cumulave drbuon funcon of he SINR for nework wh and whou auounng, for raffc neny equal o 8 moble/ Fgure 6.1. Very ghly coupled UMTS/WLAN nework Fgure 6.2. Selecon procedure n very ghly coupled UMTS/WLAN nework...85 Fgure 6.3. Load baed VHO algorhm beween UMTS and WLAN nework...86 Fgure 6.4. Heerogeneou nework layou wh 59 UMTS cell (quare wh arrow) and 22 WLAN AP (crcle) Fgure 6.5. Call ucce rae of RT raffc a a funcon of NRT raffc arrval rae for he nework wh (quare) and whou (damond) auo-unng...89 Fgure 6.6. Call ucce rae for NRT raffc a a funcon of NRT raffc arrval rae for he nework wh (quare) and whou (damond) auo-unng...90 Fgure 6.7. Average hroughpu a a funcon of NRT raffc neny...90 Fgure 6.8. Impac of auo-unng on he execuon rae of UMTS o WLAN vercal handover Fgure 6.9. Impac of auo-unng on he ucce rae of UMTS o WLAN vercal handover. 91 Fgure C.1. Man bloc of he mul-yem mulaor archecure Fgure C.2. Tme evoluon of he mul-yem mulaor Table 3.1. Fuzzy rule...36 Table 4.1. Correlaon beween qualy ndcaor...50 v

16 Acronym 2G 3G 3GPP AAA AOE AP ARRM ASU B3G BS BSS BSC BCCH CAC CAPEX CBR CDMA CDR CN CPICH CRMS CRRM CS CSR CSSR DCCH DCH DTCH DPO ECSD EDGE EGPRS EIR enb EPC e-utran ETSI FDD FQLC FLC HARQ HCS Second Generaon Thrd Generaon Thrd-Generaon Parnerhp Projec Auhencaon, Auhorzaon and Accounng Auo-unng and Opmzaon Engne Acce Pon Advanced Rado Reource Managemen Acve Se Updae Beyond 3G Bae Saon Bae Saon Subyem Bae Saon Conroller Broadca Conrol Channel Call Admon Conrol CApal Expendure Call Blockng Rae Code Dvon Mulple Acce Call Droppng Rao Core Nework Common Plo Channel Common Reource Managemen Server Common Rado Reource Managemen Crcu Swched Call Succe Rae Call Seup Succe Rao Dedcaed Conrol CHannel Dedcaed Channel Dedcaed Traffc CHannel Dynamc Programmng Operaor Enhanced Crcu-Swched Daa Enhanced Daa rae for GSM Evoluon Enhanced General Packe Rado Servce Equpmen Ideny Reger Evolved Node B Evolved Packe Core evolved UMTS Terreral Rado Acce Nework European Telecommuncaon Sandard Inue Frequency Dvon Duplex Fuzzy Q-Learnng Conroller Fuzzy Logc Conroller Hybrd Auomac Repea Reque Herarchcal Cell Srucure x

17 HLR HO HSCSD HSDPA HSPA HSUPA ITU-T IEEE IETF JRRM GSM GPRS GERAN GGSN KPI LTE MAC MBMS MBSFN MDP MPM MS MSC NMS Node B NRT NSS OAM OFDM OMC OMS OPEX PLMN PS PSK QoS RAN RAT RL RLC RNC RNS RRM RSCP RSSI RT Home Locaon Reger HandOver Hgh-Speed Crcu-Swched Daa Hgh Speed Downlnk Packe Acce Hgh Speed Packe Acce Hgh Speed Uplnk Packed Acce Inernaonal Telecommuncaon Unon Telecommuncaon econ Inue of Elecrcal and Elecronc Engneer Inerne Engneerng Tak Force Jon Common Rado Reource Managemen Global Syem for Moble Communcaon General Packe Rado Servce GSM/EDGE Rado Acce Nework Gaeway GPRS Suppor Node Key Performance Indcaor 3GPP Long Term Evoluon Medum Acce Conrol Mulmeda Broadca Mulca Servce Mulca-Broadca Sngle-Frequency Nework Markovan Decon Proce Managemen and Proceng Module Moble Saon Moble Swchng Cener Nework Managemen Syem UMTS bae aon Non Real Tme ervce Nework SubSyem Operaon Admnraon and Manenance Orhogonal Frequency Dvon Mulplexng Operaon and Manenance Cener Operaon and Manenance Subyem OPeraonal EXpendure Publc Land Moble Nework Packe Swched Phae-Shf Keyng modulaon Qualy of Servce Rado Acce Nework Rado Acce Technology Renforcemen Learnng Rado Lnk Conrol Rado Nework Conroller Rado Nework Subyem Rado Reource Managemen Receved Sgnal Code Power Receved Sgnal Srengh Indcaor Real Tme ervce x

18 SAE SDS SGSN SHO SIM SINR SMDP SNR SON SSID SSSR TDD TS UE UMTS UTRAN VHO VLR WAG WCDMA WLAN Syem Archecure Evoluon Sem Dynamc Smulaor Servng GPRS Suppor Node Sof HandOver Subcrber Ideny Module Sgnal-o-Inerference plu Noe Rao Sem-Markov Decon Proce Sgnal-o-Noe Rao Self Opmzng Nework Servce Se ID Seon Seup Succe Rao Tme Dvon Duplex Tme Slo Uer Equpmen Unveral Moble Telecommuncaon Syem UMTS Terreral Rado Acce Newor Vercal HandOver Vor Locaon Reger WLAN Acce Gaeway Wde-band Code Dvon Mulple Acce Wrele Local Area Nework x

19 Inroducon Chap. 1 Inroducon 1.1 Background and problem defnon Durng he la few year, Wrele mulmeda nework have known an exploon and an evoluon n erm of ophcaed echnologe and offered ervce. Th due o he exponenal raffc ncreae relaed o he mave demand of dvere ervce. For he remedy of hee nenve demand ue, moble elecommuncaon acor (operaor, conrucor, reearcher) have exended her acual nework or/and mgraed o oher echnologe. Th ha reuled n evoluon oward a heerogeneou wrele acce nework compromng a e of dvere rado echnologe, bu offerng a ngle e of negraed ervce o he end uer. The exence of new and vared rado echnologe evenually leadng o greaer choce and beer avalably of rado capacy, and ulmaely ervce o he end uer. Nework core and ervce are evolvng n parallel o he rado acce mechanm, deally reulng n an negraed ervce envronmen offerng a range of moble, ecure, qualy aured ervce n a managed fahon over a dvere e of rado acce echnologe. Fgure 1.1 depc an exenve, f no nmdang compleely heerogeneou envronmen negrang exng and envaged nework ype and echnologe. Saelle Broadband S-UMTS DVB-S Saelle Hgh Alude Plaform Broadcang DVB-T DAB GSM GPRS Cellular EDGE Fourh Generaon UMTS++ UMTS IP MBS 60 MBS 40 Broadband FWA Qua - Cellular Broadband W-LAN MWS xmds Fgure 1.1. Fully heerogeneou acce nework. Body-LAN W-LAN Peronal Area Nework Blueooh IR Local Area Nework Fxed Wrele Acce Wrele Local Loop Th heerogeneou mx poe gnfcan challenge, nally a he level of he phycal neroperably/compably beween yem and ubequenly when aempng o run conen ofware ervce over wha are fundamenally dfferen elecommuncaon echnologe. A ypcal uer acceng a ervce on a hand-held devce may fnd her phycal connecon o ha ervce wchng from a WLAN lnk n he offce, o a GPRS connecon n a 1

20 Inroducon car and o a 3G connecon n a congeed cy cenre. Throughou all hee connecon, he nework nfrarucure mu keep he uer conneced, wchng acce echnologe and mananng a conen uer experence and ervce. In addon, he fa evoluon of moble nework lead o a hghly complex and heerogeneou rado acce nework landcape. In h conex, nework managemen become crucal o guaranee hgh qualy and opmum cooperaon beween nework ubyem. Nework managemen and opmzaon ak are oday prmarly manual procee. Saff carre ou a ere of check and dagno of qualy ndcaor o eablh he caue of he problem; hen analye poble oluon and fnally launche he be healng acon. In h proce, everal applcaon and daabae mu be quered n order o analye performance daa and updae he confguraon of he nework. Thu, operaor have red o cope wh he ncreae of nework complexy by ncreang her aff and over-dmenonng reource. However, he growng ze of cellular nework, ogeher wh he ncreang complexy of nework elemen, make h raegy no longer praccal. Lkewe, due o effor and expene, defaul value for parameer of RRM algorhm are e all over he nework, even f non-opmum performance acheved. Therefore, he flexbly from he large parameer e defned on a cell (or even adjacency) ba no fully ezed. Thee naccurae nework eng lm he nework capacy, leadng o premaure ncreae of capal expendure (CAPEX) and avodable reducon n operaor revenue. Conequenly, operaor currenly demand auomac ool ha mplfy plannng, rollou and operaon of her nework. In addon, hee auomaed procee wll mechanze her curren repeve procedure, and alo provde new opmzaon procedure ha ncreae nework performance a a mnmum co. In h conex, auo-unng and elf opmzng nework ak are nowaday more han ever key ue o replay effecvely o he need of moble nework operaor and o provde hgh-qualy ervce for end-uer. The mgraon from manual o auomac cell plannng ha already nduced a gnfcan qualy enhancemen and deploymen co reducon. Smlar gan are expeced from he auomac and dynamc opmzaon of all managemen ak for he operaon of a moble rado nework. Th expecaon epecally jufed n complex yem lke UMTS and long erm evoluon or even heerogeneou acce nework. 1.2 Scope and objecve of he he The man objecve of h he he developmen of dynamc and auomac opmzaon algorhm o elec he mo approprae value for RRM nework parameer. The auomac and dynamc opmzaon refer n h work o auo-unng or elf-opmzaon echnque. Th he propoe new rado reource managemen algorhm ogeher wh mehod of auounng o opme he overall yem performance n a 3G or n he heerogeneou nework. The fr arge o gve a mplfed archecure for he mplemenaon of he auo-unng module n curren 3G nework. The econd arge of he he o apply he auo-unng and he elf-opmzaon concep o RRM algorhm n general and mobly managemen n parcular. The auo-unng of mobly algorhm uded n UMTS nework exended o long erm evoluon and ackled agan n he heerogeneou nework where only UMTS cell 2

21 Inroducon and WLAN hopo coex. The he demonrae hrough nework mulaon he feably of he auo-unng concep. The arge of he he are preened n Fgure 1.2. The objecve of he applcaon of he auo-unng o dfferen RRM algorhm (lef hand par of fgure 1.2) o evaluae he gan n erm of capacy and converge ha can be acheved n he nework compared o he cae whou any auo-unng. In he applcaon par, he auo-unng concep appled n a feedback baed regulaon loop. The elf-unng module calculae opmal value for he RRM parameer by dynamcally proceng he qualy ndcaor delvered by he nework. The conrol of parameer performed baed on a uly funcon called n he he renforcemen funcon. Th funcon gude he auo-unng algorhm o fnd he be parameer eng of he nework. Auo-unng of reource allocaon n UMTS Archecure of auo-unng Technque ued for auo-unng (Fuzzy logc and renforcemen learnng) Auo-unng of mobly parameer n UMTS Auo-unng of mobly parameer n 3GPP LTE UMTS-WLAN neryem mobly auounng Fgure 1.2. Targe of he he. 1.3 Orgnal conrbuon Th he nclude hree major conrbuon: preenng a mplfed archecure of auo-unng, applyng he auo-unng o dynamc reource allocaon and evaluang performance of elf opmzng mobly parameer n 3 cenaro: UMTS of handover, 3GPP LTE hard handover and neryem mobly. For he fr conrbuon, major dfference of he work preened here and earler publhed reul relaed o he archecure and o he way of preenng ool ued for auo-unng. In h work, an exenve explanaon of fuzzy-q-learnng baed auomac opmzaon preened wh proof. The auo-unng archecure publhed n [1] and n [5]. Wh repec o he fr ue-cae of auo-unng, a dynamc and auonomc approach o harng rado reource beween RT and NRT ervce n UMTS nework uded n [10]. In h conrbuon, a Fuzzy Q-Learnng Conroller (FQLC) ued o adap he reource reerved for 3

22 Inroducon he RT raffc accordng o perceved qualy ndcaor of each ervce. The FQLC combne boh fuzzy logc heory and renforcemen learnng mehod. Each bae aon fed ou wh a conroller whch manage reource. Learnng reul from each bae aon are uppled o a cenral FQLC of he nework. In h he, he auo-unng of mobly parameer furher uded and analyed n 3 dfferen echnologe. For he UMTS, we have addreed n [4] and [11] he auo-unng of of handover (SHO) parameer. Unlke earler ude, he auo-unng proce ue a fuzzy Q-learnng conroller o dynamcally adap SHO parameer o varyng nework uaon uch a raffc flucuaon and load dfference beween nework cell. The propoed mehod mprove he yem capacy compared o a clacal nework wh fxed parameer, balance he load beween bae aon and mnmze human nervenon n he UMTS mobly opmzaon ak. The auo-unng of SHO parameer and dynamc reource allocaon publhed laer n a book chaper [6]. The mobly auo-unng exended o he 3G Long Term Evoluon (LTE), ee [7]. Due o he fac ha he LTE yem a new andard, modellng nerference and capacy and developng a yem level mulaor hall be requred before udyng he mobly auo-unng. Th conrbuon acceped a a echncal repor and exended laer o a echncal pecfcaon n 3GPP andard [3]. The paper [9] nroduce a new WLAN-UMTS neryem mobly algorhm whch nclude boh coverage-baed and load-baed handover. The auo-unng of he parameer governng he propoed handover preened n [8]. The recen approache, defnng he yem ae n erm of load and congeon, are n her lm epecally n UMTS yem where dfferen merc are correlaed uch a capacy and coverage. In [2], we have propoed a new approach o deermne he cell ae n erm of load and congeon by jonly combnng dfferen qualy ndcaor. Th approach erve a an npu for new admon and mobly managemen algorhm. 1.4 The rucure Fr, he he preen an exenve ae of he ar of auo-unng n moble communcaon. The urvey of he auo-unng preceded by a bref preenaon of he relaed echnology. The auo-unng archecure and he preenaon of fuzzy renforcemen learnng algorhm are gven n chaper 3. The fuzzy renforcemen learnng erve a he ool ued o apply auo-unng n moble communcaon. In chaper 3, we hghlgh he veron fuzzy-q-learnng algorhm becaue of ue n he whole he excep chaper 5. Chaper 4 preen wo ue-cae of auo-unng n UMTS yem, he fr he dynamc reource allocaon and he econd he auo-unng of of handover parameer. Performance evaluaon of each ue-cae preened baed on a yem level dynamc mulaor. Chaper 5 preen an n deph nerference analy of a 3GPP LTE yem. The nerference analy and yem modellng erve a prelmnare o udy he mpac of LTE mobly auo- 4

23 Inroducon unng on he yem performance defned a he average uer hroughpu and congeon ndcaor of he nework. Chaper 6 devoed o nvegang he auo-unng of UMTS-WLAN neryem mobly. The auo-unng carred ou aumng a very gh couplng cenaro where a WLAN acce pon condered a a par of a UMTS cell. The reul reveal ha he capacy gan of ung he auounng concep furher greaer han he cae whou any auo-unng. The la chaper ummare and conclude he work preened, hghlgh he lmaon and pon oward poenal fuure work. 5

24 Auo-unng n moble communcaon: Relaed work 2 Chap. 2 Auo-unng n moble communcaon: Relaed work 2.1 Inroducon Auo-unng ha been uded recenly n he conex of 3G nework and mul-yem envronmen. The man focu ha been gven o une ome Rado Reource Managemen (RRM) algorhm (uch a mobly, admon conrol and reource allocaon). The adapaon of ome RRM algorhm ha already been nvegaed for he 2 nd generaon nework. A he complexy of wrele communcaon nework ncreae from day o day, he need for auo-unng become crcal. Reearch acvy on h opc ha been conduced n boh ndury and academa and ha been repored n he leraure. The objecve of h chaper o gve a comprehenve urvey on he reearch done n he area of auo-unng n wrele communcaon. Snce he work n h he cover he auo-unng of dfferen echnologe, he urvey gven for hree echnologe: GSM yem, 3G and beyond 3G yem whch nclude he cae of cooperave nework. To faclae he readng of he deraon, an overvew of each echnology gven wh a pecal focu on RRM algorhm and relaed parameer ha can be canddae for he auo-unng proce. The rucure of h chaper a follow: Secon 1 preen an overvew of he GSM echnology and relaed work on he auo-unng of RRM algorhm. Secon 2 decrbe he UMTS yem and relaed auo-unng work. Secon 3 rea dfferen approache of auonomc nerconnecon and adapaon of mul-yem nework. A he decrpon of hee yem nvolved, we lm he overvew o opc drecly relaed o auo-unng. 2.2 Auo-unng of GSM man funconale Overvew of GSM Nework and her evoluon "GPRS and EDGE" GSM became popular very quckly becaue provded mproved peech qualy and, hrough a unform nernaonal andard, made poble o ue a ngle elephone number and moble un around he world. The European Telecommuncaon Sandardzaon Inue (ETSI) adoped he GSM andard n 1991, and GSM now ued n 135 counre. The name GERAN ued by 3GPP (3 rd Generaon Parnerhp Projec) o refer o GSM rado acce echnology. The be way o creae h ucceful communcaon yem o dvde no varou ubgroup ha are nerconneced ung andardzed nerface [16]. A GSM nework can be dvded no hree group (ee Fgure. 2.1): The moble aon (MS), he bae aon ubyem (BSS) and he nework ubyem (NSS). A he begnnng, GSM yem uppor only voce ervce. Alhough he fr evoluon of he andard, namely hgh-peed crcu-wched daa (HSCSD) [14], enable moble phone o uppor daa rae up o 38.4 kbp, compared wh 9.6 kbp for regular GSM nework. 6

25 Auo-unng n moble communcaon: Relaed work Tranmon peed of up o kbp are avalable wh moble phone ha uppor he GSM andard for General Packe Rado Servce (GPRS) [15]. The hgh bandwdh acheved by ung egh melo, or voce channel, mulaneouly o he packe wchng ervce. NSS OM Fgure 2.1. GSM yem archecure. From an operaor pon of vew, he nroducon of GPRS faclae he arrval of everal new moble daa applcaon and offer advanage of managng rado reource. Wh GPRS, no only poble o ue nework reource n a more effcen way by reang applcaon daa flow regardng her acual need, bu alo o dfferenae among ervce uer regardng her ubcrbed qualy of ervce (QoS). The evoluon of GSM oward UMTS enal he exploaon of GPRS enhanced veron, called EDGE (Enhanced Daa Rae for GSM Evoluon) [16]. Applyng new modulaon, new codng cheme and opmzed lnk qualy conrol algorhm, EDGE allow o reach hgher hroughpu han GPRS of up o 59.2 kbp per GSM phycal channel. The addonal ue of 8-PSK modulaon enable he nroducon of new modulaon and codng cheme. The advanage of he new modulaon cheme he uppor for hgher daa rae under good channel condon, and o reue a he ame me he channel rucure of he GPRS yem. The ucce of GSM nework drven by RRM algorhm. RRM n uch nework nclude admon conrol and reource allocaon, congeon conrol, packe chedulng, mobly conrol and cell reelecon RRM algorhm and arge auo-uned parameer In h ubecon, we are gong o gve an overvew of RRM algorhm n GSM nework. We don' hope nor do we aemp o cover all algorhm, bu only a broad overvew of admon, 7

26 Auo-unng n moble communcaon: Relaed work congeon and mobly conrol gven. The relaed RRM parameer ha can be auo-uned are hghlghed. Admon conrol and reource allocaon The purpoe of he admon conrol o calculae whch nework reource are needed o provde he qualy of ervce requeed. Accordng o reource avalably, he new uer wll be acceped or dened acce. The admon conrol procedure am o maxmze he number of admed uer and o guaranee he QoS of call beng carred ou. The admon conrol funcon ake no accoun a varey of dfferen reource: rado reource, ranpor reource, Rado Lnk Conrol (RLC) buffer ze and bae aon phycal reource (me lo). The avalably of all hee reource requeed when performng an admon conrol decon. Accordng o h nformaon, dfferen decon can be made, uch a accep/rejec he connecon, queue he reque or perform a dreced rery. The reource conrol performed by he admon conrol alo uppor he ervce reenon hrough he allocaon/reenon prory arbue. Th feaure ervce dependen, provdng uer' dfferenaon dependng on her ubcrpon profle. If here no enough capacy, hgh prory uer are acceped before lower prory uer. Furhermore, once uer are admed, admon conrol allow hgh prory uer o rean her connecon agan lower prory uer n cae of overload. In crcu-wched doman, admon conrol decon normally baed on he avalably of me lo n he arge cell. However, no n all uaon he avalably of me lo enough o accep a new connecon. I alo poble ha new call are blocked due o hgh nerference level. However, n packe-wched doman, admon conrol manage boh non-real me and real me connecon reque dfferenly. In cae of non-real me ervce uch a neracve and background raffc clae, he experenced hroughpu decreae gradually when he number of uer ncreae, up o he blockng uaon. By lmng he non-real me load, poble o provde a beer ervce o already admed uer. Recenly, dynamc reource allocaon beween CS and PS ervce ha been mplemened n GSM/GPRS nework [15]. For h purpoe, a number of me lo dedcaed o each ervce and a hared band can be ued dynamcally by boh CS and PS uer (Fg. 2.2). The CS call have prory on GPRS raffc on he hared band.th concep of "capacy on demand" ued o adap he nework o an ncreang GPRS raffc. The capacy of each band defned by he operaor hrough pecfc parameer, or can be dynamcally adaped a a funcon of raffc of each ervce. To monor he nework and o check he well funconally of he admon conrol algorhm, ome key performance ndcaor (KPI) are perodcally monored. We ce ome of hem: Syem accebly ndcaor: Cover he uer capably o ge acce o he rado reource. I nclude for example Call Seup Succe Rao (CSSR), call eup delay, Seon Seup Succe Rao (SSSR), cell load ec. In general, he yem accebly meaured by he Admon rae or by complemenary call blockng rae. 8

27 Auo-unng n moble communcaon: Relaed work Dedcaed GSM TS (CS) Shared reource TRX 1 CCCH TS TS TS TS TS TS TS TRX 2 TS TS TS TS TS TS TS TS Shared reource and Dedcaed GPRS TS change dynamcally baed on CS and PS raffc load Dedcaed GPRS TS (PS) Fgure 2.2. Dynamc reource allocaon beween CS and PS ervce. Syem reanably: Cover he ably o keep up a voce call or a eon daa connecon wh a dered qualy of ervce. Reanably could be defned wh he dropped call probably. Indcaor relaed o uer uaon: nclude for example he receved gnal rengh, he experenced b rae. For he call admon conrol algorhm, ome parameer could be canddae for he auounng procee. For example, we ce: Number of channel dedcaed o CS ervce or/and he number of channel dedcaed o PS ervce: The dynamc adapaon of hee parameer could balance he perceved QoS beween dfferen ervce clae and lead o an effcen ue of reource. Maxmum load: ued o preven he yem congeon. Threhold for guard channel: New call and handover call are compeng for he uage of he rado reource. Therefore, very derable o manan call already n he nework by reervng for hem a handover guard band. Th hrehold can be dynamcally adaped accordng o he call droppng rae and call blockng rae. Tme ou hrehold: when he perceved qualfy of ervce degrade and become below a ceran hrehold durng a perod equal o he me ou hrehold, he connecon dropped. Mobly managemen algorhm The mobly managemen compre 2 phae dependen on he uer uaon. The fr when a moble n dle mode and he econd when n he connecon mode. For he fr mode, moble perform elecon and reelecon procedure, wherea handover mechanm performed n he econd mode. 9

28 Auo-unng n moble communcaon: Relaed work GSM dle mode In he GSM dle mode, he andard pecfe wo crera [12]. The fr one, denoed C1, ued for he cell elecon and reelecon procedure, and he econd one reelecon creron C2. The pah lo creron parameer C1 defned by: Where C1 (A - Max(B,0)) A RLA_C -RXLEV_ACCESS_MIN B MS_TXPWR_MAX_CCH P RLA_C he average receved gnal level. RXLEV_ACCESS_MIN he mnmum receved gnal level a he moble aon requred for acce o he yem. MS_TXPWR_MAX_CCH he maxmum ranmed power level a moble may ue when acceng he yem unl oherwe commanded. P he maxmum oupu power of he moble. All value are expreed n dbm. The pah lo creron [13] afed f C1 > 0. The reelecon creron C2 only ued for cell reelecon and defned by: C2 C1+ CELL_RESELECT_OFFSET CELL_RESELECT_OFFSET apple an offe o he C2 reelecon creron for ha cell. Th parameer may be ued o gve dfferen prore o dfferen band when mul-band operaon ued. Cell reelecon rggered f C1 fall below zero for a perod of 5 econd or f he C2 value of he new cell exceed he C2 value of he ervng cell by a lea Cell_Reelec_Hyere db, for a perod of 5 econd. Enhanced veron of he reelecon creron gven n [12] and [13]. GSM conneced mode Handover proce ake care of enurng ha any uer alway conneced o he mo uable cell. Handover n GSM a hard handover,.e. he connecon releaed from he old cell and eablhed wh he new ervng cell. Handover can be performed for dfferen reaon. For nance, when he uer leave he domnance area of he acual ervng cell or when he call experence bad qualy. In power-budge-baed handover, he moble meaure perodcally he receved gnal level of ervng cell and he neghbourng cell. When deec anoher cell wh beer gnal level, carre ou a power-budge-baed handover o he arge cell. The qualy-baed handover urgenly rggered due o he degraded qualy or o he low receved gnal level. Furhermore, he moble nae a dance-baed handover, when become very far from beer cell. In GSM yem, a uer can nae a voce communcaon wh a cell, whch doe no uppor packe wchng raffc, and change laer o GPRS ervce. In h cae, he moble lkely o be handed over o anoher cell upporng GPRS ervce. For he mobly conrol, he mo approprae parameer ha can be auo-uned are: 10

29 Auo-unng n moble communcaon: Relaed work Handover margn: auo-unng ueful for he congeon conrol. L of neghbourng cell: he opmzaon of he neghbourng cell l aure he connecvy of a uer o he be cell. I alo reduce he meauremen me. CELL_RESELECT_OFFSET: he auo-unng of h parameer dynamcally balance he load beween he GSM band. Handover capure margn: ued n ner-band handover. Congeon conrol Congeon conrol mechanm hould be degned o face uaon n whch he yem ha reached a congeon uaon and herefore he QoS guaranee are a rk due o he evoluon of yem dynamc. The ak of congeon conrol o monor, deec and handle uaon when he yem reachng a near overload or an overload uaon wh he already conneced uer. Th mean ha ome par of he nework ha run ou, or wll oon run ou of reource. The congeon conrol hould hen brng he yem back o a able ae a eamle a poble. The congeon conrol proce dvded no 2 phae: Phae 1: Congeon deecon phae When uer ha are already admed can no afy her guaraneed QoS o her ervce for a pecfc percenage of me, he nework condered o be n an overload/congeon uaon. The cell load, congeon rae and oher blockng ndcaor are he ndcaor of he congeon uaon. Phae 2: Congeon reoluon phae Once congeon ha been deeced, all new eon and handover are rejeced a hey wll ncreae more he load of he nework. The conrol algorhm check f he overload caued due o uer ha volae her QoS rercon n erm of b rae, whch mean ha re o fnd f here are uer ha ranm wh hgher b rae han hey hould, accordng o he ervce agreemen made beween he nework and he uer. If uch uer ex, hen he algorhm lower her b rae o he value defned n he ervce agreemen. If here ll congeon, ongong eon are nered no a able, ordered by prory. The auo-unng of RRM parameer of each cell very uable o reduce he nework congeon and o dynamcally balance he load beween cell. For nance, we can conrol he ze of a cell by ju modfyng he value of C2. Wh h echnque a uer can be forced o make a cell reelecon o anoher under-loaded cell. Adapng handover margn alo a oluon for accelerang handover of uer from congeed cell o neghbourng cell. Some oher parameer are alo condered a canddae for auo-unng uch a: maxmum load and me o rgger congeon algorhm Leraure revew of GSM auo-unng The reearch on auo-unng of GSM nework ared n he md-90. Edward and Sankar nroduced n [25] a new GSM handover algorhm baed on fuzzy logc conrol [53]. They condered he receved gnal rengh and he moble dance from he bae aon a he 11

30 Auo-unng n moble communcaon: Relaed work monored ndcaor for he auo-unng proce. The GSM handover margn wa uned accordng o a e of rule. Ther mehod had been exended laer o he handover n mcrocellular envronmen ung more relevan qualy ndcaor [26]. The mo relevan example of auo-unng n GSM nework gven n [27] and [28]. Auhor have uded an auo-unng herarchcal nework baed on he dynamc adapaon of a handover hrehold. The goal of he auo-unng archecure wa o mprove capacy whle mananng qualy n a nework wh GSM herarchcal cell rucure (HCS). GSM HCS a combnaon of mcro and macro cell. Traffc balanced beween mcro and macro layer by ner-layer handover. Wh he raffc mgraon beween layer, a beer load drbuon obaned allowng he full exploaon of he avalable reource. The moble aon ue he receved gnal rengh from a cell o make handover decon. The HCS feaure defne a gnal rengh hrehold (LEVTHR) for each cell n he mcro-layer. If a moble aon conneced o a macro cell meaure a gnal rengh from a mcro-cell hgher han he hrehold, a handover o he mcro-cell performed. Lkewe, f he gnal rengh fall below h hrehold, he moble aon abandon he mcro cell. In [28], he role of he elf-unng agen o correcly emae he handover hrehold (LEVTHR), a a funcon of he capacy and he qualy (nerference) n all cell, o avod congeon n he mcro-cell, and o avod degradaon of QoS due o nerference. A ucceful feld ral ha been carred ou n Hong Kong wh he operaor SmarTone [28]. Fve GSM900 mcro-cell were choen n layer one and addonal 100 neghbourng cell were alo ncluded (alhough no menoned o whch layer hee cell belong). Every 10 econd load and QoS daa were en o he elf-unng agen. The parameer LEVTHR auo-uned Every 5 mnue. Ercon clamed ha capacy and QoS are mproved for he fve e cell. The congeon n he eed cell wa reduced from 5-10% o 1%. Wh repec o he dynamc channel reervaon and admon conrol, auhor n [29] and [33] gave a comparave udy beween wo mobly hory-baed cheme. The aggregae hory of mobly oberved n each cell ued o predc probablcally he drecon of an MS and o reerve for a band n he arge cell. I alo remarkable o ee he reference [30] n whch he auhor nroduced a mehod for dynamcally adjung he reerved bandwdh n each cell a a funcon of he monored droppng probably or he ulzaon level of he guard capacy. In [33] he moble poonng avalably, a obaned by GPS (Global Poonng Syem) or oher poonng echnque f avalable wh he parcular moble yem, erve a a ba for nex cell predcon. Th yem clamed o ake advanage of real-me meauremen nead of hory-baed cheme o make predcon [29]. In [34] a predcon baed on a pror defned raffc model propoed. In [35], auhor ue a developed mobly model and meaured droppng probable o adju he reerved bandwdh. 12

31 Auo-unng n moble communcaon: Relaed work 2.3 UMTS nework Th econ preen a comprehenve overvew of UMTS nework and evoluon, HSPA (Hgh Speed Packe Acce) [17] [18]. RMM algorhm are preened a well wh a pecal empha on dfferen RRM parameer ha can be auo-uned Overvew of UMTS Nework and evoluon UMTS nework ha been nroduced a a hrd generaon moble communcaon yem. 3GPP organzaon [20] n charge of pecfcaon. I ha pecfed dfferen echnologe for UMTS nework: for example, Frequency Dvon Duplex (FDD), Tme Dvon Duplex (TDD), and HSPA. In he majory of 3GPP pecfcaon documen, he name UTRAN ued o and for he UMTS rado acce nework. The ranmon rae capably of UMTS provde a lea 144 kbp for full-mobly applcaon n all envronmen, 384 kbp for lmed-mobly applcaon n he macro- and mcrocellular envronmen, and 2 Mbp for low-mobly applcaon parcularly n he mcro or pco-cellular envronmen. In 3GPP releae'5, he ranmon rae capably enhanced for he downlnk o reach 10 Mbp. The UMTS yem offer dfferen ype of qualy of ervce (QoS) for dfferen ype of cuomer and her applcaon. A key QoS arbue nclude prory acce for dfferen ype of uer. For example, real me prory acce ypcally apple o voce ervce and relable daa ranfer appled o neracve daa ervce. There are four dfferen QoS clae: converaonal, reamng, neracve and background cla [21]. The man dnguhng facor beween hee QoS clae how delay enve he raffc. Whl he UMTS rado nerface compleely new wh repec o any 2G yem, he core nework (CN) nfrarucure baed on an evoluon of he curren GSM/GPRS one. Fgure 2.3 how UMTS nework archecure a and n Releae'5 [19]. I con of a e of Rado Nework Subyem (RNS) conneced o he CN va he Iu nerface. The CN prmarly con of a crcu-wched (CS) doman and a packe-wched (PS) doman. Thee wo doman dffer n how hey handle uer daa. The CS doman offer dedcaed crcuwched pah for uer raffc and ypcally ued for real-me and converaonal ervce, uch a voce and vdeo conferencng. The PS doman, on he oher hand, nended for end-oend packe daa applcaon, uch a fle ranfer, Inerne browng, and e-mal. The RNS con of a conroller (he Rado Nework Conroller, or RNC) and one or more ene called Node B, whch are conneced o he RNC hrough he Iub nerface. A Node B upernend a e of cell whch may be FDD, TDD, or mxed. In UMTS, Dfferen RNC can be conneced o each oher hrough he Iur nerface. RNC he boundary beween he rado doman and he re of he nework. The proocol opened n he uer ermnal o manage he rado lnk are ermnaed n he RNC. Above he RNC are he proocol ha perm nerconnecon wh he CN and whch depend on. 13

32 Auo-unng n moble communcaon: Relaed work Node B Iub RNC Connecon Servce CN PCM Node B Node B Iur RNC Iu MSC+ VLR ATM/ AAL2 HLR Node B GSN+ IP/ GTP UTRAN Packed Servce CN GPRS + Fgure 2.3. UMTS archecure. UMTS Core Nework UMTS nework ue n he ar nerface WCDMA (Wde-band Code Dvon Mulple Acce) acce echnology. The concep of WCDMA nher from he pread pecrum CDMA. CDMA ue a form of drec equence whch, n eence, mulplcaon of a more convenonal communcaon waveform by a peudo-noe bnary equence n he ranmer. Spreadng ake place pror o any modulaon, enrely n he bnary doman. The noe and nerference, beng uncorrelaed wh he peudo-noe equence, become noe-lke and ncreae n bandwdh when hey reach he deecor. Flerng mechanm ha rejec mo of he nerference power can enhance he Sgnal-o-Inerference plu Noe Rao (SINR). I ofen ad [19], ha he SNR enhanced by he proceng gan W/R, where W he chp rae and R he daa rae. WCDMA ue a chp rae equal o 3.84 Mcp whch lead o a carrer bandwdh of approxmaely 5 MHz. The nherenly wde carrer bandwdh of WCDMA uppor hgh uer daa rae and alo ha ceran performance benef, uch a ncreaed mul-pah dvery. WCDMA ue varable preadng facor and mul-code connecon. In addon o he bac rado acce capable, UMTS archecure provde everal oher advanage, ncludng hgher bandwdh over he rado nerface and beer handoff mechanm, uch a of handover for crcu-wched bearer channel. Sof handover refer o he ably o manan an ongong connecon beween he moble ermnal and he nework hrough more han one bae aon; h capably parcularly mporan n 3G yem. UMTS yem underwen a conderable evoluon by he nroducon of he HSDPA (Hgh Speed Downlnk Packe Acce) n 3GPP releae'5 [22]. HSDPA pecfcaon wa releaed n 2002 and wa condered he mo gnfcan rado relaed updae nce releae'99. HSDPA baed on a drbued archecure where he proceng cloer o he ar nerface a he Node B for low delay lnk adapaon. To acheve a hgh-peed downlnk ranmon, HSDPA mplemen a chedulng for he downlnk packe daa operaon, hgher order modulaon, adapve modulaon and codng, 14

33 Auo-unng n moble communcaon: Relaed work Hybrd Auomac Repea Reque (HARQ) and lnk adapaon accordng o he momenary channel condon. The HSDPA concep offer over 100% hgher peak uer b rae han Releae 99 n praccal deploymen. I comparable o Dgal Subcrber Lne (DSL) modem b rae n wrelne communcaon. I exend he UMTS b rae up o 10 Mbp. Th hgher b rae are obaned wh hgher order modulaon, 16-QAM, and wh adapve codng and modulaon cheme. HSDPA able o uppor no only non real me UMTS QoS clae bu alo real me UMTS QoS clae wh guaraneed b rae. A new mprovemen n he UMTS rado nerface wa pecfed n releae'6 wh he nroducon of HSUPA (Hgh Speed Uplnk Packed Acce) [23]. HSUPA ue an uplnk enhanced dedcaed channel (E-DCH) wh dynamc lnk adapaon mehod a already enabled n HSDPA,.e. horer ranmon me nerval, hereby enablng faer lnk adapaon, and alo a hybrd HARQ wh ncremenal redundancy, hereby makng reranmon more effecve. HSUPA offer peak daa rae up o 5.5 Mbp. The la evoluon of UMTS nework oberved n releae'7 wh he nroducon of a new yem whch ha been a he 70% of pecfcaon whle wrng h deraon. Th evoluon referred o he 3GPP Long Term Evoluon (LTE) yem [97] [98]. LTE yem, called omeme "uper 3G" or "4G", expeced o offer a pecral effcency beween 2 o 3 me bgger han 3GPP releae'6. I wll provde up o 100Mb/ for 20 MHz of pecral bandwdh. Boh he rado and he core nework par of he LTE echnology are mpaced: The yem archecure more decenralzed; The RNC preen n he 3G yem removed; and RRM funconale are moved o an "upgraded" bae aon called Evolved Node B (enb). Furher decrpon of LTE yem gven n chaper UMTS RRM algorhm and relaed parameer In UMTS, RRM algorhm are needed o guaranee QoS, o manan he planned coverage area, and o offer hgh capacy. RRM funconale nclude mobly managemen, Power Conrol, Admon Conrol, Packe Schedulng, Load Conrol, Dynamc Channel Allocaon and Code Managemen [24]. Snce he WCDMA acce echnology nerference lmed, power conrol needed o keep he nerference level a mnmum n he rado nerface and o provde he dered QoS. Lke oher cellular nework, Handover mechanm are needed o handle he mobly of he uer acro cell boundare and o realze a connuou coverage. Admon conrol, load conrol and packe chedulng are requred o guaranee he qualy of ervce and o maxme he yem hroughpu wh a mx of dfferen b rae, ervce and qualy requremen. In h econ, we urvey UMTS RRM algorhm wh a pecal focu on admon conrol, mobly managemen and load conrol. For each algorhm, he RRM parameer ha can be auo-uned are gven. 15

34 Auo-unng n moble communcaon: Relaed work Admon conrol Admon conrol regulae he operaon of he nework n uch a way ha enure unnerruped ervce provon, and accommodae n an opmal way new connecon reque. Admon conrol emae wheher a new uer hould have acce o he yem whou mparng he qualy requremen of exng uer. If he accepance of a uer wll ncreae he nerference power and he load on he cell o a level whereby he qualy of he ongong call reduced and he qualy of he call elf canno be guaraneed, he uer wll no be admed o he yem. Due o he dependence beween capacy and coverage n WCDMA echnology, he coverage area of he cell reduced below planned value and he qualy of ervce canno be guaraneed f he rado load ncreae excevely. The avalably of ranmon reource alo verfed by he admon conrol proce. There no abolue number of maxmum avalable channel ha can be allocaed o poenal uer. Th he Sof Capacy propery of UMTS. The number of connecon can no pecfy he acual capacy of he cell. The lm n capacy are deermned by he nerference ha generaed a he bae aon by all he gnal ha are ranmed by he uer n he ame cell and oher cell, and by he propagaon channel condon n he coverage area. The uer poon whn he cell affec he capacy of he cell. Th mean ha he load on he cell canno be predced by he number of connecon a any one me, and capacy value alone can no be ued o adm or rejec uer. The admon conrol algorhm execued when a bearer e up or modfed. The effecve load ncreae by admng anoher bearer emaed, boh for he uplnk and downlnk. The bearer can only be admed f he uplnk and downlnk admon conrol adm, oherwe rejeced due o exceve nerference on he rado nework. The admon conrol funconaly locaed n he RNC where he load nformaon from everal cell can be obaned, a well a beng emaed n he uplnk and downlnk. The erm Conrollng RNC (CRNC) ued o defne he RNC ha conrol he logcal reource of UTRAN acce pon. In cae one moble-utran connecon requre reource from more han one RNC, he CRNC nvolved ha wo eparae logcal role wh repec o he moble connecon. When he RNC hold he Iu bearer for a ceran UE called he Servng RNC. If anoher RNC nvolved n he acve connecon, known a he Drf RNC. The adapaon of parameer nvolvng he admon conrol mprove he yem capacy and he perceved uer qualy. Dfferen parameer are ubjec for auo-unng proce. Among hem, we ce for example: The percenage of power and number of code dedcaed o gnallng channel: he auounng of hee parameer allow explong reource dedcaed o he gnallng channel for raffc channel. The dedcaed code and power o raffc channel. I very ueful o balance he qualy of ervce of dfferen raffc clae. The load hrehold: he adapaon of h parameer preven he yem from congeon 16

35 Auo-unng n moble communcaon: Relaed work The maxmum load. The dfference beween he load hrehold and he maxmum load generally reerved o he mobly. I adapaon make a be rade-off beween blockng and droppng rae. The Downlnk/Uplnk SIR arge. I ueful o adju he perceved qualy of each uer, o an amoun of power can be aved and he nerference reduced. Mobly managemen The mobly managemen mechanm nclude handover and cell elecon-reelecon. The cell elecon-reelecon occur when he uer n dle mode. Handover occur when a uer n connecon move from he coverage area of one cell o he coverage area of anoher one. I can alo be performed beween frequence or o drbue load/uer among neghbourng cell n he cae of he overload cell uaon. Effcen handover algorhm are a co-effecve way of enhancng he capacy and QoS of cellular yem. UMTS dle-mode The cell elecon proce allow he uer o elec a uable cell where o camp on n order o acce avalable ervce. In h proce he moble can ue ored nformaon (Sored nformaon cell elecon) or no (Inal cell elecon) [37]. For he nal cell elecon, he uer hall can all rado frequence n he UTRAN band accordng o capable o fnd a uable cell. On each carrer, he uer eek he ronge cell. Once a uable cell found h cell hall be eleced. The procedure of Sored Informaon Cell Selecon requre ored nformaon of carrer frequence and oponally alo nformaon on cell parameer, e.g. cramblng code, from prevouly receved meauremen conrol nformaon elemen. Once he uer ha found a uable cell he uer hall elec. If no uable cell found he nal cell elecon procedure hall be ared. For UMTS FDD mode, he cell elecon creron S fulflled when: where Srxlev >0 and Squal>0 Srxlev Qrxlevmea - ( Qrxlevmn + QrxlevmnOffe) - Pcompenaon Squal Qqualmea - ( Qqualmn + QqualmnOffe ) The Srxlev and Squal are repecvely Cell Selecon Rxlev value and Cell Selecon qualy value. Qrxlevmea he meaured Receved Sgnal Code Power of he plo channel, (CPICH_RSCP). Qqualmea he meaured cell qualy value. The qualy of he receved gnal expreed n CPICH Ec/N0 for FDD cell [37]. Qrxlevmn he mnmum requred Rxlev n he cell. QrxlevmnOffe an Offe o he gnalled Qrxlevmn aken no accoun n he Srxlev evaluaon a a reul of a perodc earch for a hgher prory nework [40]. Qqualmn he mnmum requred qualy level n he cell and QqualmnOffe Offe ued a he ame way of QrxlevmnOffe. For Pcompenaon, reader nved o ee [37]. Oher pecfcaon 17

36 Auo-unng n moble communcaon: Relaed work of nework and cell elecon are found n [38] and [39]. Procedure and creron for cell reelecon are found n [37] and [38]. UMTS conneced mode UMTS conneced mode refer generally o he procedure performed by uer when n communcaon ae. The handover mechanm conue he man procedure n conneced mode. In UMTS yem, here are 4 ype of handover: Sof handover (nra-frequency HO), ofer handover (nra-frequency and nra-e HO), hard handover (ner-frequency HO) and ner rado acce echnology handover. Sof Handover (SHO) occur when a new connecon eablhed before he old connecon releaed. Durng a of handover poble for mulple cell o mulaneouly uppor he moble aon call. An algorhm ha mplemen h knd of handover he Acve Se Updae algorhm. Th algorhm found n he pecfcaon [24], however, each conrucor ha own algorhm. The algorhm dealed n chaper 4. The of handover parameer have o be carefully monored, a exceve of handover can mpar he downlnk capacy. Each of handover connecon ncreae he ranmed nerference o he nework. More orhogonal code are ued n he downlnk ung of handover connecon han ngle lnk connecon. I he ak of rado nework plannng and opmzaon o keep he of handover overhead below 40% whle ll provdng enough dvery n he uplnk and downlnk [19]. The dynamc adapaon of of handover parameer could ackle he ue of be rade-off beween of handover overhead and cell capacy. Durng a ofer handover, a moble aon n he overlappng cell coverage area of adjacen ecor of one bae aon. A communcaon lnk beween he moble and each ecor eablhed by ung eparae code n he downlnk, o he moble can dnguh he gnal. Fgure 2.4. Inra-Frequency Sof Handover In hard handover, he uer ha prevou rado lnk removed, and replaced by oher rado lnk. Iner-frequency hard handover ued o enure he handover pah from one cell o anoher cell n he cell cluer where he frequence are no he ame. Generally he frequency reue facor for 18

37 Auo-unng n moble communcaon: Relaed work UMTS one, meanng all he bae aon ranm on he ame frequency. However, h doe no mean all bae aon are requred o ranm on a common frequency. Iner-frequency handover ued for example n Herarchal Cell Srucure (HCS) nework beween eparae layer and n heerogeneou nework beween dfferen Rado Acce Technologe (RAT). Becaue hard handover nvolve nerrupng he call whle he call uppor changed, alo known a break before make. The of handover parameer ha can be ubjec of auo-unng are: Maxmum acve e ze: he maxmum number of cell n he acve e conneced o he uer. The auo-unng of he maxmum acve e ze dynamcally opmze he number of moble n of handover condon and he frequency of he acve e updae. Th laer relaed o he png-pong effec [4]. Sof handover Hyere_even1A, Hyere_even1B and Hyere_even1C: hey are ued repecvely for addng, deleng and replacemen of a lnk n he acve e of a moble. The auo-unng of hee parameer very ueful o relef permanen localed congeon problem caued by he uneven appearance of raffc n a cellular nework boh n me and pace. Traffc balancng hrough permanen adapaon of SHO parameer on an adjacency-by-adjacency ba can grealy mnme congeon whou he need for any hardware upgrade, hu provdng a co-effecve mehod o ncreae nework capacy [4]. The hyere regon mananed o avod unneceary handover due o he png-pong effec. Neghbour cell l: auo-unng allow reducng he meauremen me of gnal comng from cell and perm he uer o elec or o be handed over o he be cell n he nework. I alo uable for reducng he neghbour cell l. Congeon conrol A n a GSM nework, congeon conrol n UMTS aure he well funconaly of he yem and brng back o a able ae when overload uaon occur. The congeon conrol procedure nclude wo phae: deecon and reoluon: Congeon deecon: Some creron mu be nroduced n order o decde wheher he nework n congeon uaon or no. A poble creron o deec when he yem ha enered he congeon uaon and rgger he congeon reoluon algorhm when he load facor η ncreae over a ceran hrehold η CD durng a ceran amoun of me, T CD,.e. f η ηcd n 90% of he frame whn T CD. The load facor η meaure he heorecal pecral effcency of a WCDMA cell and mu be 0<η<1. Uually he nework planned o operae below a ceran maxmum load facor η max and he congeon deecon hrehold hould be e n accordance o he maxmum planned value. Congeon reoluon: When congeon aumed n he nework, ome acon mu be aken n order o manan he nework ably. The congeon reoluon algorhm execue a e of rule o lead he yem ou of he congeon au. The yem can reduce he hroughpu of packe daa uer and puh moble o anoher WCDMA carrer or o a GSM nework. I can alo decreae b rae of real me uer and drop low prory call n a conrolled fahon. 19

38 Auo-unng n moble communcaon: Relaed work The auo-unng of ome RRM parameer een oo a a oluon for congeon. So, by mean of adapng handover parameer, raffc puhed from overloaded cell o under-loaded cell. The adapaon of cell-elecon crera of a cell a a funcon of load and he load of he neghbourng cell allow reducng he coverage area of overloaded cell and rechng underloaded cell' one. The ervce area of a cell can alo be affeced by adapng he power agned o he CPICH of each cell Relaed work on he auo-unng of UMTS parameer A plehora of reference addreng he ue of auo-unng and RRM parameer' adapaon found n he relaed leraure. In [1], he Gandalf projec propoe a phycal auo-unng funconal archecure. The propoal deal wh he on-lne and off-lne auo-unng, namely he conroller of he auo-unng ake acon quckly (every econd or mnue) or lowly (1 hour and more). The propoed archecure wa gven for managemen plane, conrol plane, and uer plane. The relaonhp beween layer ha been gven a well. In [41], he heorecal approach of auomac neghbour cell l opmzaon wa nroduced. An nal neghbourng cell l ued n he RNC and allow calculang nework ac. The RNC collec and end performance ac (uch a average CPICH RSCP repored by moble ermnal, HO proporon, HO ucce rao and HO effor) o an opmzaon ool. Baed on RNC HO ac, unneceary cell are removed from he l and new cell are added o he l. The auo-uned neghbour l are en back o he RNC. In [41], h mehod ha alo been eed n a commercal nework of 95 cell o opmze neghbour cell l n 3G yem. The overall yem qualy (n erm of ucceful HO procedure) ha been mproved and he average lengh of he neghbour cell l gnfcanly reduced. In [38], 3GPP ha already pecfed an onlne opmzaon approach of he neghbour cell l, called Deeced Se Reporng. In he approach, he nework command moble o deec and repor cell whch are no on he neghbour l. Deeced e reporng even-rggered bu whou any acve e updae procedure. In [42], auhor from Noka decrbed a mehod o auomae he eng of common plo power n a WCDMA nework. The CPICH power auo-unng algorhm baed on he graden decen mehod o mnmze a co funcon. Th co funcon ake no accoun wo em: he devaon of he coverage from he arge and he devaon of he load from he load n he neghbourng cell. The mehod howed lghly encouragng mulaon reul. Two mehod are nroduced n [43] o emae he uplnk and downlnk planned E b /N o (meanng he emaed E b /N o requremen per ervce for proper decodng of he gnal) of WCDMA ervce and aferward o auo-une he uplnk and downlnk E b /N o value for packe daa. Smulaon reul howed ha h auo-unng mehod allow mprovng he yem performance n erm of hroughpu. The auo-unng of WCDMA lnk power per ervce and he cell downlnk load level arge wa gven n [44]. The qualy ndcaor ued n [44] were call-blockng probably, packe queung probably and downlnk power ouage. Thee ndcaor were compared o allowed level and a able of heurc rule gave he parameer adjumen dependng on he devaon of he ndcaor from he allowed level. 20

39 Auo-unng n moble communcaon: Relaed work In [45], handover parameer are auo-uned baed on co funcon. The co funcon depend on blocked call rao and downlnk ranm power. A econd order graden mehod adoped o mnmze he co funcon. Smulaon reul howed ha HO wndow auo-unng mehod allow mprovng he yem performance n erm of blockng rae. Homnan e al. [46] dcued he feably of conrollng of handover hrehold n IS95 and CDMA2000 nework [19]. Auhor condered he E b /N 0, he ouage probably and he remanng capacy n he ervng cell a npu o he conroller. A e of rule, called fuzzy nference yem, have manually been conruced baed on he rado engneerng experence. Lkewe reference [47] nroduce fuzzy nference yem appled o he ue of of handover n CDMA moble communcaon nework, preenng new algorhm for hrehold adjumen amed a reducng call blockng probably and conrollng he qualy of ervce. Ye and h colleague have propoed n [48] a new CDMA fuzzy call admon conrol algorhm baed on he emaon of ome ndcaor o decde wheher o accep or o block a new or handover call. In her admon conrol algorhm, fuzzy logc ued o emae he uer mobly and he effecve bandwdh ha would be ued by he new uer requeng acce. Lkewe reference [49] and [50] preen call admon conrol cheme ulzng he fuzzy logc approach n CDMA yem, hghlghng effecvene n handlng mobly and raffc model uncerany. In [51], an opmzed fuzzy logc conroller ha been propoed for mulaneouly auo-unng admon conrol and macro-dvery parameer. Fuzzy rule are degned and opmzed ung a combnaoral mehod called parcle warm. Snce he opmzaon very me-conumng, carred ou off-lne. In he he deraon of Herve Dubrel [52], boh combnaoral mehod and Renforcemen Learnng (RL) [53] have been ued o opmze he auo-unng proce of handover and admon conrol parameer n UMTS yem. The auo-unng ool wa a fuzzy logc conroller. In [54] he auhor have propoed a RL-baed call admon conrol algorhm. Inead of conrollng RRM parameer, her propoed cheme a he yem n a decon proce. In each nework uaon, he conroller ha only wo acon: rejec or accep he call. A mall number of ae (yem npu ndcaor) ued, namely he number of moble uer n each raffc cla. In [55] and [56], he QoS provonng problem by mean of Q-learnng algorhm ha been preened. Two algorhm of reource managemen are condered: call admon conrol and bandwdh adapaon. Two QoS conran are aken no accoun n [56]: he probably of hand-off droppng and he normalzed average allocaed bandwdh of each raffc cla. In her paper, auhor have choen he Q-learnng algorhm, becaue doe no requre cloed form formulae of he yem. Chen e al. [57] have ued a well Q-learnng mehod o adap he ranmon rae n he WCDMA rado reource conrol and o accuraely emae he co for he mul-rae ranmon problem. All he above reference ung renforcemen learnng have modelled he yem a a Sem-Markov Decon Proce (SMDP). 21

40 Auo-unng n moble communcaon: Relaed work 2.4 Mul-yem nework Beyond 3G yem (B3G) con of a number of exng yem wh dfferen rado acce echnologe uch a GERAN, UTRAN, WLAN, ec. provdng dfferen QoS and coverage. The neroperaon beween hem an eenal requremen o acheve a eamle ervce a well a effcen moble envronmen wh end-o-end QoS. When hgh degree of negraon beween dfferen echnologe ex, he provonng of ervce more effcen and he elecon mode of he be rado acce faer a well a he handover procedure. On he oher hand, a hgher level of negraon requre undoubedly bgger effor n he defnon of nerface and mechanm able o uppor he neceary exchange of daa and gnallng beween he dfferen rado acce nework. Therefore, more ophcaed RRM mechanm (e.g. aware of he dfferen reource of each avalable RAN, of he dfferen offered ervce, ec.) are needed o allow a fruful yem ner-workng. Th ophcaed RRM mechanm, called equally Common RRM (CRRM) [64], Advanced RRM (ARRM), Jon RRM (JRRM) [1] or Mul-RRM (MRRM), provde a oluon o ranform overlappng rado acce nework no a jonly, effcenly co-ordnaed, pool of rado reource. Thee CRRM algorhm nclude ner-yem mobly mechanm, admon conrol, load conrol and packe chedulng. CRRM eny hall have nformaon on he ae of he rado reource n each RAN. For example, he meauremen uch a load nformaon of he neghbourng cell are an mporan npu of he CRRM algorhm and mu be colleced from he dfferen RRM n each RAN GERAN UTRAN ner-workng Inerconnecon beween UTRAN and GERAN ha recenly been preened by 3GPP n echncal repor [58] [59] and he handover from an UMTS o a GSM/GPRS nework can be rggered even f h handover lef o vendor' mplemenaon. In releae'99, procedure for ner-yem handover have been defned, bu hey could reul n a falure due o hgh load n he arge cell. Th uaon ha movaed 3GPP andardzaon o nroduce cell load nformaon exchange beween yem n Releae'5. The noon of CRRM wa nroduced n 3GPP n 2001 and a echncal repor ha been eded [58], collecng propoal and mulaon from dfferen compane. The Releae'5 work reuled n he nroducon n GSM and UMTS of he pobly o exchange cell load nformaon beween RNC and BSC. In [58], wo archecure for mappng CRRM logcal funconaly no phycal ene have been propoed. The fr he cenralzed approach (CRRM erver approach) and he econd he decenralzed cheme (negraed CRRM approach), where CRRM can be mplemened n each RNC and BSC equpmen. CRRM erver approach: h approach mplemen RRM and CRRM ene no eparae node, CRRM a and-alone erver. The CRRM erver (Fg 2. 5) plo local RRM ene mplemened n he RNC and BSC. Conequenly, Each RRM eny n he funconal model may be requeed by reponble CRRM eny o repor ceran nformaon (e.g. meauremen) wh repec o rado reource. Then, CRMS (Common Reource Managemen Server) gaher meauremen from cell under coverage. For each pecfc operaon (handover, cell 22

41 Auo-unng n moble communcaon: Relaed work change order, ec.), he RNC/BSC end o he CRMS he l of canddae cell, ncludng he moble meauremen for hee cell and nformaon abou he QoS requred by he moble. Th funcon hall allow for requeng mmedae reple o a meauremen reque a well a evenor mer-rggered meauremen reporng. I aumed ha h reporng conrolled by he reponble CRRM erver. The CRMS, afer applyng ome algorhm, reurn he prored l of canddae cell. CRRM CRRM Server RRM RRM RNC/BSC RRM RNC/BSC RNC/BSC Fgure 2.5. Cenralzed CRRM eny [58]. Inegraed CRRM approach: h approach negrae he CRRM funconaly no he exng UTRAN/GERAN node: each UTRAN RNC and GERAN BSC equpped wh a CRRM and local RRM ene (Fg 2. 6). The funconal nerface beween RRM and CRRM no realed a an open nerface n h oluon. Only "Reporng Informaon" exchanged over open nerface. The Iur and he propoed Iur-g (beween BSC and RNC) already nclude almo all he requred ngreden o uppor he CRRM funconaly. The man benef of h negraed CRRM oluon ha wh lmed change and already exng funconaly poble o acheve opmal yem performance. The co-locaon of CRRM and RRM n only one equpmen doe no nfluence he decon of he local RRM. CRRM no uppoed o be conuled for channel wchng or for of handover for example, nce he RRM n RNC hall handle hee cae. CRRM wll be ued only for ner-yem mobly ncludng elecon, reelecon and handover. CRRM RRM CRRM RRM RNC/BSC CRRM RNC/BSC RRM RNC/BSC Fgure 2.6. Decenralzed CRRM no every RNC/BSC [58]. 23

42 Auo-unng n moble communcaon: Relaed work GPP-WLAN ner-workng Ineroperably beween 3GPP yem (UTRAN and GERAN) and WLAN nework an eenal requremen o acheve a eamle a well a effcen moble envronmen acro hee dfferen acce echnologe. The WLAN may be an negral par of he UMTS nework where one operaor conrol boh WLAN and UMTS nework. Th refer o he Tgh Couplng and Very Tgh Couplng preened n [64]. In he oher hand yem can be eparae,.e., an ndependen WLAN nerconneced o he nework of one UMTS operaor or hared by mulple operaor. Reference [63] nvegae dfferen nerconnecon level beween 3G cellular nework and WLAN. The neracon level or cenaro, a defned by 3GPP n he echncal repor [60], pan from he mple common bllng o he eamle ervce connuy when movng from he 3G acce nework o he WLAN acce nework and vce-vera. WLAN- 3GPP yem nerconnecon defned a a wrele IP connecvy ervce where he uer can oban acce va a Wrele LAN echnology. The nerworkng mu be ndependen of he underlyng WLAN rado echnologe. Noe ha n comparon o he 3GPP work developed o normalze he neroperably beween RRM of GERAN and UTRAN, here ll no andard eny devoed o manage RRM when connecng UTRAN and WLAN. The fr neracon level, preened n [60], am ju a unfyng bllng and cuomer care procedure for uer n 3G and WLAN nework,.e., a ngle cuomer relaonhp. I doe no requre any parcular neroperably requremen a rado reource managemen. The econd level focue on he 3GPP yem baed acce conrol and chargng. I nclude auhencaon, auhorzaon and accounng (AAA) procedure provded by he 3GPP yem for WLAN uer n 3GPP nework. Th allow a drec acce o nerne from he WLAN acce nework wh an auhencaon baed on he moble operaor' nfrarucure. The hrd level of 3GPP propoal offer o WLAN uer he acce o all he packe-wchng baed ervce provded by 3G yem. Thee level are nvegaed by 3GPP n [61] where a pecal aenon o bac UMTS AAA and chargng ue addreed. The nework elecon procedure for WLAN nework and for 3GPP nework are dealed n [61] and [62]. The name of he WLAN nework provded n WLAN beacon gnal n o-called SSID (Servce Se ID) nformaon elemen. There are wo mode n WLAN nework elecon: Manual mode and auomac mode. In he manual mode, he ermnal hall ry o fnd all avalable SSID hrough cannng. Once a l of all avalable SSID ha been obaned, hall be poble for he ermnal o oban a l of all avalable WLAN name from each SSID and hall preen hem o he uer o elec one. In he auomac mode, afer he cannng procedure, he elecon of he WLAN done auomacally accordng o predefned crera, for example baed on preference l. A of UMTS elecon, he ermnal can ue procedure mlar o exng 3GPP procedure for nework elecon. Regardng he la wo level,.e. level 4 and 5, hey are no condered n [61] and refer repecvely o he gh and very gh couplng propoed by 3GPP. Level 4 allow ervce connuy when uer change he acce beween 3G and WLAN nework. In level 5, eamle ervce connuy uppored when uer move from 3G nework o WLAN and vce-vera. Many open ue o be condered n level 4 and 5 uch a he crera and he decon 24

43 Auo-unng n moble communcaon: Relaed work mechanm for he change of he acce nework, he change of he offered QoS ha can occur, he eamle mobly managemen, ec Leraure revew of auo-unng n mul-yem nework Several ude n he leraure have propoed an effcen and nellgen JRRM n mulyem' conex. Snce he JRRM algorhm mu decde whch RAT every uer n he nework aached o a a gven me, mo of he propoed ude formulae he JRRM a a fuzzy decon, where engneer experence aken no accoun, or/and mul-objecve decon problem whch can be olved wh echnque from he Mulple Crera Decon Makng (MCDM) feld [65] [66] [67]. On he oher hand, he propoed mehod manly dffer n whch pernen yem ndcaor are ued o make a good decon. In [68] and [69], fuzzy layer elecon algorhm are preened o decde o whch nework a uer can be handed over. In a vercal handover, he JRRM agn uer o layer n a yem conng of macro-cell and mcro-cell baed on nework ndcaor. For h purpoe, nework meauremen (.e. moble peed, layer occupancy) are fed o a fuzzfer, where a value beween 0 and 1 agned, correpondng o he degree of memberhp o gven fuzzy e [53]. A fuzzy e a lnguc repreenaon of he npu varable (e.g. hgh, low). Subequenly, an nference engne make ue of predefned fuzzy rule o ndcae, for each RAT, he uably of elecng. Thu, he decon proce baed on heurc decon rule exraced from prevou experence n h ype of nework. The man advanage of fuzzy logc over clacal rule-baed exper yem ha, durng he nference proce, everal rule can be fred n parallel, whch he key for approxmae reaonng. A mlar approach baed on heurc decon rule appled n [70] o olve he ner-workng beween WLAN and 3GPP nework. In [71] and [72], he prevou approach exended wh MCDM echnque n order o combne nework meauremen, operaor polce and uer preference n he JRRM decon proce. A vercal handover algorhm preened, where boh handover naon and handover decon procee rely on fuzzy logc. Durng he naon age, he need for a handover decon evaluaed baed on he heurc rule redng n a fuzzy nference yem. Alhough h rggerng mechanm ha been radonally baed on hrehold crong, more refned mechanm can ake advanage of exper knowledge by mean of fuzzy nference. Thu, nework performance ndcaor from each connecon (e.g. RSSI, BER/BLER, nework coverage and perceved QoS) are monored o deec abnormal behavour ha requre urgen acon. Once he handover decon proce rggered, he dfferen RAT are compared n erm of dfferen crera. For h purpoe, addonal nformaon from he canddae nework (e.g. RSSI and avalable bandwdh) and he uer (e.g. uer peed, baery au) mu be rereved. The acual value of hee performance ndcaor are fed o a fuzzfer, whoe oupu a memberhp value proporonal o he degree of fulflmen of he correpondng creron for each RAT. A decon marx hu obaned, where all he canddae oluon are ranked baed on he core for every creron. To buld a unque overall core for each RAT, he MaxMn aggregaon mehod [76] ued. In h mehod, he memberhp value agned o each RAT repreen he mnmum degree of fulflmen among crera. The RAT wh he maxmum memberhp value fnally eleced. 25

44 Auo-unng n moble communcaon: Relaed work In [73], a mlar approach adoped, where only hree crera relaed o he nework, he uer and he operaor preference are condered for he handover decon. To exrac he nework core for each RAT, everal nework performance ndcaor (.e. RSSI, reource avalably and moble peed) are aggregaed by fuzzy heurc rule. Thu, a unque ndcaor of adequacy from nework perpecve raed for every RAT. To buld a fnal core for each RAT, he reulng value from h nework creron are laer combned wh operaor and uer crera. An addonal dfference he naon of he handover decon proce a a perodc even. In he mer-baed cae, handover rggered when a mer expre. The laency of h mechanm ha radonally favoured ue for alernave (.e. non-urgen) handover, whch do no have rngen delay conran. Hence, h mechanm uable o acheve beer overall nework performance (e.g. hrough load balancng beween layer) or afy uer preference (e.g. QoSco rade-off). Nonehele, mer mgh be ued ndncvely o rgger mperave or alernave handover acon, a he cae, a long a he mer nalzed o a mall value (e.g. 100m). Smulaon reul n a cenaro wh hree concenrc cell prove he robune of he mehod. However, he compuaonal load mgh well be an ue n a real envronmen, nce he handover decon proce hould be performed every me for every uer n he nework. In [74], he prevou model convered no a neuro-fuzzy decon model o ake advanage of learnng capably of neural nework. The parameer of he fuzzy model (e.g. mean, devaon and hape of he memberhp funcon) are adjued by he back propagaon mehod [77]. The objecve of h adapaon proce o mnmze ome overall nework performance ndcaor, uch a he rao of non-afed uer (.e. rao of uer ha receve a bandwdh below a ceran dered value). An error meauremen defned a he dfference wh he arge performance value. The graden of h error repec o he nework' modfable wegh calculaed. Fnally, h graden ued n a mple graden decen algorhm o fnd wegh ha mnmze he error. In [75], underlned ha he ue of fuzzy logc n prevou work no o deal wh mprece nformaon, bu o eae he ue of clacal (.e. no fuzzy) MCDM mehod. Fuzzy e have been radonally ued only o model he flexbly of of conran. Thu, he memberhp funcon n he fuzzfcaon age are employed o rae a core from crp meauremen value o repreen he degree of afacon of he dfferen crera. The reulng crp value are ubequenly ued by clacal MCDM o deermne he rankng order of he alernave, a uggeed n [76]. Followng h approach, he performance of he hree popular MCDM mehod evaluaed n he conex of vercal handover. The mehod condered are he mple addve weghng mehod [65], he MaxMn mehod [76] and he echnque for order preference by mlary o deal oluon [65]. Reul how ha he mple addve weghng mehod provde a relave conervave rankng, le enve o uer preference and arbue value. Lkewe, argued ha MaxMn mehod gve dpuable reul, nce only ue a mall par of he nformaon of he decon marx. 26

45 Auo-unng n moble communcaon: Relaed work 2.5 Concluon Th chaper provde an overvew of curren yem wh pecal empha on RRM algorhm and auo-unng of relaed parameer. A he begnnng of he chaper, GSM nfrarucure and RRM algorhm are gven. For each RRM algorhm, he relaed parameer ha can be auouned are hghlghed. A leraure urvey on GSM auo-unng gven for admon and congeon conrol and mobly managemen. Lkewe, he UMTS archecure and evoluon are gven n he econd econ baed on 3GPP pecfcaon. RRM algorhm and relaed parameer are preened a well. Achevemen and curren ude of UMTS RRM auo-unng are preened llurang he benef of elf-unng for mprovng he nework capacy and QoS. In he la econ, ner-yem cooperaon and JRRM algorhm are preened. A pecal focu gven o ner-connexon mechanm beween GSM and UMTS on he one hand and beween 3GPP yem and WLAN on he oher hand. In he laer cae, prelmnary ude have been conduced n 3GPP wh dfferen nerconnecon cenaro. New reearch avenue of neryem mobly and admon conrol are ummarzed. Mo of hee reearche formulae JRRM algorhm a mul-objecve decon problem ha can be olved by mple (no fuzzy) or fuzzy MCDM mehod. Many apec n auo-unng echnque and applcaon reman unexplored and have movaed he preen he. 27

46 Auo-unng archecure and ool 3 Chap. 3 Auo-unng archecure and ool 3.1 Inroducon In auo-unng and elf confguraon conex, he core challenge we face how o degn an effcen archecure eay o be mplemened n real nework and wha ool can be choen o opmze he auo-unng ak. The auo-unng funconale hould be merged wh he nework elemen accordng o wheher a long erm or hor em auo-unng requred. The mple way for auo-unng mplemenaon o add a new layer n he Operaon and Manenance Cenre (OMC) of he nework; however h oluon can only uppor long erm or off-lne auo-unng [27]. Bede, ool ued n he elf-confguraon hould be baed on heurc and opmal mehod characerzed by hgh level of nellgence. We ue n h he, Fuzzy Logc Conroller (FLC) a open loop regulaor of RRM parameer. The fuzzy logc heory ranlae experence from rado engneer no a mple e of rule whch are no alway opmally developed. To opmze FLC, we ue Renforcemen Learnng (RL) algorhm a a dynamc FLC-opmzng engne. In he preen chaper, we ar by nroducng he elf-unng archecure concep a propoed by he Eureka Celc Gandalf projec [1]. Boh on-lne and off-lne auo-unng approache are dcued. The nex econ dcu he fuzzy logc conroller and he renforcemen learnng algorhm and more pecfcally he fuzzy Q-learnng algorhm [4]. 3.2 Auo-unng archecure Gandalf managemen archecure The archecure propoed by he Gandalf projec concern he ue of auo-unng and roublehoong [1] [83] n advanced- (for a dnc Rado Acce Nework (RAN)) and jon- (for wo or more RAN) RRM [1]. Fgure 3.1 how he dfferen managemen ak rangng from he off-lne o he onlne auo-unng accordng o he correpondng me cale. In he long erm opmzaon, we have he off-lne auo-unng and he roublehoong wherea for he hor me cale, dynamc auo-unng merged wh he RRM and JRRM funconale. Nework off-lne auo-unng are long erm managemen ak, wh me cale varyng from a day o monh, n whch opmal fxed nework parameer value and hrehold are derved and manually njeced n he nework. Dagno and roublehoong are alo long erm managemen ak and ued o deec faul and ub-opmal parameer va alarm and monorng of couner and Key Performance Indcaor (KPI), ofen cenralzed n he Nework Managemen Syem (NMS). Th funcon can be performed on daly ba or longer me perod. The onlne auo-unng ak hould be nananeou, ypcally of he order of mnue, econd or le. To faclae he nformaon exchange beween RRM funconale and he on-lne auo- 28

47 Auo-unng archecure and ool unng managemen, he auo-unng hould be cloe o he RRM funcon. In oher word, mu lnk he managemen and he conrol plane Gandalf auo-unng archecure In [1], he Gandalf projec propoe an auo-unng funconal archecure. Fgure 3.2 llurae a modfed and mplfed veron of he archecure propoed n [1], where only auo-unng par aken no accoun. The propoed archecure, n eence, compre everal bloc: RRM Daa Collecor, Managemen and Proceng Module (MPM), he JRRM and RRM funcon, and he Auo-unng and Opmzaon Engne (AOE). Managemen Plane Parameerzaon Off-lne opmzaon Troublehoong Long-erm me cale Conrol Plane J-RRM On lne opmzaon RRM Shor-erm me cale Inananeou me cale Uer Plane RAN Mul-echnology Rado Acce Nework Fgure 3.1. Nework managemen ak wh he correpondng me cale [1]. The RRM Daa Collecor collec meauremen concernng he performance of each RAN. Daa he KPI delvered by he RNC or anoher equvalen eny. Example of KPI could be: raffc blockng, reource avalably and acce, handover ucce and falure, recever level qualy, voce call qualy, packe call qualy, dropped call rae... The crude daa njeced n he MPM. The MPM expeced o handle boh mul-echnology and mul-vendor envronmen. MPM harmonze and render ranparency he KPI obaned from dfferen echnologe and dfferen vendor. A menoned n he prevou chaper, he JRRM elemen reponble for he global, mul- RAN rado reource managemen, and am a mprovng he qualy of ervce n he global nework. The JRRM dffer from he RRM, nce explo he oal avalable heerogeneou nfrarucure o provde effcen RRM. Whenever he RRM eny unable o reconfgure he 29

48 Auo-unng archecure and ool yem n uch a way o olve a problem, he KPI of each RAN can be dreced o a hgher level, and he problem may be reaed and olved n JRRM or n he AOE. Indeed, he JRRM can reconfgure RRM parameer ha wll beer ue he nework need, or ake a decon uch a ner-ran vercal handover, elecon and reelecon a well a roamng of uer from one ervce provder o anoher. JRRM enable o reconfgure RRM parameer whenever an auounng engne drecly mplemened n he JRRM eny or he laer receve auo-unng command from he AOE. The AOE con n defnng parameer arge o be acheved baed on he avalable KPI. The man parameer ha are fne-uned are gnallng parameer, RRM or JRRM parameer. The auo-uned parameer are perodcally njeced n he JRRM or RRM elemen. The onlne auounng proce can be condered a a par of he RRM or JRRM funconale. In h cae, RRM called advanced RRM [1] [85]. Furher deal of he AOE are found n econ Auo-unng & Opmaon Engne Managemen Plane J-RRM RRM Re-Confguraon Daa managemen & proceng Daa Collecor Supplemenary Servce Performance Daa Conrol Plane RRM Conrol J-RRM Daa Collecor RRM KPI-PU RRM KPI-PU RRM KPI-PU RNCF RNCF RNCF Uer Plane RAN 1 RAN 2 Mul-echnology Rado Acce Nework RAN 3 Sgnallng Aocaon OAM Aocaon RNCF: Rado Nework Conrol Funcon Fgure 3.2. Auo-unng archecure n uer, conrol and managemen plane. Daa en from he upper level o he AOE hrough he OAM (Operaon Admnraon and Manenance) lnk f off-lne auo-unng ued becaue requre a hgh me o be exchanged. To collec more nework daa, upplemenary ervce performance daa elemen can be added. Thee allow complemenng couner and KPI mplemened by vendor, and can be ued o e, valdae or monor par of he nework. There ex many capure ool whch are ued o gve more nformaon on he nework performance and o defne pecfc KPI needed by he operaor. Such ool are Vallen, Sunre, Cgale and Term nvegaon [84]. 30

49 Auo-unng archecure and ool In he cae of onlne auo-unng, gnallng meage are en beween ene. In [85], example of gnallng meage and auo-unng nformaon flow are preened Auo-unng nformaon flow An example of auo-unng meage equence char (MSC) preened here baed on [85]. The MSC expree how gnallng meage could be en beween ene n cae of auo-unng mplemened. The MSC depced n fgure 3.3 and decrbed by 7 ep: AOE Daa collecor Monorng RRM & J-RRM 1. KPI reque KPI en 2. Servce arge KPI reque Servce arge KPI en 3. Reque for curren J-RRM parameer eng Sen curren J-RRM parameer eng May requre J-RRM o RNCF-RRM confguraon daa reque() Proceng KPI and calculaon of new parameer eng J-RRM parameer eng change order 6. Parameer eng change acknowledge 7. May requre J-RRM o RNCF-RRM parameer eng change order() Fgure 3.3. Sgnallng meage beween AOE and JRRM/RRM module. Sep (1): The AOE reque and receve KPI from he daa collecor module. Sep (2): The AOE reque and receve ner yem ervce arge ndcaor from he monorng module, uch a arge blockng and droppng rae n each yem. Thee ndcaor ogeher wh hoe from he daa collecor module are ued o gude he opmzaon proce. Sep (3): The AOE check he curren parameer eng of he J-RRM o guaranee coherence of he propoed modfcaon (n ep (4-5)). Sep (4) (oponal): I may be neceary for he J-RRM o reque curren RRM parameer value from he nework elemen. 31

50 Auo-unng archecure and ool Sep (5): KPI are proceed and he new RRM parameer are e by he fuzzy Q-learnng conroller nde he AOE. To allow learnng, new funcon dependng on he conroller may be bul and ored n a look-up able (ee nex econ). Sep (6): Modfcaon order of RRM or JRRM parameer eng ranmed o RRM module, and an acknowledge meage en back. Sep (7): The JRRM module map he parameer change requeed by he AOE no parameer change of RRM for each RAN Auo-unng and opmzaon engne The AOE preened by he mplfed fgure 3.4. I chefly conan 4 block: daa proceng, Fuzzy Logc Conroller (FLC), Q-learnng algorhm and a look-up able. The fr par erve a he nermedary beween he nework and he AOE. In order o ablze he auo-unng proce, he daa proceng module hould fler KPI, receved from he nework (from any RRM or JRRM module), over a flerng perod. In fac, averagng KPI allow moohng ou random flucuaon. I make acon or decon of he conroller o be baed on underlyng rend and no nananeou change. Mahemacally, he flerng funcon of any receved qualy ndcaor x made by averagng he nananeou value of he ndcaor x durng a dcree me wndow T f : T 1 ( ) f x 1 Fl x( ) (3.1) T f 0 The daa proceng module allow alo defnng an objecve funcon from he receved KPI. The objecve funcon, called alo reward gnal, hould be maxmzed by he Q-learnng algorhm. The objecve funcon obaned by makng operaon beween receved KPI. The FLC reponble for dynamcally adapng RRM or JRRM parameer accordng o receved qualy ndcaor. The adapaon performed opmally hank o he Q-learnng algorhm. The combnaon of boh FLC and Q-learnng algorhm jufe he name Fuzzy-Q- Learnng Conroller (FQLC). FLC and Q-learnng algorhm are furher gven n nex econ. The fourh par he look-up able, ued a he memory of he FQLC. A predefned yem ae e and correpondng be parameerzaon of he nework are ored n he look-up able. Th laer perodcally updaed durng he learnng proce of he conroller. 32

51 Auo-unng archecure and ool Auo-unng and Opmzaon Engne Qualy ndcaor Proceng daa Q-Learnng algorhm Fuzzy Logc conroller Look up able Performed acon Fgure 3.4. Auo-unng and Opmzaon Engne 3.3 Fuzzy logc conroller Fuzzy Logc Conroller (FLC) mlar o he convenonal conroller n he ene ha mu addre he ame ue common o any conrol problem, uch a yem ably and performance [86]. However here a fundamenal dfference beween FLC and convenonal conroller n erm of yem modellng. Convenonal conroller ar wh a mahemacal model of he yem and conroller are degned baed on he model. FLC, on he oher hand, ar wh heurc and human expere (n erm of fuzzy f-hen rule) and conroller are degned by ynhezng hee rule. Th exremely mporan n moble wrele communcaon becaue almo mpoble o oban an accurae ye mple mahemacal model of he yem. FLC ha been found parcularly well ued for parameer auo-unng n rado acce nework [46] [47]. Th due o mplcy and ably o conver knowledge from experence no a e of f-hen rule, whch mmc he reaonng of he operaor (nework exper). I ealy underood ha he nework operaor ha a crcal role n he fr ep of he FLC degn. A a reul, an exper yem obaned, whch able o map performance ndcaor value no conrol acon, a an operaor would do. To olve he problem aocaed o he eparae rggerng of rule, a fuzzy nference engne ncorporaed no he yem. FLC and hen for he proce of formulang he mappng from a gven npu o an oupu ung fuzzy logc heory [86] [87]. Fuzzy logc heory nvenor pl he fuzzy conrol proce no hree phae a lluraed n fgure 3.5. The fr phae he fuzzfcaon ep and con of converng he crp npu daa o fuzzy daa e. Th mappng proce nvolve fndng he degree of memberhp of he crp npu n predefned fuzzy e. The econd phae he nference proce and con of makng 33

52 Auo-unng archecure and ool decon from he "f hen" rule by combnng all fuzzy npu e. The la phae he defuzzfcaon whch map he fuzzy oupu o he fnal conrolled crp parameer [87]. Inpu crp value (QoS ndcaor) S M H VH (fuzzy value) Fuzzfcaon Droppng rae M Blockng (fuzzy value) Inference Defuzzfcaon Droppng H C 22 C 23 C 32 C 33 New parameer eng (connuou value) Correcon 0.8*0.3*C *0.7*C *0.3* C *0.7*C 33 Fgure 3.5. The concep of fuzzy logc conroller. In he leraure, here are bacally wo FLC ype: Mamdan and Takag-Sugeno approach [78] [79]. The man dfference beween Mamdan and Sugeno he oupu memberhp funcon. Mamdan' fuzzy nference approach [78] ue a ngleon oupu memberhp funcon n he defuzzfcaon proce whch mplfe he compuaon. By conra, Takag-Sugeno ype [79] ue a more complex oupu memberhp funcon for exended flexbly a he expene of compuaon load. In h deraon, we ue FLC baed on Takag-Sugeno approach a depced n fgure Mahemacal framework of FLC In he fr phae of Takag-Sugeno baed FLC, each crp npu varable x (1, 2, n) mapped no m fuzzy varable (or e), denoed E (j 1, 2, m ). Th procedure allow mappng a connuou ae pace no a dcree pace. Noe ha we can dvde each varable o exacly m fuzzy e; n h cae m,. The ndex can alo be omed from j n E. The m mappng beween he crp and he fuzzy varable made by a memberhp funcon ( ) j x µ whch defne he memberhp degree of he crp npu x wh he fuzzy varable E j. In our work we ue he rangular memberhp funcon, defned a j j 34

53 Auo-unng archecure and ool Ej x 1, f Ej 1 x Ej Ej Ej 1 Ej x µ j ( x ) 1 +, f Ej x Ej+ 1 ; { 1,2,... n} and j { 1,2,... m} (3.2) Ej+ 1 Ej 0, ele However, oher funcon omeme are employed when hgher accuracy or nonlneary requred uch a rapezod or Gauan funcon. Layer 1: Fuzzfcaon Layer 2: Inference rule Layer 3: Defuzzfcaon E 11 1 x 1 o 1 2 a 1 E 1m o 2 E n1 a p x n o K E nm K Fgure 3.6. Fuzzy logc conroller baed on Takag-Sugeno approach. Recall ha n he nference proce, "f-hen" rule are conruced baed on nework exper. The rule are he realzaon of he npu varable and he correpondng fuzzy conequence. A hown n fgure 3.6, he nference proce map he fuzzy e no a e of rule. Defnon 3.1 A predcae called an nference rule f ha he form: If x 1 E1 j and x2 E j... and x Ej... and xn Enj hen a o ; j { 1,2 m 1 2 2,... n The aemen x 1 E1 j and x E j... and x Ej... and x n E nj n and for he rule. Rule form a pace e denoed S and each rule denoed by S. The cardnal of R j1 j2... j n he pace S equal o K m 1 n whch reduced o m n when m m,. a he crp FLC oupu varable and o fuzzy realzaon n he rule R j j j n } 35

54 Auo-unng archecure and ool The rule R j j j n ha a memberhp funcon deduced from hoe of E j [79]: α n j j j µ n j 1 ( x) µ ( x) ( x ) (3.3) In he defuzzfcaon phae, he FLC oupu a correpondng o he npu x (x 1... x x n ) gven by he gravy cenre of concluon o n each rule weghed by he memberhp funcon α ( x). If he member funcon α are choen o afy he normalzed condon..e. α x, S ( ) 1 he oupu acon a : a S ( x) α (3.4) o Example: ue-cae of FLC Now ha we have aken a broad look a how FLC generally degned, le u ee how we can mply ue n a mple example of auo-unng UMTS admon conrol parameer. We wan o adju he admon conrol hrehold n an UMTS nework accordng o he oberved blockng and droppng rae. Wha we now am o do ue blockng and droppng rae a npu o he FLC o deermne he approprae load arge hrehold. Th a very mple example, a here are only wo npu varable and only one oupu. In he fuzzfcaon phae, we can e up, accordng o equaon (3.2), he memberhp funcon and fuzzy e a lluraed n fgure 3.7. For each npu varable, we have hree fuzzy e: low, medum and hgh. Fgure 3.7. Fuzzy e and menberhp funcon of he droppng and blockng rae. Blockng Low Medum Hgh Droppng Low Do nohng ncreae ncreae Medum Decreae Do nohng ncreae Hgh Decreae decreae decreae Table 3.1. Fuzzy rule. 36

55 Auo-unng archecure and ool Once he fuzzfcaon ep compleed, we move o he conrucon of he f-hen rule. Table 3.1 llurae uch rule. Each elemen of he able repreen he degree of modfcaon of he conrolled npu (load arge hrehold). A we have n2 varable, each of whch ha m3 fuzzy e, he number of rule hen K9. The oupu fuzzy acon n h example can be ncreae, decreae he admon conrol hrehold, or do nohng. One can e up everal more rule o handle more poble by addng for example he acon 'more ncreae', 'more decreae'... To how he defuzzfcaon proce, aume ha a ome gven me, he FLC receve from he UMTS nework a blockng rae equal o 7% and a droppng rae equal o 2.7%. Accordng o he predefned fuzzy e and memberhp funcon, he FLC nerpre he blockng rae a low wh 0.6 of ruh and medum wh 0.4 degree. On he oher hand, he droppng rae medum wh 0.3 degree and hgh wh 0.7 degree. If we agn value o he acon uch a -0.1 for ncreae, 0 for do nohng, and +0.1 for decreae, we mgh have he followng oupu a a correcon for he admon conrol hrehold: Crp acon (-0.1)*(0.6)*(0.3) + (0)*(0.4)*(0.3) + (-0.1)*(0.6)*(0.7) + (-0.1)*(0.4)*(0.7) Aume ha he recen admon conrol hrehold 0.75, n he nex me he bae aon wll be confgured auomacally by adjung admon conrol hrehold o Wha we have aken now a mple example wh only nne rule. Of coure, many oher parameer could be uned uch a he reerved bandwdh menoned n he prevou chaper, maxmum allowed b rae for ome admed moble (he cae of elac daa raffc), and handover parameer ec. Therefore, everal npu and oupu parameer mgh ceranly generae o many rule and hen he FLC degn become very complex. In addon, yem wh varyng condon can grealy benef from he adapaon of he conroller parameer o deal wh uch varaon. A cellular nework one example of uch dynamc yem, nce he raffc demand grealy vare n he paal and emporal doman. Thu, he conroller can be adapve for he ake of effcacy,.e. rule and parameer n he conroller can be auomacally and opmally modfed when he nework experence ubanal change n order o keep he performance merc a hgh a poble [80] [81]. If here are avalable daa abou nework behavour and performance, modfcaon of uch rule and parameer can be accomplhed by elf-learnng algorhm. Th way, auo-unng can combne wo powerful ool: he ably o expre mple opmzaon rule ung experence and know-how of operaor (FLC) and he ably o apply compuaonally-nenve elf-learnng mehod o opme conroller parameer baed on exng and fuure nework performance daa (renforcemen learnng). 3.4 Renforcemen learnng General vew of machne learnng Machne learnng a feld of arfcal nellgence (or auomac learnng) ha deal wh he degn and developmen of algorhm and echnque ha allow compuer or any oher agen o 37

56 Auo-unng archecure and ool learn and mprove performance baed on prevou reul. The machne can nduce paern, regulare, or rule from pa experence. There are hree ype of learnng: uperved, unuperved and em-uperved learnng. From a heorecal pon of vew, uperved and unuperved learnng dffer only n he caual rucure of he model. In uperved learnng, he learnng algorhm generae a funcon ha map npu o dered oupu. One andard formulaon of he uperved learnng ak he approxmaon problem: he learner requred o learn (o approxmae) he behavour of a funcon whch map npu no one or everal oupu by lookng a everal npu-oupu example of he funcon. For a yem ung uperved learnng, a eacher mu help he yem n model conrucon by defnng npu and provdng her label. In conra, n unuperved learnng, no eacher help he learner. Thu, he learner elf mu underand and dcover relaonhp beween daa componen. The em-uperved learnng beween uperved and unuperved learnng. The learner n h cae aed ndrecly by a eacher va he reward receved for each couple of npuoupu. The Renforcemen Learnng (RL) a a knd of em-uperved learnng lghly comparable o he human learnng [88]. Durng he human lfe, many problem could be me and he reoluon of ome one creae human reflex and knowledge abou ome dangerou uaon whch requre more aenon. RL con o each an agen ha ome decon are good and ome oher are bad. Gvng a reward when he agen doe omehng good and a punhmen when doe omehng bad allow o egregae he good from he bad and recognze harmful decon. Therefore, he agen could develop beer way o make good decon Mahemacal framework of RL The fundamenal purpoe of RL o mprove a curren agen polcy afer each neracon wh he envronmen. In h cae he renforcemen local and herefore doe no gve a complee evaluaon of he agen polcy. In fac, RL algorhm do no ue drecly he polcy bu evaluae he performance of he raegy va a e of value funcon reulng from he heory of Markovan Decon Proce (MDP) [89]. A MDP a conrolled ochac proce mlar o Markov chan, excep ha he ranon probably depend on he acon aken by he decon maker (agen or conroller) a each me ep. The MDP formulaed by he qunuple (S, A, T, p, r); where: S a ae pace ha conan a fne number of ae, A a fne e of acon. We denoe by A( ) A hoe acon ha are avalable a ae, T he me. T a ub-e of pove real number, p are he ranon probable beween ae, r he renforcemen funcon or reward dependng on ae and acon. A hown n fgure 3.8, a each me of T, he agen oberve he curren ae S and perform an acon a A ha hf he yem o anoher ae S wh a p, a. One ep laer, he agen receve a reward (, a) IR. probably ( ) r 38

57 Auo-unng archecure and ool a 1 22 a 2 3 a 1 a 3 a a 1 a 3 Fgure 3.8. Example of Markovan Decon Proce. Defnon 3.2: MDP proce Le h, a,...,, a, ) be he proce hory oberved a me. ( The proce (S, A, T, p, r) called an MDP proce f he ranon probably beween and +1, gven he performed acon a, depend only on. ( h, a ) P(, a ) p ( a ) h, (3.5), a, + 1 P Th doe no necearly mean ha he ochac proce ( ) elf a Markovan proce. I depend on he agen polcy. Defnon 3.3: Agen polce The agen polcy mplemen a mappng from ae pace and acon pace. RL mehod pecfy how he agen change polcy a a reul of experence. The e of all polce form a pace, Π : S a A. denoed { ( ) } For each polcy, we denoe q ( a, ) h he probably ha n a gven hory h he acon a rggered. Accordng o wheher all he hory nvolved n he agen polcy or no, wo ype of polce are defned: Hory-dependen polce: he probably q ( a, h ) Markov polce: he probably q ( a, ) h depend on he whole hory h. only a funcon of and no of he whole hory. Defnon 3.4: Goal and reward In RL, he purpoe of he agen formalzed n erm of a pecal gnal, called he reward and denoed r ha pae from he envronmen o he agen. The reward ju a ngle number whoe value vare from ep o ep. The agen' goal o maxmze he oal amoun of reward receve, called goal or reurn funcon. Th mean maxmzng no ju mmedae reward, bu cumulave reward n he long run. The ue of a reward gnal o formalze he dea of a goal one of he mo dncve feaure of RL [88]. 39

58 Auo-unng archecure and ool The mo commonly ued reurn funcon, and he one ha wll be ued hroughou h he, he dcouned cumulave fuure reward, expreed a: R + γ r (3.6) 0 where γ a parameer beween 0 e 1, called dcoun facor. The nfne um R ha a fne value a long a he reward equence (r ) bounded. The dcoun facor ued a a meaure ha ndcae he relave mporance of fuure reward. Defnon 3.5: Value funcon Almo all RL algorhm are baed on emang ome value funcon, funcon of ae (or of ae-acon par), ha emae how good for he agen o be n a gven ae (or how good o perform a gven acon n a gven ae). The noon of "how good" here defned n erm of fuure reward ha can be expeced, or, o be prece, n erm of expeced reurn. Of coure, he reward he agen can expec o receve n he fuure depend on wha acon wll ake. Accordngly, value funcon are defned wh repec o parcular polce. So for each polcy, he value funcon V expreed by he expeced reurn funcon:, V : S V ( ) IR For he dcouned cumulave reward of he equaon (3.6), he value funcon : S, V ( ) E [ R ] E [ + γ r / ] (3.7) The operaor E and for he mahemacal expecaon gven he polcy. Le Ω be he pace of all he funcon gong from he ae pace S o he real number IR. We defne n h pace he norm max V Ω, V maxv ( ). The value funcon pace Ω alo a parally ordered e: S U, V Ω U V S U ( ) V ( ) Propoon 3.1 Le be a hory-dependen polcy. For any nal ae x, here ex a Markov polcy ', uch V x V x. ha: ( ) ( ) The proof of h propoon gven n appendx A. Remark ) From he prevou propoon, we deduce ha every hory-dependen polcy can be replaced by a Markovan polcy havng he ame value funcon f he nal ae gven. From now on, we ue only he markovan polcy, unle conrary menoned. 40

59 Auo-unng archecure and ool ) If he polcy markovan, he proce ( ) elf a markovan proce wh a ranon marx P, defned by: ( a, ) p( / a), S P, q a A, (3.8) ) Accordng o he prevou noaon, he value funcon can be expreed a: V + [ / x] ( x) E γ r (, a ) S a A γ r 0 (, a) P (, a a / x) 0 Theorem 3.1 Le a A r be he reward vecor whoe elemen are q ( a, ) r(, a) and of he value funcon) he value vecor whoe elemen are V ( ) equal o he number of ae. The marx expreon of he value funcon V 1 ( I γp ) r The proof of heorem 3.1 gven a well n appendx A. V (he ame noaon. The ze of r and V hen: V (3.9) Defnon 3.6: Opmal value funcon A polcy defned o be beer han or equal o a polcy f expeced reurn greaer han or equal o ha of for all ae. In oher word, f and only f V ( ) V ( ) for all S. There alway a lea one polcy ha beer han or equal o all oher polce. Th an opmal polcy. Alhough here may be more han one, we denoe all he opmal polce by *. They hare he ame value funcon, called he opmal value funcon, denoed V * *, and defned a V ( ) maxv ( ) S. The objecve of RL hen o fnd a polcy * ha correpond o he opmal value funcon. To prove he exence of he opmal value funcon, one can ue he Dynamc Programmng Operaor (DPO). Π Defnon 3.7: DPO The operaor DPO, denoed here L (o refer o he learnng), a mappng over he pace Ω uch ha V Ω S LV ( ) max r(, a) + γ p( /, a) V ( ) (3.10) a A S Wh marx noaon, he prevou expreon 41

60 Auo-unng archecure and ool V Ω LV Π { r + γp V } max (3.11) The exence and he unquene of he opmal value funcon are gven by he followng heorem. Theorem 3.2: Bellman equaon If S and A are fne e, hen V * he unque oluon of he equaon V LV (3.12) The heorem proved n appendx A. To olve he Bellman equaon, here are wo dfferen approache: he fr he polcy eraon and he econd he value eraon. In he polcy eraon, here are wo ep: polcy evaluaon and polcy mprovemen. Each eraon preerve monooncy n erm of he polcy performance. The polcy evaluaon ep oban V for a gven polcy by olvng he correpondng fxed-pon funconal equaon over all S : V ( ) r( ( ) ) + γ p( /, ( ) ) V ( ), (3.13) The polcy mprovemen ep ake a gven polcy and oban a new polcy * ha afe he condon * ( ) arg max r(, a) + γ p( /, a) V ( ), S. a A S The polcy mprovemen ep enure ha he value funcon of * no wore han ha of. Wh repec o he value eraon approach, ofen he mo effcen compuaonal echnque for fndng he opmal value funcon and correpondng polcy. The prncple o updae a gven value funcon by applyng he operaor L uccevely. Snce L a conracon, he oluon of Bellman equaon he lm of he equencev n+1 LVn, for every arng value funconv 0. The runnng-me complexy of value eraon polynomal n S, A, 1/(1 γ); n parcular, one eraon O( A S 2 ) n he ze of he ae and acon pace [89]. A oberved n he expreon of he operaor L, he reoluon of Bellman equaon requre he knowledge of he yem (.e. he ranon probable beween ae gven a performed acon, random reward). However n complex yem, uch wrele communcaon nework, mpoble o know explcly he yem model whch make hard o olve he Bellman equaon. Th challenge ha gven re o a number of approache nended o reul n more racable compuaon for emang he opmal value funcon and fndng opmal or good ubopmal polce. The Q-learnng algorhm, nroduced by Wakn [92], ackle he nexplcly yem model. I condered he mo praccal approach hank o mplcy. A he conroller, S 42

61 Auo-unng archecure and ool ued n h he, baed on fuzzy Q-learnng, we explan n he nex econ he Q-learnng algorhm and fuzzy veron. 3.5 Fuzzy Q-learnng conroller Q-learnng algorhm Q-learnng, perhap he mo well-known example of renforcemen learnng, a ochac approxmaon-baed oluon approach o olvng Bellman equaon. I a model-free approach ha work for he cae n whch he ranon probable and one-age reward funcon are unknown. Inead of ung only one value funcon, Q-learnng algorhm employ anoher value funcon dependng on boh ae and acon, called qualy funcon or Q-funcon. I equal he expreon appearng nde he "max" operaor of equaon (3.10). (, a) r(, a) + γ p( / a) V ( ) Q, S (3.14) Ung he Q-funcon, he operaor L become LV ( ) max Q (, a) a A { }, V Ω, S Moreover, he value eraon approach can be expreed n erm of wo equence { V } and{ Q } conruced from he Q-funcon a and V ( ) LV ( ) max{ Q ( a) } (3.15) + 1, a A (, a) r(, a) + p( /, a) max{ Q ( a )} Q 1 γ, (3.16) + a A S The Q-learnng algorhm a ochac form of he value eraon. From equaon (3.16), underood ha performng a ep of value eraon requre knowng he expeced reward and he ranon probable. Alhough uch a ep canno be performed whou a model, nonehele poble o emae he approprae updae. The erm S ( /, a) max{ Q ( a )} r (, a) + γ p, a A replaced by mple unbaed emae r + max{ Q (, a )} a A γ [91]. So, he ucceor ae an unbaed emae of he um and r an unbaed emae of r (, a). Th reaonng lead o he followng relaxaon algorhm, where we ue Q (, a) o denoe he learner' emae of he Q-funcon a me. Q ( ) ( a ) ( 1 ) Q (, a ) + κ r γ maxq (, ') κ (3.17) + 1, a a' A. 43

62 Auo-unng archecure and ool The varable κ he learnng rae and equal zero excep for he ae ha beng updaed a me. The proof of he convergence of he Q-learnng algorhm wdely uded. I can be found n [91]or [92]. The ame way a n heorem 2, he proof of Q-learnng convergence baed on he conracon mappng heorem (heorem 3). A hown n [91], he convergence acheved under he aumpon ha each ae ved n an nfne number of me and he equence of he learnng raeκ afe he condon 0 2 κ and κ < Adapaon of Q-learnng o fuzzy nference yem The convergence of he Q-learnng algorhm requre ha he ae pace S hould be fne. However, n moble communcaon, we face connuou ae (connuou qualy ndcaor), uch cell load or blockng probably, ha can no be mple npu o he MDP Q-learnng algorhm. To handle connuou npu ndcaor, a mple nerpolaon procedure nroduced n he Q-learnng algorhm ung fuzzy nference yem. A explaned n econ 3.3, he e of connuou npu ndcaor are mapped no a e of rule. Now, nead of applyng he learnng drecly o he npu ndcor, he agen learn on he rule and fuzzy acon. To do h, a fuzzy qualy-value q agned o each rule and each acon. Unlke mple fuzzy nference yem, where only fuzzy acon correpond o a fuzzy rule, n fuzzy Q-learnng, each rule ha A ( ) poble compeng dcree acon{ o }. The R j 1 j2... j n predcae, gven n defnon 3.1, become for each rule k If x R j 1 j2... j n hen a o 1 wh qualy q(,o 1 ) or o 2 wh qualy q(,o 2 ) or o A() wh qualy q(, o A() ) The agen ore he parameer vecor q(,o k ) aocaed wh each of hee ae-acon couple n he look-up able. Thee q-value are updaed whenever he agen perform an acon and he yem v a new crp ae. The value funcon of crp npu x and crp acon a, a me, calculaed a a lnear nerpolaon of he q-value: (, a) ( x). q ( o ) Q x α, (3.18) S ( ) ( x) max q ( o) V x α, (3.19) S o A( ) The um performed over all he rule of he FIS. Here, we ue he noaon o refer rule becaue he e of rule form a dcree ae pace S defned a he ame way n he prevou 44

63 Auo-unng archecure and ool econ. Recall ha he number of fuzzy ae S gven n econ 3.3 by K he eleced fuzzy acon n rule (fuzzy ae). Recall ha ( x) m 1 n. o α he memberhp funcon (gven n equaon 3.3) of rule appled o he crp npu x. The updae of he q-value mlar o he updae of Q-funcon n mple Q-learnng algorhm. So, for fuzzy Q-learnng, he erave equaon (3.17) become q ( o) q (, o) + ( x ) κ ( r + V ( x ) Q ( a) ) + 1, γ + 1 x, α (3.20) Fnally, he fuzzy Q-learnng algorhm for a ochac envronmen (uch moble communcaon yem) gven below. 1. Inalze Q-look-up able: S, o A q (, o) 0 ; Tme 0; Repea: x x x,..., from he yem; 2. Receve he crp yem npu ( ), 1 2 x n 3. Fuzzfcaon: mappng from x o fuzzy ae S ( or rule R j j... j ); 1 2 n 4. For each rule S elec an acon o wh he EEP polcy o arg max q o wh a probably ε ( ) ( ), o A or random{ o o A( ) } o, wh a probably 1-ε 5. Calculae he nferred acon a (equaon (3.4)) 6. Calculae correpondng qualy (equaon (3.18)) 7. Execue he acon a ha lead he yem o he crp ae x + 1. The conroller receve he renforcemen r. 8. Calculae he memberhp funcon α ( x +1 ) for S (equaon (3.3)) 9. Calculae he value of he new ae (equaon (3.19)): 10. Updae he elemenary qualy q(, o) of each rule and acon o A() (equaon (3.20)): 11. Save he elemenary qualy q (, o) n he Q-look-up able. 12. f convergence obaned hen op he(n) learnng proce Fgure 3.9. Fuzzy Q-learnng algorhm. In he algorhm, acon are eleced durng he learnng proce ung an Exploraon/ Exploaon Polcy (EEP) [53]. Th polcy, called alo ε-greedy [82], allow he conroller o explo knowledge hroughou learnng proce. For each rule, he conroller chooe he be acon wh a probably ε and a random acon wh a probably 1-ε. 45

64 Auo-unng archecure and ool In h deraon, ε e o 0.80, he dcoun facor γ fxed o Wh repec o he leanng raeκ, equal o 1 /( 1+ v (, o) ) [54], where v (, o) he oal number of me ha h fuzzy ae-acon par ha been ved before he me. The agen op compleely he learnng proce f he convergence reached. In exploaon phae, he agen (or conroller) chooe only he fuzzy acon ha maxmze he q-value n each rule,.e. o arg max q(, o), S. The ued convergence creron : o A( ) (, o) q (, o) θ max q 1 (3.21) + S, o A The convergence reached when θ become very low (lower han 3.6 Concluon θ c 10 4 for example). In h chaper, he auo-unng archecure ha been gven for boh onlne and off-lne auounng. The requremen for effcen auo-unng archecure are hghlghed by ponng ou he relaon beween nework ene and he auo-unng engne. An example of gnallng meage beween he nework and he AOE ha been decrbed. The AOE ha been nvegaed wh a parcular focu on fuzzy renforcemen learnng mechanm for opmzng he auo-unng proce. A complee and dealed proof of he Q-Learnng algorhm ha been preened, and o our knowledge, doe no ex n he preen form. The Renforcemen Learnng wh he fuzzy Q-learnng mplemenaon ha been decrbed n deph for he degn ak of opmal fuzzy logc conroller. In chaper 4, we wll how how h conroller can be mplemened n UMTS nework o dynamcally adap reource allocaon algorhm and handover parameer. The fuzzy Q-leanng wll be alo ued for adapng ner-yem mobly n Chaper 6. 46

65 Applcaon of auo-unng o he UMTS nework 4 Chap. 4 Applcaon of auo-unng o he UMTS nework 4.1 Inroducon The complexy of UMTS nework and he permanen raffc varaon make dynamc engneerng (or elf-unng) of RRM parameer a promng avenue for mprovng nework performance. The man objecve of h chaper o how how echnque preened n he prevou chaper can dynamcally perform parameer eng of UMTS nework a a funcon of raffc flucuaon. The opmal eng ue fuzzy Q-learnng conroller whch combne boh fuzzy logc conroller and Q-learnng algorhm. The combnaon of hee wo echnque mplemened, a decrbed n chaper 3, o une exacly wo RRM algorhm. The fr he dynamc reource allocaon beween Real Tme (RT) and Non-Real Tme (NRT) ervce. The econd cae he auomac eng of of handover algorhm parameer namely Hyere_even1A and Hyere_even1B, accordng o 3GPP pecfcaon [19] [24]. A preened prevouly, he Fuzzy Q-Learnng Conroller (FQLC) need a vecor of qualy ndcaor a conroller npu and delver correcon for RRM parameer. So, for each cae udy, we are gong o ue he e of qualy ndcaor ha are well relaed o he parameer e for auo-unng. To do ha, we meaure he correlaon beween qualy ndcaor and we ue only pernen ndcaor for he parameer adapaon. The rucure of h chaper a follow: Secon 2 decrbe qualy ndcaor canddae for he conrol proce and he correlaon beween hem. The choce of ndcaor and he correlaon analy are done n he conex of UMTS auo-unng bu he ame mehodology follow n he nex chaper. Secon 3 rea he cae of dynamc reource allocaon beween RT and NRT ervce. For h cae udy, a guard band for RT call reerved and dynamcally adaped o opmze reource ulzaon and o acheve opmal radeoff beween QoS of RT and NRT uer. In econ 4 of handover parameer are auo-uned o mprove nework performance n erm of call ucce rae (CSR). We how ung mulaon reul ha he auo-unng of mobly parameer balance he raffc beween he nework Bae Saon (BS) whch he orgn of he capacy gan. Th parameer adapaon brng abou up o 30 percen of capacy gan. The nework performance whou auo-unng concep ulzed a a benchmark o evaluae he added value brough by he auo-unng proce. Concludng remark end h chaper n econ Correlaon beween qualy ndcaor Preenaon of ued qualy ndcaor From operaor pon of vew, he degree of cuomer afacon and he well operang of he nework are meaured by a e of merc and qualy ndcaor, called alo key performance ndcaor. Merc are low level meauremen carred ou on he nework nerface wherea 47

66 Applcaon of auo-unng o he UMTS nework qualy ndcaor are hgh level meauremen and quane derved from proceng low level meauremen. For example, he number of blocked call a merc bu he blockng rae a nework qualy ndcaor. So, qualy ndcaor are obaned by moohng or flerng merc couner. In h chaper, we focu on he followng ndcaor: ) Load merc In UMTS echnology, he load can be evaluaed hrough he oal ranmed power n downlnk compared o he maxmum power. However h merc no enough o judge he avalably of reource n a cell. We have o ake no accoun alo he lmaon due o he channel elemen, orhogonal code and he avalably of gnallng channel. We can alo defne he load generaed by each ervce cla by ummng power allocaed o. NRT load can alo be calculaed ung average hroughpu or average delay. In releae 4, he exchange of load nformaon on he Iur nerface already andardzed beween wo RNC and n releae 5 h exended o he Iur-g beween RNC and BSC. Bede, n releae 5, he RNC ha he capably o end and receve cell load nformaon from arge/ource yem hrough he Iu nerface. In releae 5, a dncon made beween RT load (converaonal and reamng clae) and NRT load (neracve and background clae). However he load defned a a generc meaure. The cell load aumed o be evaluaed n a vendor pecfc manner by one RNC and only a value varyng beween 0 and 100% en o oher RNC. ) Call Seup Succe Rao (CSSR) veru Call Blockng Rae (CBR) Th ndcaor defned a he rao of he number of ucceful call eup dvded by he number of call eup aemp durng a ceran perod of me. Unucceful call eup aemp can be caued by he non-avalably of rado or nework reource, he falure of nework elemen or by he lack of coverage. Th ndcaor can be defned for one cell (.e. defned baed on he number of call paed and acceped n he cell) or can be defned a een by he nework (.e. defned baed on a large number of call paed by any uer under he nework). Uually, connecon ucce rao n he nework hould be hgher han 98%. The percenage of uer blocked whle requeng acce o he nework called Call Blockng Rae (CBR). The um of CSSR and CBR 1. ) Call Droppng Rao (CDR) veru Call Succe Rae (CSR) CDR he rao of dropped call durng ongong converaon dvded by he number of uccefully ared call durng a defned perod. CSR defned a he percenage of call ha are admed o he nework and normally end her communcaon whou any underable nerrupon. CSR CSSR ( 1 CDR) (4.1) 48

67 Applcaon of auo-unng o he UMTS nework Boh ndcaor are nfluenced by changng nework condon (e.g. rado or load condon), equpmen malfuncon or he mobly of he uer. v) Average hroughpu Th ndcaor he average of uer b rae. I calculaed baed on eon call (NRT call) conneced o a cell or o he whole nework dependng on he need for global ndcor or local ndcaor per a cell. Th ndcaor nfluenced by he avalable b rae and he round-rp me (RTT), whle hee agan depend on he bearer parameer, on he load change and rado condon a well a delay n ub-nework uch a he core nework, corporae nrane and/or he publc nerne. v) Average uer afacon The afacon of a NRT moble he rao beween he allocaed b rae and he requeed b rae. For each cell, he average afacon, denoed S NRT, calculaed a he average of all NRT uer (conneced o he cell) afacon [10]: S NRT 1 Rm (4.2) max M R NRT m BS m where, M NRT he number of NRT moble conneced o he cell. R m he perceved b rae of he moble m and R requeed b rae. max m In h udy, for each menoned ndcaor, we ore he nananeou value (merc) and he flered value (qualy ndcaor). However, o avod ung hghly ocllang value n he npu of he auo-unng conroller, we ue only flered ndcaor. Recall ha, he flerng operaor of a qualy ndcaor x() over a flerng perod T f defned by: T 1 ( ) f x 1 Fl x( ) (4.3) T f 0 By ung mooh ndcaor, we wan o make ure ha he parameer eng performed by he FQLC baed on underlyng rend and no nananeou change Correlaon beween qualy ndcaor The objecve of calculang correlaon beween qualy ndcaor o avod ung, a he conroller npu, qualy ndcaor ha are auo-correlaed. In fac, ung correlaed ndcaor dor he reul and ncreae he complexy of geng dynamc opmal oluon a he end of he learnng proce. 49

68 Applcaon of auo-unng o he UMTS nework Le X and Y be wo aonary gnal repreenng 2 qualy ndcaor. The covarance funcon beween X and Y wren a: C xy 1 N N n 1 x ( ) y( ) E[ x( ) ] E[ y( ) ] n n (4.4) The correlaon beween he varable X and Y gven by: Cxy ρ ( x, y ) (4.5) CxxCyy From he la formulae, urn ou ha r alway beween -1.0 and If he correlaon near 1 or -1, he wo ndcaor varable X and Y are hghly correlaed and when ρ end o 0, he wo varable are praccally uncorrelaed. In able 4.1, we preen an example of correlaon beween dfferen global qualy ndcaor. The ndcaor value are obaned by mulang a UMTS nework for a long perod. To ge dfferen value of he ndcaor, generaed raffc, n he nework, change n me and n pace from a mulaon o anoher. CSR for RT CSR for NRT Throughpu per moble Safacon CSR for RT 1 0,63-0,5-0,39 CSR for NRT 0,63 1-0,55-0,44 Throughpu per moble -0,5-0, Safacon -0,5-0, Table 4.1. Correlaon beween qualy ndcaor. We noe ha accordng o able 4.1, RT and NRT call ucce rae are no very correlaed wh each oher and wh he oher qualy ndcaor. However, Throughpu per moble and average uer afacon are hghly correlaed wh each oher. Conequenly, for he conroller npu, we rean RT and NRT CSR. A a hrd ndcaor, we ake he uer afacon nce more correlaed o he hroughpu and mplcly o he qualave uer afacon. 4.3 Auo-unng of reource allocaon n UMTS In UMTS yem, each BS ha a lmed of capacy, defned a he oal avalable ranmon power n downlnk and an accepable hrehold of nerference n uplnk. The capacy ha o be 50

69 Applcaon of auo-unng o he UMTS nework hared effcenly among uer wh dfferen ervce and qualy requremen. Generally, ervce can be me enve (voce, vdeo reamng) or me oleran (background applcaon). The former are RT ervce and he laer are NRT applcaon. In a raffc envronmen characerzed by hgh varably and dynamcy, RT and NRT ervce can be n connuou compeon. Hence, rule and mehod have o be defned o eablh a far reparon of he rado reource beween hee ervce uer. The ar nerface capacy may be dvded no wo par: he fr hared beween RT and NRT ervce and he econd reerved for RT ervce a a guard band, noed here a X RT, o prorze RT uer. Baed on ome qualy ndcaor for boh ervce, he Fuzzy Q-Learnng conroller dynamcally regulae X RT reerved for he RT. The objecve of h econ hen o evaluae he gan ha can be acheved by auo-adapng he guard band accordng o he qualy ndcaor of each ervce cla Admon conrol raegy A hown n fgure 4.1, he downlnk capacy n each BS, defned a he maxmum ranmed power, drbued among conrol channel and raffc channel. To avod yem auraon and exceve call droppng, we keep 5% of he power margn below he oal BS ranmed power. 10% of capacy can be alo reerved n order o mnmze droppng of handoff call nce call droppng perceved wore han call blockng. The re of he capacy pl no wo par: The fr, denoed X mx, hared beween boh RT and NRT ervce and he econd, X RT, reerved a a guard capacy for RT call and common channel. Maxmum Load Xmax(e o 95%) Reerved for Sof handover Admon Load Threhold (e o 85%) Shared band beween RT and NRT raffc 5% 10% X TR X mx Droppng ome nerferng moble Blockng boh RT and NRT call Guard band reerved for RT call and common channel (auo-adjuable) Only NRT are blocked Fgure 4.1. Capacy model for a UMTS bae aon [10]. The admon conrol raegy, ued n h udy, baed on a power hrehold a decrbed preenly. Le Load be he rado load of a gven BS. I defned a he rao beween he nananeou power and he oal avalable BS power. The admon crera of a call are decrbed below [10]: If Load < X mx, boh RT and NRT call are acceped. 51

70 Applcaon of auo-unng o he UMTS nework If X mx Load < X mx + X RT, RT call are admed no he nework bu NRT call are blocked. If X mx + X RT < Load 0.95, all new call are blocked. Only lnk addon due o of handover auhorzed. When Load exceed 0.95, he yem drop ou ome moble,.e. moble wh hgh power conumpon. Snce raffc drbuon dfferen for each ervce and pace-me varable, eng a fxed X RT lead o an neffcen ue of avalable capacy. For nance, a hgh value of X RT caue a qualy degradaon of NRT ervce when he raffc of he laer hgh and he one of he RT low. The need for X RT auo-unng can be lluraed by fgure Th fgure obaned by mulang a UMTS nework compoed of 24 cell. Traffc of RT and NRT ervce generaed ndependenly and non-unformly n he nework map. We compare he evoluon of he qualy of RT and NRT ervce when varyng he guard band X RT for wo dfferen raffc cenaro. The ued qualy ndcaor he call ucce rae, denoed CSR RT and CSR NRT for repecvely RT and NRT ervce. When he raffc arrval rae of RT and NRT equal 3 and 5 moble/ repecvely, he be confguraon of he guard band abou 25% of he avalable power f no prory gven o he RT ervce. If h parameer fxed hen, for he condon ha RT raffc rae equal 5 moble/ and NRT raffc rae equal 2 moble/, he qualy of RT ervce degraded compared o he NRT ervce. In he la raffc condon, he relave mprovemen n RT qualy, when gong from he confguraon of 25% of he guard band o around 50%, come a a prce of a mall NRT qualy degradaon. Therefore, he dynamc rade-off beween RT and NRT qualy can be well acheved by opmally auo-unng he guard band. Call Succe Rae (CSR) Be value of he Guard band for each raffc cenaro QoS of RT for Trafc3 & 5 QoS of NRT for Trafc3 & 5 QoS of RT for Trafc5 & 2 QoS of NRT for Trafc5 & Guard Band for RT (X RT ) Fgure 4.2. Call ucce rae of each ervce a a funcon of he RT guard band for wo raffc uaon: (1) RT3 and NRT5; and (2) RT5 and NRT2. 52

71 Applcaon of auo-unng o he UMTS nework Qualy ndcaor, acon and renforcemen funcon The concep of ung FQLC o adap he guard band n each BS depced n fgure. 4.3 Snce each BS reponble for managng reource beween dfferen raffc clae ndependenly from oher cell, we aume ha he conroller logcally mplemened n each BS. However he conroller hould be phycally mplemened n he RNC becaue he reource allocaon algorhm nalled n he RNC. From he prevou chaper, recall ha a each me ep he conroller oberve he curren BS ae () (defned a a vecor of qualy ndcaor), and perform an acon a ( ) (changng he guard band value) ha hf he nework o anoher ae. One ep laer, he conroller receve from he nework a reward gnal or renforcemen funcon r, 5% 10% Renforcemen gnal RT & NRT qualy ndcaor Q-Learnng X TR X mx Look-up able Changng he guard band value Fuzzy logc conroller Fgure 4.3. Fuzzy Q-learnng conroller for auo-unng he RT guard band n each BS. In h cae udy, he qualy ndcaor vecor ha we have ued a an npu o he conroller (CSR TR, CSR NTR, S NTR ). Th choce baed on he correlaon able gven n he prevou econ. The average degree of afacon of moble n he BS S NTR more pernen han hroughpu becaue reflec he uer afacon and coheren wh he vecor ae. A can be oberved, all qualy ndcaor are varable n [0,1]. The acon a of he conroller modfe he guard band value accordng o he receved qualy ndcaor (or yem ae). The conroller can ncreae he guard band by 0.05, keep conan or decreae by The poble acon n each fuzzy ae belong hen o he e {-0.05, 0, +0.05}. The guard band value mu be bounded by fxed lower and upper bound (e here o 20% and 50% repecvely). To gude he conroller n fndng a be rade-off beween RT and NRT ervce, we ue a renforcemen funcon ha prorze he RT raffc n a hghly loaded BS and reward wh far conderaon n normal condon r ( ) α CSR + β CSR + ω S (4.6) TR NTR NTR 53

72 Applcaon of auo-unng o he UMTS nework (α,β,ω) he weghng vecor ha gve he dered mporance o each qualy ndcaor. In our udy, we change h vecor accordng o he raffc condon a follow: If (CSR RT 0.85) hen (α,β,ω)(1,0,0); If (0.85 < CSR RT 0.95) hen If (S NRT 0.70) hen (α,β,ω)(0.5,0,0.5); o Ele f (CSR NRT 0.9) (α,β,ω)(0.5,0.5,0); o Ele (α,β,ω)(1,0,0); If (CSR RT > 0.95) hen (α,β,ω)(0,2/3,1/3). I noed ha oher choce for he weghng vecor wll lead o dfferen opmal comprome beween RT and NRT ervce Performance evaluaon To evaluae he performance of he propoed auo-unng mehod, a Sem Dynamc Smulaor (SDS) ha been ulzed. The SDS perform correlaed napho o accoun for he me evoluon of he nework. Afer each me ep ha can ypcally vary from a enh o a couple of econd, he new moble poon and he power ranmed from/o he moble are compued. The mulaor developed n France Telecom R&D. A compendum of he mulaor preened n appendx C. Smulaon have been carred ou on a UMTS nework compoed of 24 ecor n a dene urban envronmen wh hgh raffc level. Two ervce have been ued n he mulaon. The fr voce (RT) ervce, generaed wh a Poon raffc model and havng exponenally drbued communcaon duraon wh average of 100. The econd ervce he FTP (NRT) ervce. An FTP call generaed by a Poon proce and communcaon duraon depend on he raffc condon. We model a buffer for he FTP ervce wh unlmed capacy. The buffer cleared whenever he yem allocae he neceary b rae. The FTP ource are modelled by ON-OFF even wh exponenally drbued duraon (an 'ON' even mean ha he FTP ource acvaed and vce vera for an 'OFF' even) [93]. The average duraon of he 'ON' ae e o 100 and 1 for he 'OFF' ae. FTP ervce reman acvaed unl he fle compleely downloaded. In he 'ON' ae, he volume of he ranmed daa follow a Pareo drbuon [93]. I mean lengh and medan varaon are repecvely e o 10 and 1.2 Kbye. The moble can randomly change drecon whn a lmed angular nerval. A he border of he area, he uer doe no leave he nework bu refleced back. FTP uer are pederan (3 km/h) or mmoble wherea for voce uer, he average peed e o 20 km/h. A comparon made beween he propoed dynamc veron (adaped guard band) and he clac veron (wh fxed guard band) by varyng he call arrval rae of he RT and NRT raffc. We keep he um of her call arrval rae conan. In fgure 4.4, we preen he global CSR (meaured n he whole nework) of each ervce veru he RT arrval rae for a hgh raffc condon (he um of he RT and NRT arrval rae equal 8 54

73 Applcaon of auo-unng o he UMTS nework moble/). We oberve ha when he RT raffc low (beween 0 and 2 moble/), eng a fxed guard band lead o a reource wae, epecally for a hgh guard band value. Wh h propoed guard band adapaon, he gan of NRT capacy hgh compared o a mall degradaon of RT ervce when he guard band e o 45% (0.78 of NRT CSR wh he guard band adapaon veru 0.68 of NRT CSR for he cae where he guard band fxed o 45%). When he RT raffc ncreae (beween 3 and 5), dynamc adapaon gve lghly beer CSR for RT ervce han he 25% fxed guard band. Th can be explaned by he choce of renforcemen funcon whch prorze RT ervce over he NRT one, when boh ervce coex n he nework wh equal proporon. For hgh RT raffc, he propoed algorhm doe no do anyhng nce he RT ervce occupe all he reource. Fgure 4.5 and fgure 4.6 how he drbuon of CSR beween BS for each ervce n he cae of raffc arrval rae equal o 2 moble/ for RT and 6 moble/ for NRT. A expeced, when a BS hghly loaded, and he qualy of NRT medocre compared wh he RT one, he FQLC end o degrade lghly he RT ervce qualy by decreang he guard band value and enhancng he qualy of he NRT ervce. 1 C a ll u c c e ra e 0,95 0,9 0,85 0,8 0,75 0,7 RT_D NRT_D RT_F25% NRT_F25% RT_F45% NRT_F45% 0,65 0, Arrval rae of RT (moble/) Fgure 4.4. Call ucce rae a a funcon of RT call arrval rae (RT_D mean RT CSR n he dynamc veron, RT_F25% mean RT CSR for he fxed guard band of 25%). A depced n fgure 4.5, 19 ecor have a RT CSR hgher han 0.9 for he cae wh a guard band fxed o 45%, wherea for he cae wh a guard band fxed o 25% only 9 cell acheve he qualy requremen. Dynamc adapaon ncreae he number of cell from 9 o 11 wh a CSR hgher han 0.9. A for he NRT qualy, we oberve ha only 4 cell have a NRT CSR hgher han 0.85 for he guard band fxed o 45%. Ung dynamc opmzaon, we can acheve up o 10 cell havng a NRT CSR hgher han

74 Applcaon of auo-unng o he UMTS nework number of cell Guard Band fxed o 25% Guard Band fxed o 45% Adapve Guard Band 0,65 0,7 0,75 0,8 0,85 0,9 0,95 1 Call ucce rae for RT Fgure 4.5. Hogram of RT CSR for all BS wh raffc arrval rae of RT2 and NRT6 moble/. number of cell Guard Band fxed o 25% Guard Band fxed o 45% Adapve Guard Band 0,6 0,65 0,7 0,75 0,8 0,85 0,9 0,95 Call ucce rae for NRT Fgure 4.6. Hogram of NRT CSR for all BS wh raffc arrval rae RT2 and NRT6 moble/. 4.4 Auo-unng of UMTS of handover parameer Recall ha, n UMTS yem, each cell cover a geographcal area and erve a lmed number of moble n each ervce cla wh a arge qualy of ervce (E b /N 0, CSSR, CSR,...). Connuou coverage over more han wo BS-ervce area acheved by Sof HandOver (SHO) algorhm, called alo macro dvery, whch he eamle ranfer of a call from one BS o anoher. Th ner-cellular call ranfer conrolled by a e of parameer namely acve e ze (he maxmum number of cell ervng mulaneouly he moble), Hyere_even1A (for he addon of a lnk n he acve e), Hyere_even1B (for he removal of a lnk from he acve e) and Hyere_even1C (for he ubuon of a lnk n he acve e) [19]. A moble, havng more han wo lnk n he acve e, ad o be n SHO uaon. Noe ha he parameer Hyere_even1A, Hyere_even1B and Hyere_even1C are called alo repecvely Addon Wndow (AddWn), Drop Wndow (DropWn) and Replacemen Wndow (RepWn). 56

75 Applcaon of auo-unng o he UMTS nework Each BS ha own parameer bu he algorhm mplemened n he RNC nce ha more vbly on he ae of oher BS. A unform eng of all BS lead ceranly o a ub-opmal parameerzaon. Each BS hould be opmally parameerzed wh repec o he oher BS. SHO hyere parameer of one BS rongly mpac own rado load and ha of neghbour. Conequenly, hee parameer nfluence he downlnk capacy and he qualy of ervce of he nework. For nance, by ncreang he hyere parameer value, he downlnk and he uplnk load ncreae and decreae repecvely. However, a low value of hee parameer may ncreae he rk of call droppng and may caue coverage hole [4]. Th econ preen he auo-unng proce of SHO parameer and he correpondng performance gan acheved. The auo-unng performed n each BS accordng o meaured downlnk load a well a he load meaured n neghbourng cell. Lke he prevou cae udy, he auo-unng proce baed on a fuzzy Q-learnng conroller. Here, we mprove he enre nework qualy a well a he qualy of each ndvdual BS SHO algorhm In UMTS FDD mode, he moble perform perodcally meauremen of he qualy of lnk preened n he acve e a well a he qualy of gnal comng from he declared neghbourng cell of be ervng cell. The moble compare he meauremen reul wh SHO hrehold provded by he RNC, and end a meauremen repor back o he RNC when he reporng crera are fulflled and he fnal decon for handover made. Baed on he meauremen repor receved from he moble (eher perodc or rggered by ceran even), he RNC order he moble o add/remove cell o/from acve e. Th procedure called Acve Se Updae (ASU). SHO hrehold hould be e o avod exceve ASU. The decon for an ASU manly baed on he meauremen performed on he Common Plo Channel (CPICH). The quane ha can be meaured by he moble from he CPICH are a follow [19]: Receved Sgnal Code Power (RSCP), whch he receved power on one code afer depreadng, defned on he plo ymbol. Receved Sgnal Srengh Indcaor (RSSI), whch he wdeband receved power whn he channel bandwdh. Ec/No, repreenng he RSCP dvded by he oal receved power n he channel bandwdh,.e. RSCP/RSSI. Accordng o fgure 4.7, The SHO procedure are performed eenally n hree even, dependng on he menoned SHO parameer [24]. The hree even are he addon of a new lnk or Even1A, he removal of an exng lnk, or Even1B, and he replacemen of an exng lnk, denoed a Even1C. 57

76 Applcaon of auo-unng o he UMTS nework T T T CPICH_Ec/N 0 Of BS1 DropWn AddWn RepWn CPICH_Ec/N 0 Of BS2 CPICH_Ec/N 0 Of BS3 Be ervng cell BS1 EvenA1 Even1C Even1B Fgure 4.7. UMTS SHO algorhm (even 1A, 1B and 1C). In he Even1A procedure, a BS added o he acve e f he gnal from ha BS hgher han ha of he be ervng BS mnu he hyere wndow AddWn durng a me o rgger perod T [6]: ( Ec Io) Be BS ( Ec Io) CPICH BS BS ( + CIO ) AddWn (4.7) BS CIO, or Cell Indvdual Offe of he canddae BS, an oponal offe ha can be added o he gnal of a poenal new lnk o favour he enry of h aon o he acve e. Wh repec o he Even1B, a BS removed from he acve e f he correpondng gnal maller han ha of he be BS mnu he hyere wndow DropWn durng a perod T : ( Ec Io) Be BS ( Ec Io) CPICH CPICH BS BS ( + CIO ) DropWn (4.8) In he Even1C, a BS n he acve e (upercrp In AS n (4.9)) replaced by a new BS f he correpondng gnal from he new BS bgger han ha of a BS n he acve e plu a hyere wndow RepWn durng a perod T : CPICH BS BS AS ( Ec Io) CIO ) ( Ec Io) In Re pwn CPICH + (4.9) By combnng he receved gnal from he BS of he acve e ung he maxmum rao combnng mechanm [6], he rado lnk qualy n downlnk mproved, he coverage n he cell border ncreaed, and conequenly he uplnk capacy ncreae oo. However, when he CPICH 58

77 Applcaon of auo-unng o he UMTS nework number of lnk per moble n one BS exceed a ceran hrehold, he SHO mechanm become a handcap for he downlnk capacy becaue each uer conume much more power han wh only one lnk [94]. So n very loaded nework, he SHO parameer hould be e wh a pecal care. The perfec eng o adap hem o he load of each cell, o load balancng beween cell can be reached and downlnk capacy opmzed [4], [11] FQLC-baed auo-unng of SHO parameer The ame concep of conroller a ued n he prevou ue-cae and decrbed n chaper 3 uled o une SHO parameer. A menoned prevouly, he change of he AddWn or he DropWn drecly mpac he load of he be ervng cell and he load of neghbourng cell l. So, o acheve be performance from he adapaon of SHO parameer, we hould adap hem accordng o he load of local cell (be ervng cell) and o he load of each neghbourng cell. However ung a large number of qualy ndcaor (he load of all cell) n he conroller npu conderably ncreae he complexy of he learnng proce n fndng he opmal conroller. To avod ung all he neghbourng cell' load, we model all he neghbourng cell of he be ervng one a one equvalen neghbourng cell havng a fcve load. The fcve load of a cell k defned a he weghed average of he load n he neghbourng cell. χ χ ω (4.10) f NS ( k ) k, NS(k) he cell-neghbourng e of cell k, and ωk, a weghng coeffcen defned a he flow of moble beween cell k and cell normalzed wh he oal moble flow nvolvng he cell k. Of coure, he load of each cell flered ung flerng operaor gven n equaon (4.3). Ung h modelng mplfe he yem ae or he conroller npu o ( χ, χ f ). Th ae evaluaed a each BS n each conrol loop of he conroller. In order o keep he yem a able a we can, he adapaon of he parameer AddWn and DropWn performed dependenly: durng he learnng and he conrol proce, he margn beween boh parameer kep conan (2dB). The yem qualy ha we wan o mprove here he combnaon of boh yem blockng (CBR) and droppng rae (CDR). Then, he renforcemen funcon hould conan hee ndcaor or ome oher relaed o hem. The followng renforcemen funcon afe he dered crera: * * r ( CBR CBR) + β ( CDR CDR) (4.11) where CBR * and CDR * are repecvely he operaor arge blockng rae and he arge droppng rae. β he mxng facor beween he droppng and blockng rae and depend on he operaor preference. The droppng of a call condered more penalzng han blockng, and conequenly β hould be bgger han 1. In our udy β equal 4 and he arge blockng and droppng are e repecvely o 0.05 and

78 Applcaon of auo-unng o he UMTS nework Th renforcemen raegy gve he conroller a punhmen ( r 0 ) f he nework qualy below he operaor arge and a reward ( r 0 ) oherwe. Th polcy allow he conroller o carry ou an acon ha decreae he combned blockng and droppng rae Performance evaluaon To evaluae he performance of he propoed auo-unng mehod, he ame mulaor a n he prevou ue-cae ued. The fuzzy Q-learnng conroller ha been appled o auo-une he SHO parameer of an UMTS nework wh 32 ecor n a dene urban envronmen. The uded nework exraced from a real uaon where he propagaon calbraed accordng o a profeonal model ha accoun for cluer effec. Each cell urrounded by a e of neghbourng cell conruced dynamcally by he mulaor. To model he raffc and mobly n he yem, we ue he followng aumpon: Call reque are generaed accordng o a Poon proce wh a rae λ ha vare beween 2 o 13 call reque per econd n he preen mulaon. Durng one mulaon, he raffc aonary and hen λ kep conan. To model he non-unformy of raffc, he generaed call appear n he nework area accordng o a pre-prepared raffc map. The communcaon me of each call exponenally-drbued wh a mean equal o 100. Recall ha he uer mobly baed on a wo-dmenonal em-random walk model. 80% of he generaed moble are n ndoor uaon and her peed e o zero. The peed of he re of he moble e o 60 km/h. The maxmum ranm power of each bae aon e o 20 Wa. 20% of he power agned o he common channel ncludng he common plo channel (CPICH). 65% of he power agned o raffc channel. The admon conrol hrehold hen e o 85% (20% plu 65%). When he cell load reache 85%, requeed call are blocked. The condered of handover creron he CPICH_Ec/. In a cla UMTS nework whou any conrol, AddWn and DropWn are e o 4 and 6 db repecvely. In he preen mehod, hee parameer are conrolled accordng o yem reacvy equal o 1/50-1 (.e. he perod of parameer regulaon e o 50 econd) unle conrary ndcaon. We fr llurae he convergence of he propoed auo-unng algorhm and he evoluon of he ae-acon qualy n he learnng proce. Once he algorhm converge and he ably of ae-acon qualy reached, we op he learnng proce, a decrbed n chaper 3, and we explo he obaned opmal fuzzy logc conroller. Nex, we how how he onlne conroller mprove he nework qualy. Fgure 4.8 how he convergence behavour of he learnng proce. In each me ep, we ake he maxmum varaon of he qualy (gven n equaon 3.19) of all ae and acon Q(,a). A he UMTS nework a ochac yem, he maxmum varaon of he qualy behave a a random varable ha end o a zero for a long perod of learnng. By nvegang he mahemacal expecaon of h random varable, we noce he convergence of he algorhm for 60

79 Applcaon of auo-unng o he UMTS nework me uperor o econd, equvalen o 4 day and 7.5 hour of learnng n a real nework. Th learnng proce performed by compuer mulaon for around 20 mnue. The convergence clear n fgure 4.9 whch how he me-evoluon of he qualy of ome rule (ae) and acon. Above econd, all he ae-acon quale become nearly conan. The learnng proce goe hrough hree phae. A he begnnng of learnng, he conroller gnore he be acon n each rule. I eem ha he qualy beer becaue reache for hor me a maxmum uaon whch a local opmum. Nex and rapdly, he qualy goe down nce he local opmum no good n long-erm cumulave revenue (reward). Fnally, he fuzzy Q-learnng mprove behavour and he ae-acon qualy goe up agan and reache an opmal able uaon. Once he convergence of he algorhm reached, we ae he mpac of he obaned conroller on he global and local yem qualy Maxmum varaon of acon-ae quale Tme (econd) x 10 4 Fgure 4.8. Evoluon of he convergence crera of he fuzzy-q-learnng conroller. Fgure 4.10 plo he global CSR n he enre nework a a funcon of he call arrval rae for he dynamcally-conrolled nework compared o a clacal nework wh fxed parameer. We oberve ha he propoed mehod gnfcanly mprove he yem capacy for each raffc level. Alhough, he learnng performed for pecfc level of raffc, he obaned conroller goe on mprovng nework performance n oher raffc uaon. We can oberve ha for 95% of CSR (he convenonal operaor qualy arge) he conrolled nework erve 7.85 moble per econd, wherea he clacal nework erve only 6 moble per econd. So, he capacy mprovemen a 95% of CSR equal o 30%. 61

80 Applcaon of auo-unng o he UMTS nework Q u a l y 1,8 1,6 1,4 1,2 1 0,8 0,6 0,4 0,2 0 Qualy(Rule0-acon0) Qualy(Rule2-acon4) Qualy(Rule3-acon0) Qualy(Rule3-acon4) Tme (*50 econd) Fgure 4.9. Evoluon of he qualy of rule-acon par. 100 Call Succe Rae (%) Clac nework (4/6) Opmzed nework Arrval raffc ( moble/) Fgure Call ucce rae veru ncommng arffc for he opmzed nework wh auonomc managemen compared o a clacal nework. To furher ae mprovemen n qualy of ervce a each cell, we how, n fgure 4.11, he cumulave drbuon funcon of he CSR n each cell. A expeced, he CSR ha been alo mproved n he cell a n he enre nework. For he conrolled nework, 75.8% of cell have a CSR uperor o 95%. For he clacal nework, only 72.5% of cell have a CSR uperor o 95%. 62

81 Applcaon of auo-unng o he UMTS nework Opmzed nework (raffc8) Clac nework (raffc8) 30 CDF (%) Call Suce Rae (%) Fgure Cumulave drbuon funcon of he cell call ucce rae n he opmzed nework compared o he clacal nework. Fgure 4.12 how he drbuon of cell load for he opmzed nework compared o a nework wh fxed confguraon. A expeced, conrollng dynamcally and opmally SHO parameer can lead o a beer load drbuon beween he cell nce allow a cell wh hgh raffc o gve n ceran lnk o a neghbourng cell wh low raffc. A can be een n fgure 4.12, he number of cell wh medum load hgher for he opmzed nework han for he clac one. Alo, he auo-unng reduce he number of cell wh very hgh and very low load namely ha mproved raffc balancng n he nework. 0,18 Probably Deny Funcon (PDF) 0,16 0,14 0,12 0,1 0,08 0,06 0,04 0,02 0 Opmzed nework Clac nework 4/6dB 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 Cell Load Fgure Drbuon of cell load for he opmzed nework compared o a clac nework wh fxed confguraon. 63

82 Applcaon of auo-unng o he UMTS nework Fgure 4.13 llurae he percenage of moble n SHO uaon a a funcon of raffc arrval rae for he nework wh auo-unng compared o he cae whou auo-unng. From he fgure, we oberve ha for hgh raffc level, he auo-unng proce decreae faer he percenage of moble n SHO wh repec o he nework wh fxed AddWn and DropWn parameer e repecvely o 4 and 6 db. The auo-unng proce re o allevae overloaded BS whch uffer from poor QoS and allow he nework o provde beer capacy. For low level raffc, h endency revered, namely he auo-uned nework allow more moble o be n SHO. So he auo-unng allow ncreang he coverage when here no lmaon n capacy. However, when he yem become loaded, he auo-unng lghly decreae he coverage and hghly ncreae he capacy compared o a clac nework Sof-handover rae (%) clac nework (4/6) Auo-uned nework Arrval rae (moble/) Fgure Percenage of moble n SHO uaon a a funcon of arrval rae for he nework whou and wh auo-unng Sgnallng overload due o auo-unng Durng a rggerng of SHO algorhm, dfferen gnallng procedure are nvolved [95]. The gnallng meage for an acve e updae (lnk addon or deleon) are manly ranpored n he DCCH (Dedcaed Conrol Channel). The DCCH ranpor he RRC layer (Rado Reource Conrol) meage for SHO procedure uch a meauremen repor and acve e updae. The qualy of h logcal channel very mporan for he ucce of SHO procedure. The channel DCCH and DTCH (Dedcaed Traffc Channel) are mapped no DCH (Dedcaed Channel) and SRB (Sgnallng Rado Bearer) repecvely [19]. Someme, he SRB a par of he DCH and b rae uually fxed o 3.4kb/. For a call, he RRC gnallng raffc low compared o DTCH raffc bu a hgh number of acve e updae generae more raffc n he 64

83 Applcaon of auo-unng o he UMTS nework DCCH. So he queon ha can be aren here wheher he auo-unng nfluence he capacy n he gnallng channel and he ably of he yem. Fgure 4.14 how he mpac of he conroller on he drbuon of Png-Pong effec. The Png- Pong effec relaed o he frequency of acve e updae nce whenever a moble add or delee a lnk n he acve e, ome gnallng meage are nvolved n he rado and core nerface. Here he frequency of he acve e updae meaured a he number of acve e updae, generaed by each moble, over ojourn me n he nework. In order o gve promnence o h effec, we mulae wo nework envronmen: low mobly (peed 3km/h for 20% of uer) and hgh mobly (peed 60km/h for 20% of uer) envronmen. When he conroller perform a regulaon a every 50 n a low-mobly envronmen, he frequency of acve e updae ncreaed by 10% for more han 10% of moble compared o a clacal nework. So he propoed conrollng proce ncreae he gnallng meage by 10%. Th rao goe up o 14.3% n a hgh-mobly envronmen. By decreang he reacvy of he conroller o one regulaon per 100, he addonal gnallng nroduced by he conroller decreaed o 7.1% n a hgh-mobly envronmen. The adapaon of he conroller or he auo-unng reacvy need o be carefully uded n each nework envronmen. The rade-off beween he capacy gan and he aocae gnallng nroduced by he auo-unng ou of cope CDF (%) Clac nework wh mobly3km/h for 20% of uer Opmzed nework wh mobly3km/h for 20% of uer and reacvy50 Clac nework wh mobly60km/h for 20% of uer Opmzed nework wh mobly60km/h for 20% of uer and reacvy50 Opmzed nework wh mobly60km/h for 20% of uer and reacvy ,00 0,05 0,10 0,15 0,20 0,25 0,30 0,35 0,40 0,45 0,50 Frequency of Acve Se updae (1/) Fgure Cumulave drbuon funcon of he acve e updae frequency. 65

84 Applcaon of auo-unng o he UMTS nework 4.5 Concluon Th chaper ha preened reul for he applcaon of auo-unng and opmzaon algorhm n UMTS nework. The followng cenaro have been uded: The fr cae he auo-unng of reource allocaon: he RT guard band dynamcally regulaed o acheve opmal radeoff beween QoS of RT and NRT uer. Smulaon reul have hown an effcen comprome beween he perceved QoS n RT and NRT ervce epecally when he raffc unbalanced. However, when he raffc of boh ervce very hgh, he auo-unng doe no mprove he QoS for boh ervce. The econd cenaro he mobly auo-unng: The auo-unng algorhm dynamcally change he SHO parameer eng a a funcon of raffc condon, and mplcly perform raffc balancng beween cell. Th la cae udy how an mporan gan n he overall nework performance. The global capacy gan brough by he auo-unng of SHO parameer are mporan and ypcally reach 30% compared o a nework wh fxed parameer. We have hown ha he auo-unng ncreae he frequency of acve e updae and hen ncreae he gnallng meage n he rado nerface a well a n he core nework. The Png-Pong phenomena can be reduced by brngng down he reacvy of he auo-unng. So he auo-unng hould be appled o a nework wh a pecal care nce mpac he ably of he yem. The udy on auo-unng of mobly parameer n UMTS nework currenly nvegaed n he long erm evoluon (LTE) of UMTS n 3GPP TSG-RAN WG3. The nex chaper deal wh he LTE mobly auo-unng. Inead of ung fuzzy renforcemen learnng for he auo-unng, he LTE mobly adapaon baed on a predefned auo-unng funcon. 66

85 Self opmzaon of mobly algorhm n LTE nework 5 Chap. 5 Self opmzaon of mobly algorhm n LTE nework 5.1 Inroducon 3GPP (3 rd Generaon Parnerhp Projec) organzaon ha defned he requremen for an evolved UTRAN (e-utran: UMTS Terreral Rado Acce Nework) [96] and are currenly n advanced proce of pecfcaon [97]. The evoluon of 3G UTRAN referred o a he 3GPP Long Term Evoluon (LTE). Dfferen workng group are nvolved n defnng he archecure and he echnology of he rado acce and he core nework [98]. In he framework of he workng group 3GPP TSG-RAN WG3, here have been dcuon and ude on he ue of elf-confguraon, elf-unng and elf-opmzaon n he e-utran yem [99]. In he fr phae of he nework opmzaon/adapaon, neghbour cell l opmzaon and coverage and capacy conrol have been propoed [99]. The udy of auo-unng of mobly parameer ha been furher denfed a a relevan cae udy of elf-confguraon and ha been propoed n dfferen echncal repor [100] [101]. The purpoe of h chaper o preen an approach for auo-unng of handover algorhm n LTE yem and o preen hrough a cae udy he performance acheved by he propoed auoadapaon approach. Unlke n UTRAN where of-handover ued for mobly, n e-utran a hard handover oluon for mobly ha been adoped. The handover algorhm ha no been pecfed by 3GPP for he e-utran and for h reaon, we adap an algorhm mlar o he one ued n GSM nework. The chaper organzed a follow: n he econd econ, an overvew on he LTE yem preened, ncludng he yem requremen, archecure and he phycal layer. The hrd econ develop yem and nerference model. The fourh econ deal wh he LTE mobly managemen algorhm ncludng auo-unng cheme. Smulaon reul are gven n he ffh econ. Fnally, a concluon ummarze he chaper. 5.2 Overvew of LTE yem The objecve of he LTE o nroduce a new moble-communcaon yem ha wll mee he need and challenge of he moble communcaon ndury he comng decade [98] [101]. LTE ha been ofen referred o a 4h generaon echnology. I characerzed by a fla archecure; a new rado acce echnology wh an OFDM (Orhogonal Frequency Dvon Mulplexng) baed phycal layer; and conderably enhanced performance wh repec o curren 3G nework, ncludng delay, hgh daa rae and pecrum flexbly. The LTE echnology pecfed by 3GPP and developed n parallel wh he evolved HSPA. Unlke he evolved HSPA ha compre a mooh evoluon of 3G nework, LTE fully baed on packe wched ranmon wh IP baed proocol and wll no uppor crcu wched ranmon. The LTE rado acce can be deployed n boh pared and unpared pecrum, namely wll uppor 67

86 Self opmzaon of mobly algorhm n LTE nework boh frequency- and me-dvon baed duplex arrangemen. In Frequency Dvon Duplex (FDD) downlnk and uplnk ranmon are carred ou on well eparaed frequency band wherea n Tme Dvon Duplex (TDD) downlnk and uplnk ranmon ake place n dfferen non-overlappng me lo. A pecal aenon gven n LTE o effcen mulca and broadca ranmon capable. Th ranmon denoed a he Mulca-Broadca Sngle-Frequency Nework (MBSFN). Sandardzaon of LTE ha been carred ou n Releae 7 and 8, and wll connue n Fr commercal deploymen are expeced from Syem requremen 3GPP ha defned ambou performance arge o he LTE yem, and he mporan one are ummarzed below [96] [101]. Some of he performance arge are gven relave o hoe of HSPA Releae 6 [22]. A he bae aon, one ranm and wo receve anenna are aumed and a he moble ermnal de, one ranm and maxmum wo receve anenna are aumed. Peak daa rae of 100 Mb/ and 50 Mb/ n downlnk and uplnk ranmon repecvely n a 20 MHz bandwdh, Improvemen of mean uer hroughpu wh repec o HSPA Releae 6: 3-4 me n downlnk; 2-3 me n uplnk; and 2-3 me n cell-edge hroughpu meaured a he 5 h percenle, Sgnfcanly mproved pecrum effcency: 2-4 me ha of Releae 6, acheved for low mobly, beween 0 o 15 km/h, bu hould reman hgh for 120 km/h, and hould ll work a 350 km/h, Sgnfcan reducon of uer and conrol plane laency wh a arge of le han 10 m uer plane round-rp me and le han 100 m for channel eup delay, Specrum flexbly and calably, allowng o deploy LTE n dfferen pecrum allocaon: 1.25, 1.6, 2.5, 5, 10, 15 and 20 MHz, Enhanced Mulmeda Broadca/Mulca Servce (MBMS) operaon Syem archecure The requremen of reducng laency and co have led o he degn of mplfed nework archecure, wh a reduced number of node. The RAN ha been conderably mplfed. Mo funcon of he RNC n UMTS have been ranferred n he LTE o he enodeb (enb) ha conue now he RAN par, and denoed a he e-utran. The e-utran con of enb nerconneced wh each oher by mean of he X2 nerface (ee Fgure 5.1). The enb are alo conneced by mean of he S1 nerface o he Evolved Packe Core (EPC), and more pecfcally, o he Mobly Managemen Eny (MME) va he S1-MME nerface, and o he Servng Gaeway (S-GW) va he S1-U nerface. The S1 nerface uppor a many-o-many relaon beween MME / Servng Gaeway and enb. Among he funcon of he enb are RRM funcon, uch a rado admon conrol, rado bearer conrol, connecon mobly conrol, dynamc reource allocaon (chedulng) o he Uer 68

87 Self opmzaon of mobly algorhm n LTE nework Equpmen (UE) n boh uplnk and downlnk; IP header compreon and encrypon of uer daa ream; roung of uer daa oward he Servng Gaeway (S-GW); chedulng and ranmon of pagng meage; and chedulng and ranmon of broadca nformaon [97]. The MME reponble for he followng funcon: drbuon of pagng meage o he enb; ecury conrol; dle ae mobly conrol; SAE bearer conrol; and Cpherng and negry proecon of Non-Acce Sraum (NAS) gnallng. The erm SAE, or Syem Archecure Evoluon ha been gven by 3GPP o he evoluon of he core nework, and wa fnally denoed a he EPC. The Servng Gaeway he mobly anchor pon. The dfferen funcon of he enb, MME and he S-GW are depced n Fgure 5.2. EPC MME S-GW S1-MME S1-U E-UTRAN X2 enb enb Fgure 5.1. LTE archecure. enb Iner Cell RRM Rado bearer conrol Connecon mobly conrol Rado admon conrol MME NAS ecury Idle ae mobly handlng SAE bearer conrol enb meauremen confguraon & provon Dynamc reource allocaon (chedulng) S1 S-GW Mobly anchorng E-UTRAN EPC Fgure 5.2. E-UTRAN (enb) and EPC (MME and S-GW). 69

88 Self opmzaon of mobly algorhm n LTE nework Phycal layer LTE ue OFDMA (Orhogonal Frequency Dvon Mulple Acce) a he downlnk ranmon cheme [98] [103]. OFDMA ue a relavely large number of narrowband ubcarrer, ghly packed n he frequency doman. The ubcarrer are orhogonal, hence whou muual nerference. The OFDMA cheme can be rendered robu o me-dperve channel by he cyclc-prefx neron, namely he la par of he OFDM ymbol coped and nered a he begnnng of he OFDM ymbol. Subcarrer orhogonaly preerved a long a he me dperon horer han he cyclc-prefx lengh. To acheve frequency dvery, channel codng ued, namely each b of nformaon pread over everal code b. The coded b are hen mapped va modulaon ymbol o a e of OFDM ubcarrer ha are well drbued over he overall ranmon bandwdh of he OFDM gnal [104]. In he uplnk, LTE ue he Sngle-Carrer FDMA (SC-FDMA: Frequency Dvon Mulple Acce) ranmon cheme. Th cheme can be mplemened ung a DFTS-OFDM, namely an OFDM modulaon preceded by a DFT (Dcree Fourer Tranform) operaon. I allow flexble bandwdh agnmen and orhogonal mulple-acce n he me and frequency doman. The OFDM ranmon cheme allow dynamcally harng me-frequency reource beween uer. The cheduler conrol a each nan o whch uer allocae he hared reource. I can ake no accoun channel condon n me and frequency o be allocae reource. Accordng o channel varaon, n addon o choong he moble o be erved, he cheduler deermne he daa rae o be arbued o each lnk by choong he approprae modulaon. Hence rae adapaon can be een a par of he cheduler. In he downlnk, he malle agnmen reoluon of he cheduler 180 khz durng a 1 m whch called a reource block. Any combnaon of reource block n a 1 m nerval can be agned o a uer. In uplnk, for every 1 m, a chedulng decon aken n whch moble ermnal are allowed o ranm durng a gven me nerval, on a conguou frequency regon, wh a gven arbued daa rae. Schedulng n LTE a key elemen o enhance nework capacy. To enhance he RAN performance, fa hybrd ARQ (Auomac Repea-reQue) wh of combnng wll be ued o allow he ermnal o rapdly reque reranmon of erroneou ranpor block [98]. From he fr releae, LTE wll uppor mulple anenna n boh enb and he moble ermnal. Mulple anenna are among he feaure ha wll allow he LTE o acheve ambou argeed performance, ncludng mulple receve anenna, mulple ranm anenna, and MIMO (Mulple-Inpu Mulple-Oupu) for paal mulplexng Self opmzng nework funconale Whn 3GPP Releae 8, LTE conder Self Opmzng Nework (SON) funcon. Some of he SON funcon have already been andardzed and oher are n ll beng uded. I noed ha he erm Organzng omeme ued nead of Opmzng, bu have he ame meanng. SON concern boh elf-confguraon and elf opmzaon procee. Self confguraon proce defned a he proce where newly deployed node are confgured by auomac nallaon procedure o ge he neceary bac confguraon for yem operaon [97]. The deermnaon of auomac neghbour cell relaon l [105] [106] an example of elf-confguraon proce 70

89 Self opmzaon of mobly algorhm n LTE nework ha beng andardzed n LTE Releae 8. Self-opmzaon proce defned a he proce where uer equpmen and enb meauremen and performance meauremen are ued o auoune he nework. The problem of auo-unng of mobly parameer a a mean o acheve raffc balancng and o conderably enhance he nework capacy, ha been dcued whn 3GPP [100], and wll be preened n deal n h chaper. 5.3 Inerference n e-utran yem Lle maeral addreng performance and capacy analy of e-utran avalable oday. Th movae u o preen n h chaper an nerference model for he e-utran. The nerference gven baed on a yem model whch nclude he enb drbuon and he propagaon model. The nerference model ued n a econd ep by he nework level mulaon Syem model and aumpon In h econ, we analyze only he nerference n he downlnk. For he uplnk, he ame concep hould be followed. In downlnk, each ermnal repor an emae of he nananeou channel qualy o he cell. Thee emae are obaned by meaurng on a reference gnal, ranmed by he cell and ued alo for demodulaon purpoe. Baed on he channel-qualy emae, he downlnk cheduler gran an arbrary combnaon of 180 khz wde reource block n each 1 m chedulng nerval. Snce he me cale of chedulng very mall, we wll no ake no accoun he chedulng proce n he nerference model and n he yem level mulaon. Only propagaon lo and hadow fadng, namely channel varaon over large me cale are condered. However, mall-cale varaon (mul-pah fadng) are condered n he lnk level mulaon whch erve a an npu o he preen udy. The lnk level mulaon reurn a lnk curve whch repreen he hroughpu a a funcon of he receved Sgnal o Inerference plu Noe Rao (SINR). The Ukumara-Haa propagaon model ued n he 2 GHz band. The aenuaon L gven by L l d γ o ζ, where l o a conan dependng on he ued frequency band, d he dance beween he enb and he moble, γ he pah lo exponen and ζ a log-normal random varable wh zero mean and andard devaon σ repreenng hadowng loe Inerference model In e-utran yem, uer gnal are orhogonal n he ame cell hank o he OFDMA acce echnology. A a conequence here no nra-cell nerference. On he oher hand, he ame frequency band can be ued by a gven (cenral) cell and by ome oher neghbourng cell. Th generae ner-cell nerference whch lm he performance of he e-utran yem. In he downlnk for nance, ner-cell nerference occur a a moble aon when a nearby enb ranm daa over a ubcarrer ued by ervng enb. The nerference neny depend on uer locaon, frequency reue facor and load of nerferng cell. For nance, wh a reue facor equal 1, low cell-edge performance are acheved wherea for reue facor hgher han 1, he cell-edge problem reolved on he expene of reource lmaon. To make an opmal 71

90 Self opmzaon of mobly algorhm n LTE nework rade-off beween ner-cell nerference and reource ulzaon, dfferen nerference mgaon cheme are propoed n he andard [98]. One of he echnque for nerference mgaon he ner-cell nerference coordnaon. I a chedulng raegy n he frequency doman ha allow ncreang he cell edge daa rae. Bacally, ner-cell nerference coordnaon mple ceran (frequency doman) rercon o he uplnk and downlnk cheduler n a cell o conrol he ner-cell nerference. By rercng he ranmon power of par of he pecrum n one cell, he nerference een n he neghbourng cell n h par of he pecrum reduced. Th ame par of he pecrum can hen be ued o provde hgher daa rae for uer n he neghbourng cell. Th mechanm called alo paral (or fraconal) frequency reue becaue he frequency reue facor dfferen n dfferen area of he cell (Fgure 5.3). The paral frequency reue cheme propoed for e- UTRAN a combnaon beween reue 1 and 3. In a fraconal frequency reue, an admed uer ge reource block from he poron of he band a a funcon of poon n he cell. Of coure, f he uer locaed n he cell edge, he ge reource from he cell edge band, (denoed alo a proeced band). Aumng ha he pecral band compoed of C reource block, one hrd of he band reerved for he cell edge uer and he re for cell cenre uer. Reduced Tx power n cell-cener Fgure 5.3. Iner-cell nerference coordnaon cheme. The reource allocaon made accordng o uer' poon. Le L h be he pah lo hrehold eparang uer from he wo dfferen ub-band. A uer wh a pah lo hgher han L h graned reource block n he cell-edge band and oherwe he ge reource n he cell cener band. The enb ranm power n each cell-edge reource block equal he maxmum ranm power P. To reduce nercell nerference, he enb ranm power n he cell-cener band mu be lower han P. Le ε P (whereε < 1) be he ranm power n he cell-cener band. The nerference hould be deermned for wo dfferen uer accordng o her poon: he cell-cener uer and he cell-edge uer. Le m c and m e be wo uer conneced o a cell k. he moble m c ue he cenral band wherea m e ue he cell-edge band. Le Λ denoe he 72

91 Self opmzaon of mobly algorhm n LTE nework nerference marx beween cell, where he coeffcen Λ(,j) equal 1 f cell and j ue he ame cell-edge band and zero oherwe. For cell-edge uer m e, he nerference come from uer n he cell cener of he cloe adjacen cell and from he cell-edge uer n oher cell. The moble m e conneced o he cell k and ung one reource block n he cell-edge band, receve an nerferng gnal from a cell equal I G c e, m e ( 1 Λ ( k, ) ) β εp + Λ ( k ) β P ) (5.1) L, m e,, m e where P he downlnk ranm power per reource block of he cell. G, and m e L, me repecvely he anenna gan and he pah lo beween cell and he moble aon m e. The facor β (repecvely β ) he probably ha he ame reource block n he cener-cell c e band (repecvely he cell-edge band) ued a he ame me by anoher moble conneced o he cell. Ung analy gven n appendx B, he oal nerference perceved by uer m e he um of all nerferng gnal are I me k ~ Λ e ( k, ) P G, m χ e (5.2) L, me ~ 1 α he erm ( ) ( ( )) ( ) Λ k, 3 1 Λ k, ε + Λ( k, ) α e nerpreed a a new nerference 2 marx, denoed here a he fcve nerference marx for cell-edge uer. The facor α defned a he proporon of raffc erved n he cell-edge band of cell, For he cell-cener uer m c, he nerference come from uer n he cell-edge and cell-cener of cloe adjacen cell and alo from he cell-cener and cell-edge uer n oher cell. Smlarly o he cell-edge uer, he moble m c conneced o he cell k and ung one reource block n he cellcener band, receve an nerferng gnal equal o I mc k ~ Λ c ( k, ) P G, m χ e (5.3) L, me Here, he fcve nerference marx n he cell-cener band gven by ~ Λc( k, ) (5.4) α 1 α ( 1 Λ( k, ) ) ε + α + Λ( k, ) ε 73

92 Self opmzaon of mobly algorhm n LTE nework The downlnk SINR hen gven by SINR m L k, m P G k ( I + N ) m k, m h (5.5) In equaon (5.5), he ubcrp m and for m e f he moble condered a cell-edge uer and m c for he cell-cener uer. N h he hermal noe per reource block. For more deal abou he LTE nerference model, reader nved o ee he appendx B. In e-utran yem, an adapve modulaon and codng cheme are ued [103] [104]. So, he choce of he modulaon depend on he value of he SINR hrough he perceved Bloc Error Rae (BLER). The decreae of he SINR wll ncreae he BLER, forcng he enb o ue a more robu (le frequency effcen) modulaon. The laer may have negave mpac on he communcaon qualy. For nance, a lower modulaon effcency reul n a lower hroughpu and a larger ranfer me for elac daa connecon. In he preen chaper, he hroughpu per reource block for each uer deermned by lnk level curve. The uer phycal hroughpu N m me he hroughpu per reource block, where N m repreen he number of reource block allocaed o he uer m. 5.4 Auo-unng of e-utran handover algorhm Mobly n e-utran baed on hard handover raher han on of handover a n UMTS [11]. The mechanm of hard handover ha been ued n 2 nd generaon GSM nework and ha hown o be effcen for mobly managemen. In a hard handover, he uer keep he connecon o only one cell a a me, breakng he connecon wh he former cell mmedaely before makng he new connecon o he arge cell. The bac concep of handover a n GSM lkely o be mplemened n e-utran excep for he handover preparaon phae, whch requre new mechanm. The reaon for abandonng of handover relaed o he exra-complexy nvolved n mplemenaon, and he fac ha no uable for ner-frequency handover. Furhermore, a explaned n he prevou chaper, of handover handcap yem capacy n hghly loaded nework condon and wh hgh number of uer n of handover uaon. To guaranee eamle and lole hard handover n e-utran, he handover rggerng me hould be a low a poble [98]. In general, here are dfferen caue for handover: a uer can move o anoher cell becaue he verfe a power budge condon or becaue he experence bad channel qualy. In h work we conder only he fr cae, namely power budge baed handover. Th handover baed on he comparon of he receved gnal rengh from he ervng cell and from he neghbourng cell. 74

93 Self opmzaon of mobly algorhm n LTE nework E-UTRAN handover algorhm In order o udy he performance of he auo-unng of e-utran hard handover, ome aumpon for he call admon conrol (CAC) and reource allocaon are made. In he CAC algorhm, a uer can be admed o he nework only when he followng condon are fulflled: Good gnal rengh: he moble elec he cell ha offer he maxmum gnal. If h gnal lower han a pecfed hrehold hen he moble blocked becaue of coverage horage. Th condon n fac a elecon creron. I noed ha n 3GPP, here no pecfcaon for LTE cell elecon and reelecon. Reource avalably n he eleced cell: he moble can be graned phycal reource n erm of reource block beween a mnmum and maxmum hrehold. When he gnal rengh condon afed, he enb check for reource avalably. If he avalable reource lower han a mnmum hrehold, he call blocked. Hard handover performed n h udy ung a mlar algorhm o he one ued n GSM: whle n communcaon, he moble perodcally meaure he receved power from ervng enb and from he neghbourng enb. The moble, nally conneced o a cell k, rgger a handover o a new cell f he followng condon are afed: The Power Budge Quany (PBQ) hgher han he handover margn: * * PBQ P Pk HM, + ( k ) Hyere (5.6) * where P k he receved power from he enb k expreed n db; HM(k,) he handover margn beween enb k and ; he Hyere a conan ndependen of he enb and moble aon and fxed n h udy o 0. The receved power from he arge enb mu be hgher han a hrehold. Th he ame condon a n he CAC proce. Enough reource block are avalable n he arge enb. The la condon requre nformaon exchange beween enb becaue he orgnal cell ha o know a pror he load of he arge cell; oherwe he handover blnd and he communcaon rk o be dropped. In an ner-enb handover procedure, he ource enb reponble for performng handover preparaon o he arge enb baed on meauremen repor ranmed by he moble. If he hghe ranked cell led n he moble meauremen repor congeed and can no adm ncomng call epecally real-me ervce call, he ource enb perform handover preparaon o he cell wh he nex hghe rank. On he oher hand, f he ource enb ha he load knowledge of neghbour' enb, could effcenly decde o whch cell he handover hould be performed before nang handover preparaon procedure [107]. Therefore, he ource enb need o perform handover decon conderng load nformaon. In 3GPP propoal, here are manly wo oluon for he ource enb o know wheher he hghe ranked cell can adm he ncomng call or no. The fr oluon o andardze load 75

94 Self opmzaon of mobly algorhm n LTE nework nformaon exchange on he nerface X2 a a common meauremen, wherea he econd oluon concern he mplcaon of he arge cell n he handover preparaon phae. The arge cell can reply a handover preparaon falure for example f loaded. The fr oluon acheve horer delay n handover preparaon phae; however requre he defnon of a new enb meauremen for load nformaon. The exac defnon of enb load ll under dcuon n 3GPP. The econd oluon eam o be mlar o he exng handover procedure (n UMTS and GSM) where bae aon are no aware of he load au of her neghbour. In h econd oluon, he arge enb ha o repond o each handover reque meage regardle of congeon ae. A a reul, he proceor load and gnalng meage n he X2 nerface can re. If he hghe ranked cell can no accep he call, he ource enb compelled o ry handover preparaon o anoher enb reulng n a longer delay for handover preparaon phae. The fr oluon preferred o he econd one becaue of he requremen ha he hard handover be lole and he handover rggerng me hould be a low a poble Handover adapaon and load balancng From he prevouly decrbed handover algorhm, a low handover margn allow uer o be conneced o he cloe cell everywhere n he cell, bu png-pong effec may occur frequenly. On he oher hand, a hgh handover margn generae hgh nerference for cell-edge uer epecally wh he ue of low frequency reue facor, bu png pong effec problem are avoded. Adapng he handover margn allow o acheve nereng rade-off beween dfferen nework ae. Fgure 5.4 preen a ypcal handover uaon, wherea Fgure 5.5 how a uaon where handover hrehold are adaped o he relave cell load, raher han beng conan. In Fgure 5.5 Some of uer n he handover zone ha would oherwe be erved by he congeed cell (cell k) are now handed over o he le congeed cell. Th can be acheved by delayng he handover o he congeed cell and advancng he handover from he congeed cell or n oher word decreang he handover margn from cell k o cell and ncreang he handover margn for he oher drecon. In fac, decreang handover margn from cell k o cell allow more uer o verfy he power budge handover condon. So, lead o a decreae of he ervce area of congeed cell and o an ncreae of he ervce area of le-loaded cell. Auo-unng n a non-unformly loaded nework could be parcularly benefcal n e-utran. I mplemen a mple load balancng mechanm whch ncreae he overall capacy of he yem, by mply drbung he load more evenly beween he neghbourng cell. 76

95 Self opmzaon of mobly algorhm n LTE nework Cell k Handover from o k Handover from k o Cell Fgure 5.4. Typcal paern of geographcal drbuon of HO procedure. Cell k Cell Handover from o k Handover from k o Fgure 5.5. Example of geographcal drbuon of HO procedure wh raffc balancng Auo-unng of handover margn The auo-unng am a dynamcally adapng handover margn beween cell a a funcon of her load, o opmze nework performance. Each coeffcen of he marx HM govern he raffc flow beween wo cell. The coeffcen HM(k,) depend only on he dfference beween he load of cell and k. Defne he handover margn marx HM a The funcon f hould afy: HM ( k ) f ( χ χ ), (5.7) () f a decreang funcon from he nerval [-1,1] o [HM mn, HM max ], () ( x) + f ( x) 2 f ( 0 ), x [ 1,1] f, k where HM mn and HM max are repecvely he mnmum and he maxmum value of he handover margn. f(0) he value of he planned handover margn nce he plannng proce aume he unformy of he cell load. 77

96 Self opmzaon of mobly algorhm n LTE nework The fr condon mple ha when he cell k fully loaded and doe no erve any moble, (.e. χ k -χ approache 1) worh keepng he handover margn HM(k,) o he lowe value. For he econd condon, le x be defned a he dfference beween load of cell k and (.e. xχ k -χ ); h condon ued o avod png pong effec. I mple ha when he cell k over loaded and he cell le-loaded, cell k puhe moble o cell and converely, cell delay handover o cell k. The funcon f can be approxmaed by a polynomal (wh Taylor ere expanon). The polynomal coeffcen can be dynamcally deermned ung learnng echnque. The concep of fuzzy renforcemen learnng preened earler n prevou chaper could be well ued. In he preen udy, we rerc he developmen of f(x) o he order 1: f ( x) f ( 0) + ( f ( 0) HM ) x (5.8) The developmen of order 0 correpond o he clacal cae whou any auo-unng. The mulaon am a comparng he developmen of order 1, namely wh auo-unng o he cae whou auo-unng. max 5.5 Smulaon and reul To evaluae he performance of he propoed auo-unng mehod, a dynamc mulaor developed ung Malab ool ha been ulzed. The mulaor concepually mlar o he UMTS mulaor, preened n appendx C. Smulaon have been carred ou on a 3G LTE nework compoed of 45 enb (Fgure. 5.6). Each enb ha a fxed capacy equal o 25 reource block (correpondng o a 5 MHz bandwdh). The uded cenaro ue a non-unform raffc drbuon reulng n unbalanced cell load. Only an FTP ervce cla condered. An FTP call generaed by a Poon proce and he communcaon duraon of each uer depend on b rae. Each uer allocaed a lea one reource block and a mo 4 reource block o download a fle of 5 Mbye. The value of he funcon f n 0 equal 6 db. The mnmum and he maxmum handover margn value, HM mn and HM max, are e repecvely o 0 db and 12 db. Fgure 5.7 preen he acce probably (he complemenary of he blockng rae) veru he raffc neny for he cae of auo-unng compared wh he clac cae whou auo-unng. A expeced, he gan of ung auo-unng mporan when he raffc neny low becaue he dperon of cell load ll hgh. For hgh raffc nene, all cell load approach 1 and he load dfference of adjacen cell become oo mall o benef from raffc balancng. Accordng o he auo-unng of order 1, he handover margn end o he defaul handover margn, f(0), when he raffc ncreae and all load end o 1. 78

97 Self opmzaon of mobly algorhm n LTE nework Fgure 5.6. The nework layou ncludng coverage of each enb. Fgure 5.7. Admon probably a a funcon of he raffc neny for auo-uned handover compared wh fxed handover margn nework (6dB). In fgure 5.8, we preen he connecon holdng rae (he complemenary of he droppng rae) a a funcon of he raffc neny. We noce ha he varaon of he holdng rae mall when he raffc ncreae. Th due o he fac ha a moble no dropped when here no enough reource bu nead hroughpu decreae (.e. he number of allocaed reource block 79

98 Self opmzaon of mobly algorhm n LTE nework decreae). The auo-unng gan for h qualy ndcaor mode. The rend of he wo curve for he hgh raffc neny, confrm agan ha he auo-unng end o he clac cae n very hgh raffc condon. Fgure 5.9 how he average hroughpu per uer a a funcon of he raffc neny. The hroughpu per uer a decreang funcon of he raffc rae nce an ncreang funcon of he SINR. The acheved gan of he auo-unng hgh. For nance, for he raffc neny equal 5 uer/, he hroughpu per uer approxmaely 1.15Mbye/ wherea for he clac cae, only 0.975Mbye/. Th gan explaned by he followng wo reaon: 1. The mplemenaon of reource allocaon: when here are enough reource n he cell, he uer ge he maxmum number of reource block. So b rae hgh and he uer end quckly communcaon. A a conequence, rapdly releae reource for new uer. Th explan he gan n ucceful acce rae brough by he auo-unng. 2. Inerference dvery: due o he auo-unng, he drbuon of ner-cell nerference become more or le he ame n each enb nce he nerference experenced by each uer depend on he load of he neghbourng cell. Hence, load balancng lead o nerference dvery. Fgure 5.8. Connecon holdng probably a a funcon of he raffc neny for auo-uned handover compared wh fxed handover margn nework (6dB). 80

99 Self opmzaon of mobly algorhm n LTE nework Fgure 5.9. Average hroughpu per uer veru he raffc neny for auo-uned handover compared wh fxed handover margn nework (6dB). In fgure 5.10, he cumulave drbuon for SINR preened for boh he auo-unng and he clacal cae. The raffc neny ha been e o 8 moble/. The nerference dvery generaed by he auo-unng mechanm lead o an ncreae of he perceved SINR Cumulave drbuon funcon Whou auo-unng Wh auo-unng SINR for acceped uer (db) Fgure Cumulave drbuon funcon of he SINR for nework wh and whou auounng, for raffc neny equal o 8 moble/. 81

100 Self opmzaon of mobly algorhm n LTE nework 5.6 Concluon Th chaper ha nvegaed auo-unng of mobly algorhm n e-utran yem. The mobly baed on hard handover. The handover margn nvolvng each couple of enb govern he hard handover and value drecly affec he rado load drbuon beween he cell. The auo-unng of he handover margn parameer balance he raffc beween neghbourng cell. A a conequence, he yem capacy ncreaed and he uer perceved qualy of ervce, namely he uer hroughpu, enhanced. The auo-unng funconaly ha been ncorporaed no a dynamc yem level mulaor and ha been mplemened o a non regular (e.g. cell are no hexagonal) LTE nework. Sgnfcan mprovemen n he cumulave drbuon of he gnal over nerference ha been acheved, and more han 15 percen ncreae n uer hroughpu ha been aaned. Thee reul how he mporance of mobly auo-unng o he performance of he e-utran yem. 82

101 UMTS-WLAN load balancng by auo-unng ner-yem mobly 6 Chap. 6 UMTS-WLAN load balancng by auo-unng ner-yem mobly 6.1 Inroducon WLAN nework become he mo popular wrele echnology o cover ho po area. WLAN are hgh-capacy nework ha can offer hgh b rae for uer wh low mobly. They are ued by cellular nework operaor o aborb raffc n localzed hgh raffc zone and o releve he wde area cellular nework. The negraon of WLAN whn a cellular nework dffcul and challengng. Sandardzaon bode, uch a 3GPP, IETF, IEEE and ETSI, are acvely workng on RAT ner-workng [108]. The am of h chaper o propoe an effcen UMTS-WLAN algorhm for load balancng by mean of neryem call admon and forced handover. To furher mprove he nework performance, he Vercal Handover (VHO) algorhm dynamcally opmzed ung auounng proce a decrbed n chaper 3 and ued n chaper 4. The raffc balancng raegy he followng: If a moble havng a packe-baed applcaon demand acce n a WLAN coverage zone, admed o he nework ha can offer a hgh b rae. When a UMTS bae aon or a WLAN acce pon ge congeed, a VHO oward he oher yem rggered o balance he raffc beween he wo yem. The Jon Rado Reource Managemen (JRRM) algorhm compare he UMTS or he WLAN load o a arge hrehold o decde wheher or no o perform a VHO. Numercal mulaon ung a em-dynamc nework mulaor llurae he effecvene of he propoed approach. The chaper organzed a follow: n he econd econ, we preen he aumpon ued n he udy. Thee aumpon cover he UMTS-WLAN ner-workng mode and he ner-yem elecon and admon conrol. The hrd econ deal wh he propoed vercal handover and auo-unng. In econ 4, we preen he yem performance n erm of capacy and hroughpu. Fnally, a concluon end he chaper. 6.2 Aumpon UMTS-WLAN ner-workng mode In he preen chaper, a WLAN nework baed on he IEEE b pecfcaon [108] condered. The UMTS nework uppoed o be deployed accordng o 3GPP releae 5 or 6. A depced n fgure 6.1, we ue an acce nework whch a very ghly coupled UMTS/WLAN nework. Recall ha he very gh couplng mode nvolve eablhng he WLAN nework a a econd rado nework yem, negraed a he UMTS RNC. The benef of h mode ha he reource conrol of WLAN co-locaed wh he reource conrol of UMTS. In h way, poble o manage he WLAN hopo a a UMTS cell or a a par of. 83

102 UMTS-WLAN load balancng by auo-unng ner-yem mobly Furhermore, he very gh couplng mode allow a eamle handover beween boh echnologe. We aume ha he nework envronmen compoed of UMTS cell wh WLAN hopo arranged n a herarchcal cell rucure. Th mean ha all WLAN Acce Pon (AP) are under UMTS coverage. WLAN AP are aocaed wh pecfc UMTS locaon area and managed by JRRM procedure nalled n he RNC. The deploymen of WLAN under he UMTS converge mple ha coverage-baed handover allowed only from WLAN o UMTS. Over me, he moble ermnal move from an area upporng only UMTS coverage, o an area wh boh UMTS and WLAN. The moble hould be capable of reporng he WLAN RSSI value o he RNC, o updae he Common RRM eny wh regard o he rado condon and faclae an ner-yem load-baed handover. Of coure, moble ermnal are aumed o have boh UMTS and WLAN capable. In addon, moble uer are aumed o have Subcrber Ideny Module (SIM) card auhorzed o ge acce o boh echnologe. Fgure 6.1. Very ghly coupled UMTS/WLAN nework Technology elecon and admon conrol The elecon procedure concern he dle mode ae of a moble. Selecon or reelecon rggered on eher nal power-up or on change of avalable nework coverage bu doe no concern he acve call. Snce he UMTS coverage alway avalable n he nework, he moble elec he cell a n he cae of a UMTS nework alone whou any oher negrang echnology. Once he moble elec a UMTS cell, ar o receve nework nformaon n gnallng meage. Campng n a cell allow he moble o reger n a UMTS locaon area. Whle ll n he coverage of he UMTS echnology, he moble can deec a WLAN hopo by cannng perodcally he medum, by acvely ranmng Probe Reque o denfed acce 84

103 UMTS-WLAN load balancng by auo-unng ner-yem mobly pon or by pavely wang o receve Beacon Frame. The RSSI value on he lnk can be meaured when he managemen frame are receved. The WLAN SSID ranmed a par of he Beacon Frame and he moble can ue h nformaon o nae he aocaon procedure. The moble ha nally camp n he UMTS cell, elec a WLAN AP and aocae o be UMTS cell. Th allow he moble o reger and acce WLAN ervce whle ll beng able o receve gnallng, pagng and yem nformaon from UMTS. The uer denfed whn he UTRAN and Core Nework, and gnallng bearer are e up for hm. To change from dle o conneced mode, he moble uer mu end an RRC connecon reque. Th procedure common o all call ype, a he gnallng ha o be e up pror o acual call eablhmen. Beng n conneced mode allow he moble uer o eablh call eon. Selecon of an UMTS cell Camp normally n UMTS WLAN no found Len for WLAN Beacon/Probe Camp normally n WLAN Aocae he AP o he be UMTS cell Fgure 6.2. Selecon procedure n very ghly coupled UMTS/WLAN nework For he admon conrol proce aumed ha call requrng gh QoS requremen uch a voce/vdeo conferencng and voce call (RT ervce) ue UMTS yem a he preferred nework and are no allowed o connec o he WLAN yem. However, reamng and background ervce (NRT call) are enabled over WLAN. When a uer generang a daa call under he WLAN coverage and ge a mnmum b rae, ue he WLAN a defaul echnology. I noed ha oher CAC raege can be mplemened a well. The man dfference beween eablhng a call on WLAN a oppoed o UMTS he ndcaon of where he rado bearer eablhed. When he JRRM eny 85

104 UMTS-WLAN load balancng by auo-unng ner-yem mobly nerrogaed durng he admon conrol procedure, he reply for a daa call o eablh he bearer over WLAN. Wh gnallng for he daa call beng provded by UMTS, he meage equence mlar o ha of he orgnang voce call. The dfference n h cae are nroduced when he bearer beng e up and negoaed. The bearer reul n daa beng ranpored over WLAN raher han UMTS, o he gnallng mu reflec h. 6.3 UMTS-WLAN vercal handover and auo-unng Vercal handover decrpon The load baed VHO algorhm beween UMTS BS and WLAN AP preened n fgure 6.3. Le T U-W be he load hrehold of UMTS BS ha ued o rgger VHO from UMTS o WLAN, and T W-U he hrehold of mnmum hroughpu offered by an AP ha ued o rgger a VHO beween WLAN o a UMT BS. Denoe by L UMTS he load of a UMTS BS and le L WLAN be he hroughpu offered by a WLAN AP. The mnmum hroughpu ha can be offered by an AP a good ndcaor of he AP load. So neceary o keep he mnmum offered hroughpu hgher han T W-U o guaranee uer-qualy of ervce. The UMTS BS loaded f L UMTS > T U-W bu an AP loaded f L WLAN < T W-U. To avod BS congeon, T U-W aumed o be lower han he UMTS admon hrehold. Perodc Check N N L UMTS > T U-W L WLAN > T W-U L UMTS < T U-W L WLAN < T W-U No acon Y Y No acon UMTS o WLAN VHO WLAN o UMTS VHO Selec moble() wh hgh b rae for VHO Selec moble() wh Low SNR for VHO N WLAN coverage Y Selec be UMTS cell & Execue VHO No acon Selec be AP & Execue VHO Fgure 6.3. Load baed VHO algorhm beween UMTS and WLAN nework. 86

105 UMTS-WLAN load balancng by auo-unng ner-yem mobly Perodcally load of BS are checked. If a BS load exceed T U-W, moble conneced o ha BS are ored accordng o her b rae. The one wh hghe b rae are handed over o a neghbourng AP of he BS unl BS-load become lower han T U-W. Of coure, coverage of handed-over moble and AP load are checked n he denaon AP before handover rggerng. Lkewe, he mnmum hroughpu ha can be offered by an AP checked perodcally. If an AP unable o offer a hroughpu hgher han T W-U, moble wh low SNR are handed over o a non-loaded neghbourng BS. When he hrehold TU-W and TW-U are well parameered, he raffc balancng enhance nework capacy Auo-unng of vercal handover parameer To dynamcally and opmally e he vercal handover parameer TU-W and TW-U, we ue he fuzzy Q-learnng conroller a decrbed n chaper 3 and ued n chaper 4. For h cae udy, we ue qualy ndcaor from boh yem and opmze jonly boh parameer TU-W and TW-U. The conroller performed only n UMTS cell bu he change affec alo WLAN AP. When he conroller perform a change of he parameer TU-W n a UMTS cell he parameer TW-U changed n all aocaed AP. The ued conroller npu he vecor u u nu ( CSR, CSR CSR ), (6.1) RT where CSR RT and CSR NRT and for call ucce rae of real me (RT) and non-real me (NRT) ervce repecvely n an UMTS bae aon. The ndcaor nu CSR NRT of a UMTS bae aon defned a nu NRT NRT CSR NRT ωu,csr (6.2) NRT NS ( u ) where NS(u) he neghbourng e of acce pon aocaed o he UMTS BS u, and ω u, a weghng coeffcen defned a he normalzed raffc flux of moble orgnang a BS u and handed over he AP. All quane n (6.1) and (6.2) are flered ung an averagng ldng wndow a preened n equaon (4.3). The conroller produce a oupu a mulaneou modfcaon o he load hrehold T U-W governng he handover from UMTS o WLAN and he hroughpu hrehold T W-U reponble for he VHO from WLAN o UMTS. The renforcemen funcon defned mlarly o equaon (4.6), wh ω0: r ( ) α CSR TR + βcsr NTR (6.3) (α,β) he weghng vecor ha gve he dered mporance o each qualy ndcaor. The change of he mporance vecor carred ou accordng o he raffc condon a follow: If (CSR RT 0.95) hen (α,β) (0.5, 0.5); 87

106 UMTS-WLAN load balancng by auo-unng ner-yem mobly If (CSR RT >0.95) hen (α,β) (0, 1). 6.4 Smulaon and performance evaluaon A mul-yem nework wh full UMTS coverage condered wh 59 UMTS cell and 22 acce pon (AP) n a dene urban envronmen (ee Fgure 6.4). Recall ha he WLAN nework condered baed on he IEEE b pecfcaon [108]. The AP are locaed n he hgher raffc zone whn he UMTS covered area. Each BS ha a fxed downlnk capacy defned a he maxmum ranm power and each AP ha a fxed capacy defned a he maxmum b-rae ha can be graned o a NRT uer. Th b-rae depend on wo facor. The fr he uer condon, nce moble wh low SNR ge lower b rae due o he lnk adapaon [108]. The econd facor he number of uer: n he CSMA/CA acce medum mechanm, when he number of uer ncreae, collon probably ncreae, reulng n he reducon of he average offered b-rae. The WLAN nework uppor only NRT FTP raffc, wherea UMTS uppor boh RT voce and NRT FTP raffc. The FTP daa call arrve n he nework accordng o a Poon proce and each uer download daa raffc fle of one Mbye. Arrval rae of voce moble 6 moble/ec. Boh ndoor and oudoor raffc preen, wh 40 and 60 percen repecvely. The oudoor uer are moble wh 3km/h peed. Fgure 6.4. Heerogeneou nework layou wh 59 UMTS cell (quare wh arrow) and 22 WLAN AP (crcle). Two cenaro are condered: he fr ue he raffc balancng algorhm. The econd cenaro boh raffc balancng and auo-unng are combned. The auo-unng performed aon by aon n he nework. In boh cenaro, he real me voce raffc kep fxed o 6 moble per econd, wherea, he packe wched (PS) FTP raffc ncreaed n a ere of dnc mulaon. Fgure 6.5 and 6.6 compare he call ucce rae (CSR) of RT and NRT raffc repecvely a a funcon of he FTP raffc neny, wh (quare) and whou (damond) auo-unng. The 88

107 UMTS-WLAN load balancng by auo-unng ner-yem mobly auo-unng of he raffc balancng algorhm conderably mprove he CSR and hence he overall nework capacy. Fgure 6.7 preen he average hroughpu of he nework a a funcon of raffc neny, for he wo cenaro, wh and whou auo-unng. The auo-unng proce ncreae he overall nework hroughpu and hence he nework profably. By correcly balancng he nework load, and by adapng he algorhm o he nework raffc, boh CSR and hroughpu are mproved. Defne he VHO execuon rae a he rao beween he number of VHO rggered o he oal number of moble admed o he nework; and he VHO ucce rae a he rao beween he number of ucceful VHO o he number of rggered VHO, boh durng he enre mulaon. We preen n fgure 6.8 and 6.9 he execuon rae and he ucce rae of VHO from UMTS o WLAN, repecvely. The nal nework ha an exce of VHO execuon rae, ncludng pngpong handover. The auo-unng proce reduce he number of VHO and mprove he ucce rae of he execued one. So, he auo-unng proce conderably avod unneceary VHO generaed by he propoed load balancng algorhm. 1 0,95 No auo-unng Auo-uned nework CSR-RT 0,9 0,85 0,8 0,75 2,5 5 7, , , ,5 25 NRT raffc arrval rae (moble/) Fgure 6.5. Call ucce rae of RT raffc a a funcon of NRT raffc arrval rae for he nework wh (quare) and whou (damond) auo-unng. 89

108 UMTS-WLAN load balancng by auo-unng ner-yem mobly 1 0,95 0,9 No-auo-unng Auo-uned nework CSR - NRT 0,85 0,8 0,75 0,7 0,65 0,6 0,55 2,5 5 7, , , ,5 25 NRT raffc arrval rae (moble/) Fgure 6.6. Call ucce rae for NRT raffc a a funcon of NRT raffc arrval rae for he nework wh (quare) and whou (damond) auo-unng. Average hroughpu No-auo-unng Auo-uned nework 50 2,5 5 7, , , ,5 25 NRT Traffc arrval rae (moble/) Fgure 6.7. Average hroughpu a a funcon of NRT raffc neny. 90

109 UMTS-WLAN load balancng by auo-unng ner-yem mobly 3G-WLAN VHO execuon rae 0.45 No auo-unng 0.4 Auo-uned nework NRT raffc arrval rae (Moble/) Fgure 6.8. Impac of auo-unng on he execuon rae of UMTS o WLAN vercal handover. 3G-WLAN VHO ucce rae No auo-unng Auo-uned nework NRT Traffc arrval rae (moble/) Fgure 6.9. Impac of auo-unng on he ucce rae of UMTS o WLAN vercal handover. 6.5 Concluon Th chaper ha preened a load balancng algorhm beween a UMTS nework and WLAN acce pon, and dynamc auo-unng. The auo-unng proce performed ung a fuzzy logc conroller ha opmzed ung a fuzzy Q-learnng algorhm. The conroller adap he UMTS load hrehold of he VHO algorhm for each BS accordng o qualy ndcaor from he BS and neghbourng WLAN acce pon. Smulaon reul how ha he auo-uned raffc balancng algorhm conrol he amoun of VHO beween he wo ub-yem, and preven unneceary handover. I conderably ncreae he overall call ucce rae for boh RT and NRT raffc. 91

110 Concluon and perpecve 7 Chap. 7 Concluon and perpecve 7.1 Concluon In h deraon, a ae of he ar for mul-yem auo-unng and elf-opmzaon preened. An overvew of each echnology and he arge parameer ha could be auo-uned gven o defne he dfferen cenaro and ue-cae condered n he he. The gven overvew nclude alo mul-yem managemen, and he advanced (or jon) rado reource managemen. The auo-unng archecure ha been decrbed for boh onlne and off-lne mode of operaon. The requremen for effcen auo-unng archecure are hghlghed by ponng ou he relaon beween nework ene and he auo-unng engne. An example of gnallng meage beween he nework and he auo-unng and opmzaon engne (AOE) ha been decrbed. The compoon of he auo-unng and opmzaon module preened wh a pecal empha on ued auo-unng ool. In h he we have manly ued he fuzzy Q-learnng algorhm and ha why we have decrbed n deph Fuzzy Logc Conrol (FLC) and renforcemen learnng. The fuzzy logc conroller hown o be a mple and effecve framework for degnng a conroller ha orcherae he auo-unng proce. In he degn of h conroller, engneerng rule are preened n erm of lnguc predcae ha are drecly ranlaed no mahemacal form. The opmzaon of he conroller hown o be eenal o guaranee a hgh qualy auo-unng proce, and may be requred when he condon of ulzaon of he conroller change,.e. nework envronmen or raffc compoon. The Q- learnng mplemenaon of he Renforcemen Learnng doe no requre a yem model and uable o fully auomae he FLC degn proce and o opmze nework parameer. In order o effcenly auo-une nework parameer, he FQLC requre decorrelaed npu ndcaor ha are relaed o he parameer, arge for auo-unng. The auo-unng of reource allocaon n UMTS nework uded. In h ue-cae, he RT guard band dynamcally adaped o acheve opmal radeoff beween QoS of RT and NRT uer. Smulaon reul have hown an effcen comprome beween he perceved QoS of RT and NRT ervce, epecally when he raffc unbalanced. However, when he raffc of boh ervce very hgh, he auo-unng doe no mprove he QoS for boh ervce. The degn and he opmzaon of he UMTS SHO algorhm have been decrbed n deal. Two parameer are opmally and dynamcally adaped, namely he SHO addwn and dropwn. The propoed auo-unng algorhm ulze downlnk load of a cell and of neghbour a npu ndcaor and ha hown o be mple and effecve. The auo-unng brng abou a capacy ncreae of around 30 percen for a nework n a dene urban envronmen. Th example llurae he mporance of auo-unng n UMTS engneerng and ha movaed u o nvegae elfopmzng handover algorhm n he 3GPP LTE yem. LTE handover opmzaon ackled ung a predefned auo-unng funcon nead of he FQLC employed n he UMTS mobly ue-cae. A GSM-lke hard HO algorhm governed by a marx handover margn condered. A margn relae each cell wh one neghbour and reponble for conrollng raffc flux beween hem. Smulaon reul how ha he 92

111 Concluon and perpecve opmzaon of handover margn n a LTE nework can mprove bac KPI, namely blockng rae and yem hroughpu by a few percen. The opmzaon of he handover margn balance he raffc beween he enb n he nework and paally moohe nerference n he nework. The la conrbuon of he he deal wh he dynamc adapaon of an ner-yem vercal handover algorhm. The VHO algorhm jonly baed on he coverage and he yem load. Lke he prevou menoned ue-cae, he VHO governed by a e of hrehold. Ung FQLC, he auo-unng adap VHO hrehold ha conrol raffc flux beween UMTS and WLAN yem and perform raffc balancng. The WLAN acce echnology aumed o be compleely managed by he UMTS RNC. A FQLC havng double oupu ha been formulaed and ha hown o be well adaped o he opmzaon of he auo-unng proce when everal nework ubyem coex. 7.2 Lmaon and perpecve Several apec relaed o he auo-unng and elf-opmzaon procee need furher nvegaon. Auo-unng mpac yem gnallng and hence worh makng a be radeoff beween he capacy gan and he gnallng overhead nroduced by he auo-unng proce. The auo-unng may requre new upplemenary channel n addon o he exen gnallng channel reerved for uer n he rado and core nework. Alhough mo of he reul obaned from he mulaon and cae ude carred ou n h he have been very ueful o llurae auo-unng mechanm nfluencng he nework performance, hee reul provde rend. Th ha varou reaon. Some uncerane and naccurace have been nroduced durng he modellng phae. Lmaon of he ued mulaon ool, aumpon and approxmaon made when buldng he yem model affec he degree o whch he realy refleced. Teng on real expermenal nework requred o provde a rch ource of nformaon and clear knowledge of elf-opmzaon behavour. In fuure reearch, more aenon could be focued o mul-crera elf-opmzaon apec. In h he, we have lmed ourelve o he ue of mono-objecve opmzaon n he framework of he Q-learnng algorhm by aggregang he varou crera. Fnally, he problem of mulaneouly acvang everal auo-unng conroller ha adap dfferen RRM parameer of parcular nere a a mean o furher boo he capacy gan. Smulang uch auo-unng procee requre complex yem modellng allowng o decrbe he coordnaon beween he dfferen conroller. Such concep known a a mul-agen problem. 93

112 Reference [1] Z. Alman, R. Skehll, R. Barco, L. Molen, R. Brennan, A. Samha, R. Khanafer, H. Dubrel, M. Barry, B. Solana, "The Celc Gandalf Framework", 13h Mederranean Elecroncal Conference, MELECON 2006, May 16-19, Malaga, Span. [2] R. Nar, A.E. Samha, Z. Alman, Procédé de calcul de l'éa de élémen du réeau an fl pour la geon de e reource à parr de ndcaeur de qualé, Breve No [3] Orange, "Self-opmzaon ue-cae: elf-unng of handover parameer", 3GPP TR [4] R. Nar, Z. Alman and H. Dubrel, Fuzzy-Q-learnng-baed auonomc managemen of macro dvery algorhm n UMTS nework, Annal of Telecommuncaon, Vol. 61, N 9-10, Sepembre - ocobre [5] H. Dubrel, R. Nar, Z. Alman, Ingénere Auomaque de Réeaux moble, chapre dan le lvre "L'auonome dan le réeaux", Traé Hermè IC2, Sepembre [6] Z. Alman, H. Dubrel, R. Nar, O.B. Amor, J.M. Pcard, V. Dacorn and M. Clerc Auounng of RRM parameer n UMTS nework, book chaper n Underandng UMTS Rado Nework Modellng, Plannng and Auomaed Opmzaon: Theory and Pracce, Wley & Son [7] R. Nar, Z. Alman, "Handover adapaon for dynamc load balancng n 3GPP Long Term Evoluon Syem", 5h Inernaonal Conference on Advance n Moble Compung & Mulmeda (MoMM2007), Jakara, December [8] R. Nar, A.E. Samha, Z. Alman "A new approach of UMTS-WLAN load balancng; algorhm and dynamc opmzaon", IEEE WoWMoM Workhop on Auonomc Wrele Acce 2007 (IWAS07), Helnk, Fnland, June 18h, [9] A.E. Samha, R. Nar, Z. Alman, "Jon Mobly RRM Algorhm for Heerogeneou Acce Nework", 13h European Wrele Conference (EWC2007), Par, France, Aprl [10] R. Nar, Z. Alman, H. Dubrel, "Opmal radeoff beween RT and NRT ervce n 3G- CDMA nework ung dynamc fuzzy Q-learnng", IEEE PIMRC 06, Helnk, Sep., [11] R. Nar, Z. Alman and H. Dubrel, WCDMA downlnk load harng wh dynamc conrol of of handover parameer, IEEE VTC2006 Sprng, Melbourne, Aurala, 7-10 Ma [12] 3GPP TS V "Rado Acce Nework; Rado ubyem lnk conrol", (Releae 1999), [13] 3GPP TS 03.22, "Dgal cellular elecommuncaon yem (Phae 2+); Funcon relaed o Moble Saon (MS) n dle mode and group receve mode". [14] 3GPP TS , "Hgh Speed Crcu Swched Daa (HSCSD) - Sage 2", (Releae 1999), [15] 3GPP TS v 5.2.0, "General Packe Rado Servce (GPRS); Servce decrpon; Sage 2" (Releae 5), June [16] X. Lagrange, P. Godlewk, S. Tabbane, "Réeaux GSM-DCS, de prncpe à la norme", Hermè Scence publcaon,

113 [17] 3GPP Techncal Specfcaon "UTRAN Overall Decrpon". [18] M. Nawrock, M. Dohler, H. Aghvam, "Underandng UMTS Rado nework modellng, plannng and auomaed opmzaon: heory and pracce" john Wley & on, [19] H. Holma and al., "WCDMA for UMTS", john Wley & on 2004, hrd edon. [20] 3rd Generaon Parnerhp Projec (3GPP) webe, hp:// [21] 3GPP TS , "Qualy of Servce (QoS) concep and archecure", Releae 6. [22] 3GPP TR : "Hgh Speed Downlnk Packe Acce (HSDPA): Overall UTRAN Decrpon". [23] H. Holma, A. Tokala, "HSDPA/HSUPA for UMTS: Hgh Speed Rado Acce for Moble Communcaon", John Wley & Son [24] 3GPP TR V6.0.1, Rado reource managemen raege, (Releae 6), [25] G. Edward, R. Sankar, "Handoff ung fuzzy logc," IEEE GlobeCom, Sngapore (November 1995) pp [26] G. Edward, R. Sankar, "Mcrocellular Handoff Ung Fuzzy Technque," Wrele Nework, Vol. 4, No. 5, pp , [27] P. Magnuon, J. Oom, "An Archecure for elf-unng cellular yem", Proc. of he 2001 IEEE/IFIP Inernaonal ympoum on Inegraed Nework Managemen, 2001, pp [28] P. Gua, P. Magnuon, J. Oom, N. Sorm, "Real-me performance monorng and opmzaon of cellular yem", Ercon Revew, n. 1, 2002, pp [29] S. Cho, K.G. Shn, A comparave udy of bandwdh reervaon and admon conrol cheme n QoS-enve cellular nework, Wrele Nework 6(4): , [30] C. Olvera, J.B. Km, T. Suda, An adapve bandwdh reervaon cheme for hgh-peed mulmeda wrele nework, IEEE Journal on Seleced Area n Communcaon, Vol. 16, No 6, pp , [31] C. Lndemann, M. Lohmann, A. Thümmler, Adapve Call Admon Conrol for QoS/Revenue Opmzaon n CDMA Cellular Nework, ACM Journal on Wrele Nework (WINET), Vol 10, pp , [32] P. Ramanahan e al. Dynamc Reource Allocaon Scheme Durng Handoff for Moble Mulmeda Wrele Nework JSAC July 1999 [33] Jongkuan Hou, Mobly-baed call admon conrol cheme for wrele moble nework ; Wrele Communcaon Moble Compung, Jul-Sep 2001; Wley [34] B.M.Epen, M Schwarz, Predcng QoS-Baed Admon Conrol for Mulcla Traffc n cellular Wrele Nework, JSAC March 2000 [35] Tao Zhang e al. Local Predcve Reource Reervaon for Handover Mulmeda Wrele IP nework, JSAC 2001, Ocober 2001, pp [36] H. Holma, J. Laako Uplnk Admon Conrol and Sof Capacy wh MUD n CDMA, IEEE VTC Fall 1999, Vol. 1. [37] 3GPP TS : "UE Procedure n Idle Mode and Procedure for Cell Reelecon n Conneced Mode" [38] 3GPP TS : "Requremen for Suppor of Rado Reource Managemen (FDD)". 95

114 [39] 3GPP TS : "Requremen for Suppor of Rado Reource Managemen (TDD)". [40] 3GPP TS : "NAS funcon relaed o Moble Saon (MS) n dle mode ". [41] R. Guerzon, I. Ore, K. Valkeahla D. Soldan, "Auomac Neghbor Cell L Opmzaon for UTRA FDD Nework: Theorecal Approach and Expermenal Valdaon", WPMC2005. [42] K. Valkealah, A. Höglund, J. Parkknen, A. Flanagan, "WCDMA Common Plo Power Conrol wh Co Funcon Mnmzaon", VTC-Fall 2002, Sepember [43] A. Hämälänena, K. Valkealaha, A. Höglunda, J. Laakob, "Auo-unng of Servce-pecfc Requremen of Receved EbNo n WCDMA", VTC-Fall 2002, Sepember [44] A. Höglund, K. Valkealah, "Qualy-baed Tunng of Cell Downlnk Load Targe and Lnk Power Maxma n WCDMA", VTC-Fall 2002, Sepember [45] J. A. Flanagan, T. Novoad, "WCDMA Nework Co Funcon Mnmzaon for Sof Handover Opmzaon wh Varable Uer Load", VTC-Fall 2002, Sepember [46] B. Homnan, W. Benjapolakul, "QoS-conrollng of handoff baed on mple ep conrol and a fuzzy nference yem wh he graden decen mehod", IEEE Tranacon on Vehcular Technology, Vol. 53, pp , May [47] V. Kunrrakakul, B. Homnan, W. Benjapolakul, "Comparave evaluaon of fxed and adapve of handoff parameer ung fuzzy nference yem n CDMA moble communcaon yem", IEEE VTS 53rd Vehcular Technology Conference, VTC 2001 Sprng. Volume 2, 6-9 May 2001 Page(): vol.2. [48] J. Ye, X. Shen, J.W. Mark, Call Admon Conrol n Wdeband CDMA Cellular Nework by ung Fuzzy Logc, IEEE Tranacon on Moble Compung, Vol 4, No2, pp , Aprl [49] P. Dn, S. Guglelmucc, "Call admon conrol raegy baed on fuzzy logc for WCDMA yem, IEEE Inernaonal Conference on Communcaon, June 2004, Page(): [50] R.N.S. Naga, D. Sarkar, "Call admon conrol n moble cellular CDMA yem ung fuzzy aocave memory", IEEE Inernaonal Conference on Communcaon, June 2004, Page(): Vol.7. [51] H. Dubrel, Z. Alman, V. Dacorn, J.M. Pcard, M. Clerc, Parcle Swarm opmzaon of fuzzy logc conroller for hgh qualy RRM auo-unng of UMTS nework, IEEE Inernaonal Sympoum VTC 2005, Sockholm, Sweeden, 29 May-1 June [52] H. Dubrel, "Méhode d'opmzaon de conrôleur de logque floue pour le paramérage auomaque de réeaux moble UMTS", hèe de docora, ENST [53] L. Jouffe, "Fuzzy Inference Syem Learnng by renforcemen Mehod", IEEE Tranacon on Syem, Man, and Cybernec, Vol. 28, pp , Aug [54] S.M. Senouc, A.L. Beylo, G. Pujolle, "Call admon conrol n cellular nework: a renforcemen learnng oluon", Inernaonal Journal of Nework Managemen, No. 14, pp , [55] F. Yu, V.W.S. Wong, V.C.M. Leung, "Effcen QoS Provonng for Adapave Mulmeda n Moble Communcaon Nework by renforcemen Learnng," Fr Inernaonal Conference on Broadband Nework, BROADNETS'04 IEEE,

115 [56] F. Yu, V.W.S. Wong, V.C.M. Leung, A New QoS Provonng Mehod for Adapve Mulmeda n Cellular Wrele Nework, IEEE Conference on Compuer Communcaon (INFOCOM 04), Hong Kong, Chna, Mar [57] Y. S. Chen, C. J. Chang, F. C. Ren, "A Q-learnng-baed mul-rae ranmon conrol cheme for RRM n mulmeda WCDMA yem," IEEE Tranacon on Vehcular Technology, Vol. 53, No. 1, pp , Jan [58] 3GPP TR V5.0.0, Improvemen of RRM acro RNS and RNS/BSS, [59] 3GPP TR V0.3.0, Improvemen of RRM acro RNS and RNS/BSS (Po Rel-5), Releae 6, 2003/2. [60] 3GPP TR V6.2.0, "Feably udy on 3GPP yem o Wrele Local Area Nework (WLAN) nerworkng". [61] 3GPP TR , "Requremen on 3GPP yem o Wrele Local Area Nework (WLAN) nerworkng" [62] 3GPP TR , "3GPP yem o Wrele Local Area Nework (WLAN) nerworkng; Syem decrpon" [63] A. K. Salknz, Inerworkng Technque and Archecure for WLAN/3G Inegraon Toward 4G moble Daa Nework, IEEE Wrele Communcaon, June 2004, pp [64] J. Perez-Romero e al, "Common rado reource managemen: funconal model and mplemenaon requremen", PIMRC [65] M. K. Sarr, M. Zeleny, "Mulple Crera Decon-Makng", Mc.Graw-Hll. [66] C.L. Hwang, K. Yoon, "Mulple Arbue Decon Makng", Sprnger-Verlag, Berln, [67] R. Seuer, "Mulple Crera Opmzaon: Theory, Compuaon and Applcaon",, John Wley & Son, [68] K. Shum and C.W. Sung, "Fuzzy Layer Selecon Mehod n Herarchcal Cellular Syem, IEEE Tran. Vehcular Technology, vol. 48, pp , Nov [69] N.D. Trpah, J.H. Reed, J. H. VanLandngham, "Adapve handoff algorhm for cellular overlay yem ung fuzzy logc", Proc. 49h IEEE Vehcular Technology Conference, vol. 2, 1999, pp [70] A. Majle, B.H. Khalaj, "An Adapve Fuzzy Logc Baed Handoff Algorhm For Inerworkng Beween Wlan And Moble Nework", Proc. IEEE 49 h PIMRC, pp , [71] P.M.L. Chan, R.E. Sherff, Y.F. Hu, P. Conforo C. Tocc, "Mobly Managemen Incorporang Fuzzy Logc for a Heerogeneou IP Envronmen", IEEE Communcaon Magazne, pp , Dec [72] P.M.L. Chan, Y.F. Hu, R.E. Sherff, "Implemenaon of Fuzzy Mulple Objecve Decon Makng Algorhm n a Heerogeneou Moble Envronmen", Proc. IEEE Wrele Communcaon and Neworkng Conference, Orlando, FL, USA, 17-21, pp , March [73] R. Agu, O. Sallen, J. Perez-Romero, L. Guppon, "A fuzzy-neural baed approach for jon rado reource managemen n a beyond 3G framework", Proc. Of 1 Qualy of Servce n Heerogeneou Wred/Wrele Nework conference, pp , Oc

116 [74] L. Guppon, J. Perez-Romero, R. Agu, O. Sallen, "A Novel Jon Rado Reource Managemen Approach wh Renforcemen Learnng Mechanm", IEEE Inernaonal Workhop on Rado Reource Managemen for Wrele Cellular Nework, Apr [75] W. Zhang, "Handover Decon Ung Fuzzy MADM n Heerogeneou Nework", IEEE WCNC 2004, Alana, pp , March [76] R.R. Yager, Mulple Objecve Decon Makng ung Fuzzy Se, Inernaonal Journal Man Machne Sude, no. 9, 1977, pp [77] C.T. Ln, C:S.G. Lee, Neural-Nework-Baed Fuzzy Logcal Conrol and Decon Syem, IEEE Tran. Compuer, vol. 40, no. 12, pp , Dec [78] Mamdan, E.H. Applcaon of fuzzy logc o approxmae reaonng ung lnguc ynhe. IEEE Tranacon on Compuer, Vol. 26, No. 12, pp , 1977 [79] Sugeno, M. Indural applcaon of fuzzy conrol. Elever Scence Pub. Co., 1985 [80] Arom, K.J.; Wenmark, B. Adapve conrol, 2nd Ed. Addon-Weley 1995 [81] Janzen, J. A uoral on fuzzy adapve conrol. Proc. EUNITE 2002 [82] P. Y. Glorennec, Apprenage par renforcemen e logque floue, n Ace de renconre francophone ur la logque floue e e applcaon (LFA'01), November [83] A Bayean approach for auomaed roublehoong for UMTS nework", he 17h Annual IEEE Inern. Symp. PIMRC 06, Helnk, Sep., [84] Emang GPRS Lnk B Rae n TEMS Invegaon, Ercon whe paper, [85] Gandalf Delverable D3.2, "Overvew and Defnon of he Global Archecure for Mul- Syem Self-Tunng", Sepember [86] K. Tomocc, M.Y. Chow, "Tuoral on Fuzzy Logc: Applcaon n Power Syem", IEEE- PES Wner meeng n Sngapore, January 2000 [87] J.M. Mendel, "Fuzzy logc yem for engneerng: a uoral" Proceedng of he IEEE, V83, n3, Mar 1995, pp [88] S. R. Suon and A.G. Baro, " Renforcemen Learnng: An Inroducon", MIT Pre, Cambrdge, MA, [89] H.S. Chang, M.C. Fu, J. Hu, S.I. Marcu, " Smulaon-baed algorhm for Markov decon procee", Sprnger, [90] S.P. Meyn, R.L. Tweede, "Markov Chan and Sochac Sably", Sprnger-Verlag, [91] T. Jaakkola, M.I. Jordan, S.P. Sngh, "On he Convergence of Sochac Ierave Dynamc Programmng Algorhm", Neural Compuaon, 6(6): , [92] C.J. Wakn, "Learnng from Delayed Reward", PhD The, Unvery of Cambrdge, England, [93] B. J. Prabhu, E. Alman, K. Avrachenkov, J. A. Domnguez, A mulaon udy of TCP performance over UMTS downlnk, n IEEE VTC [94] A. Reba, Concepon e développemen d un mulaeur UMTS, ranng repor, SupCom, [95] Sébaen Bare and al., "Analy of gnallng procedure for end-o-end QoS modellng n UMTS" echncal repor July

117 [96] 3GPP TR , V7.1.0 Requremen for Evolved UTRA (E-UTRA) and Evolved UTRAN (E-UTRAN), (Releae 7), ( ). [97] 3GPP TS , "Evolved Unveral Terreral Rado Acce (E-UTRA) and Evolved Unveral Terreral Rado Acce Nework (E-UTRAN); Overall decrpon; Sage 3 (Releae8), Sep [98] E. Dahlman, S. Parkvall, J. Sköld and P. Bemng, 3G Evoluon, HSPA and LTE for Moble Broadband, Academc Pre, London [99] 3GPP TR "Self-Confguraon and Self-Opmzaon". [100] 3GPP TR , "Self-opmzaon ue-cae: elf-unng of handover parameer", Orange. [101] 3GPP TR , "Load Balancng SON Ue cae", Alcael-Lucen. [102] H. Ekröm, A. Furukär, J. Karlon, M. Meyer, S. Parkvall, J. Torner, and M. Wahlqv, "Techncal Soluon for he 3G Long-Term Evoluon," IEEE Commun. Mag., vol. 44, no. 3, March 2006, pp [103] 3GPP TR , V7.1.0, "Phycal Layer Apec for evolved Unveral Terreral Rado Acce (UTRA) (Releae 7)" ( ). [104] 3GPP TS , V0.3.1; "Phycal Channel and Modulaon (Releae 8)" ( ). [105] 3GPP TSG RAN WG3, R , "Auomac neghbour cell confguraon", Aug [106] 3GPP TSG RAN WG3, R , "On auomac neghbour relaon confguraon", Ocobre [107] 3GPP TR , "Meauremen for handover decon ue cae", NTT DoCoMo, Orange, Telecom Iala, T-Moble, Telefonca, November [108] ANSI/IEEE Sd , Wrele LAN Medum Acce Conrol (MAC) and Phycal Layer (PHY) Specfcaon, 1999 Edon. 99

118 100 Appendx A: Convergence proof of he renforcemen learnng algorhm Propoon 3.1 Le be a hory-dependen polcy. For any nal ae x, here ex a Markov polcy ', uch ha: ( ) ( ) x V x V. Proof Le ' be he Markovan polcy defned from he hory-dependen polcy by: ( ) ( ) x a a P a a q A a S T 0, /,,, Then ( ) ( ) x a a P a a P T 0, / /, We can prove by recurrence over he relaon: ( ) ( ) x a a P x a a P T 0 0 /, /,, By conrucon, he relaon verfed for 0. We aume ha he equaly verfed unl -1 and we prove ha reman rue for. ( ) ( ) ( ) ( ) ( ) ( ) x P a p x a a P a p x a a P x P S A a S A a /, / /,, / /, / Therefore ( ) ( ) ( ) ( ) ( ) ( ) x a a P x P x a a P x P a a P x a a P /, /, / / / /, So he propoon proved. Remark v) From he prevou propoon, we deduce ha every hory-dependen polcy can be replaced by a Markovan polcy havng he ame value funcon f he nal ae gven. From now on, we ue only he markovan polcy, unle conrary menoned. v) If he polcy markovan, he proce ( ) elf a markovan proce wh a ranon marx P, defned by: ( ) ( ) A a a p a q P S, /,,, becaue

119 101 ( ) ( ) ( ) ( ) ( ) ( ) A a A a P a a P a q a a P a a P P /, /,,,...,, /,...,, /,...,, / v) Accordng o he prevou noaon, he value funcon can be expreed a: ( ) ( ) [ ] ( ) ( ) /,, /, S A a x a a P a r x a r E x V γ γ Theorem 3.1 Le r be he reward vecor whoe elemen are ( ) ( ) a A a r a q,, and V (he ame noaon of he value funcon) he value vecor whoe elemen are ( ) V. The ze of r and V equal o he number of ae. The marx expreon of he value funcon V hen: ( ) γ r P I V 1 Proof We have from he prevou remark ha ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) / /,, /,, S S A a S A a r P P a q a r a a P a r V γ γ γ ( ) P 0 / he probably of gong from ae o ae ' n me ep. Ung Chapman-Kolmogorov equaon [90], ha for any ' uch ha 0 < ' <, ( ) ( ) ( ) S P P P / / / A he pace ae fne, he -ep ranon probably compued a he 'h power of he ranon marx [90]. So ( ) P 0 / he elemen of he marx P. The expreon of he value funcon become hen: ( ) ( ) + 0 r P V γ Now he marx γ P a ochac marx and all her egenvalue have a complex modulu lower han 1 < γ. Therefore he marx γp I nverble and nvere

120 1 ( ) + I γ P 0 γ P The marx expreon of he value funcon hen 1 V I γp r ( ) ( ) ( ) Theorem 3.2: Bellman equaon If S and A are fne e, hen V * he unque oluon of he equaon V LV Proof To prove he exence of he oluon, we ue he conracon mappng heorem whch nvolve Banach pace. Theorem: conracon mappng heorem Le B be a non-empy Banach pace (.e. complee normed vecor pace) and T be a conracon mappng on B (.e. here a nonnegave real number 0 λ < 1 uch ha x, y B Tx Ty λ x y ) hen ) The map T adm one and only one fxed pon x * * * n B (h meantx x ). n+ 1 ) For any arng pon x 0 B he erave equence{ x n }, defned by xn+ 1 Txn T x0, converge o he pon x *. The pace Ω wh he norm max, gven n defnon 3.5, a complee normed vecor pace. We now how ha he DPO operaor a conracon mappng onω. For ha, conder U and V wo value funcon n Ω and a ae n S. * Aume ha LV ( ) LU ( ). Le a ( ) ( arg max r(, a) + γ p /, a V ). I follow a A S from he defnon of he DPO operaor ha 0 LV LU LV LU ( ) ( ) ( ) ( ) r(, a r(, a γ γ S S p p γ V U * ) + γ p( /, a ) V ( ) max r(, a) + γ p( /, a) U ( ) a A S S * * * ) + γ p( /, a ) V ( ) r(, a ) γ p( /, a ) U ( ) S S * ( /, a )( V ( ) U ( )) * * * ( /, a ) V U Therefore LV LU max LV ( ) LU ( ) γ V U one fxed pon. S. The mappng L adm hen only 102

121 103 We how now ha he fxed pon exacly V *, defned a ( ) ( ) S V V Π max *. We wan o how ha f LV V hen V V *. Le V uch ha LV V. We have hen { } V P r P V r V γ γ + + Π Π max. If we apply he nequaly V P r V γ + over V n me, we oban V P r P V P r P I P V r P r P V r V n n n k k k γ γ γ γ γ γ γ ) ( ) ( The rgh hand erm equal V P r P V n n n k k k γ + γ +. Therefore γ γ r P V P V V n k k k n n + The erm γ γ r P V P n k k k n n + end o 0 for n large enough becaue V V P n n n γ γ and ( ) { } a r r P A a S n n k k k, max 1, + γ γ γ. Th lead o he nequaly V V for every Π, and n parcular for V Π max arg *. Hence, V V V LV V Π max * Converely, conder V uch ha LV V. Le * be he polcy ha maxmze he value funcon over all polce. We have hen V P r P V P r P r V P r V n n n k k k * * * * * * * * * ) ( γ γ γ γ γ Then * * * * γ γ r P V P V V n k k k n n + The ame way a n he prevou cae, he hand rgh erm converge o 0 for n large enough. Hence * V V LV V. By combnng boh cae, we oban ha * V V LV V : every oluon of he Bellman equaon necearly equal he opmal value funcon V *.

122 Appendx B: LTE nerference model Sarng from he nerference coordnaon cheme, preened n econ 5.3.2, we aume ha he pecral band compoed of C reource block, one hrd of he band reerved for he cell edge uer and he re for cell cenre uer. The reource allocaon made accordng o uer' poon. Le L h be he pah lo hrehold eparang uer from he wo dfferen ub-band. A uer wh a pah lo hgher han L h graned reource block n he cell-edge band and oherwe he ge reource n he cell cenre band. The enb ranm power n each cell-edge reource block equal he maxmum ranm power P. To reduce nercell nerference, he enb ranm power n he cell-cenre band mu be lower han P. Le ε P (whereε < 1) be he ranm power n he cell-cenre band. The nerference hould be deermned for wo dfferen uer accordng o her poon: he cell-cenre uer and he cell-edge uer. Le m c and m e be wo uer conneced o a cell k. he moble m c ue he cenral band wherea m e ue he cell-edge band. Le Λ denoe he nerference marx beween cell, where he coeffcen Λ(,j) equal 1 f cell and j ue he ame cell-edge band and zero oherwe. For cell-edge uer m e, he nerference come from uer n he cell cenre of he cloe adjacen cell and from he cell-edge uer n oher cell. The moble m e conneced o he cell k and ung one reource block n he cell-edge band, receve an nerferng gnal from a cell equal I G c e, m e ( 1 Λ ( k, ) ) β εp + Λ ( k ) β P ) (B.1) L, m, e where P he downlnk ranm power per reource block of he cell. G, m e and L, are me repecvely he anenna gan and he pah lo beween cell and he moble aon m e. The c facor β (repecvely β e ) he probably ha he ame reource block n he cell-cenre band (repecvely he cell-edge band) ued a he ame me by anoher moble conneced o he cell. Snce he analy conder a long me cale (of he order of econd), he nerference c e averaged. So, he facor β he percenage of uer ung cell-cenre band and β he percenage of hoe ung cell-edge band β β occuped reource block n cell cener band oal capacy of cell cener band, m e c # c e # e occuped reource block n cell edge band oal capacy of cell edge band M C M 2C

123 M c and M e are he number of reource block ued n he cell cenre and cell edge repecvely, and he um M + M he oal number of reource block ued n he cell. c Le χ be he load of cell gven by e M c + M e χ (B.2) C Defne he facor α a he proporon of raffc erved n he cell-edge α M M + M. The facor β and β become repecvely band, ( ) e c e c e β c ( 1 α )( M + M ) 3( 1 α ) 3 c e χ 3α 2C ( M + M ) e c e β 3 C α χ Remark I very hard o fnd an exac expreon of he facor α. However, baed on he aumpon ha:. he cell approxmaed by a crcle and he enb locaed n he cell cenre,. he raffc unformly drbued n he cell and. he effec of hadow fadng negleced he facor α can be approxmaed by 1 α max,1 3 L h L max 2 / γ where L max he maxmum pah lo defnng he cell urface. Proof Aume ha a cell can erve uer drbued unformly n a crcle wh a radu R max. The radu of he nner crcle erved by he reource block of he cell-cenre band denoed R h. Then R max and R h are deermned by he propagaon model a: 2 L 1/ γ max R max ; l0 R L l 0 1/ γ h h. 105

124 106 Wh he unform raffc aumpon and f we don' ake no accoun he lm of phycal reource (maxmum of one hrd of he capacy can be agned o he cell-edge uer), he facor α he rao beween he area of he cell-edge and he oal area of he cell. Hence γ α 2 / max 2 max 1 1 L L R R h h In he followng, α calculaed numercally ung he relaon ( ) e c e M M M + α a menoned above. Baed on he prevou aumpon, he expreon of equaon (5.1) become ( ) ( ) ( ) ( ) Λ + Λ m e m m k k L P G I e e α ε α χ, 2 1, 1 3,,, (B.3) The erm ( ) ( ) ( ) ( ) ( ) + Λ Λ Λ e k k k α ε α, 2 1, 1 3, ~ can be nerpreed a a new nerference marx, denoed here a he fcve nerference marx for cell-edge uer. The oal nerference perceved by uer m e he um of all nerferng gnal Λ k m m e e m e e L P G k I,, ), ( ~ χ (B.4) For he cell-cenre uer m c, he nerference come from uer n he cell-edge and cell-cenre of cloe adjacen cell and alo from he cell-cenre and cell-edge uer n oher cell. Smlarly o he cell-edge uer, he moble m c conneced o he cell k and ung one reource block n he cellcenre band, receve an nerferng gnal from a cell equal o ( ) ( )( ) ( ) ( ) m c m c e c m L G P k P P k I c c,, 2 1,,, 1 ε β β ε β Λ + + Λ (B.5) The oal nerference hen Λ k m m e c m e c L P G k I,, ), ( ~ χ (B.6) where he fcve nerference marx n he cell-cenre band gven by ( ) ( ) ( ) + Λ + Λ Λ ε α α ε α 2 1, 2 1, 1 3 ), ( ~ 2 1 c k k k (B.7)

125 Appendx C: Nework yem level mulaor The mul-yem mulaor archecure depced n fgure C.1, whch how he man block repreenng he yem funconale and her neracon. The mulaon nvolve cooperaon beween hee block accordng o he nvegaed cenaro and confguraon. In a mul-yem cenaro, JRRM nerac wh he mulaor core and wh he RRM of each yem or RAN (Rado Acce Nework). In uch a cenaro, he auo-unng can be appled ulzng flered KPI. The raffc generaon and he mulaor core and a lea one of he RRM module are requred o run a mulaon wh one RAN. The auo-unng proce may be appled o one yem and n h cae, he JRRM funconale are neural,.e. he JRRM block ranparen. The ypcal arrval procee, a well a he packe lengh drbuon of he raffc, are uppored by he raffc generaor block. In addon, he raffc generaon module can be fed by meauremen ung obervaon ool. An objec-orened archecure ulzed o mplemen he mulaor block. Generc objec are developed wh flexble exenbly n each yem. In addon, each module or algorhm can be ealy replaced by an equvalen module: For example he admon conrol algorhm n a yem can be replaced by anoher admon conrol algorhm whou modfyng he re of he mulaor. Th feaure parcularly aracve for eng new RRM algorhm and mobly cheme. Meauremen Daabae Auo-unng Opmzaon Engne Traffc generaor JRRM Envronmen and Nework nfrarucure Smulaor Core RRM GERAN RRM UMTS RRM WLAN Calbraon KPI calculaon Engne Fgure C.1. Man bloc of he mul-yem mulaor archecure The em-dynamc mulaor allow a monorng he me evoluon of he nework. To acheve a fa compuaon me requred n he dynamc paradgm, he mulaor perform correlaed napho o accoun for he me evoluon of he nework wh a me reoluon, Tc, of he order 107

126 of a econd. A general cheme decrbng he nework evoluon beween wo correlaed napho depced n fgure C.2. A each napho, he new uer poon (due o nra- and ner-ran mobly wh dfferen peed) are deermned, and qualy ndcaor are compued a n ac mulaor (power, nerference, ec). Beween wo napho, arrval and deparure of uer can occur and he correpondng aon load, neceary for admon conrol, are updaed. The nework performance ac for each RAN deermned every Tc. Curren Syem RAN 1, RAN 2 Arrval and deparure of moble n RAN Admon conrol RAN load updae Syem updae Performance evaluaon and mobly: RAN RAN RAN RAN k + T c Fgure C.2. Tme evoluon of he mul-yem mulaor A he end of each me nerval, he mulaor perform he followng operaon for each MS: The raffc ranmed n he nework compued New moble poon deermned n a mehed urface nework. The MS movemen mplemened accordng o everal cenaro coverng ypcal moble veloce and moble orenaon. Rado condon of each moble are updaed (SIR, Power, SNR, ec.) Mobly beween yem execued f requeed. Beween wo me ep,.e. durng he perod T c, he followng even occur: Arrval and deparure of moble. Execuon of he CAC algorhm a each MS arrval. Recordng he volume of he raffc ranmed for he ougong moble. For he UMTS ub-yem mulaor, he followng procedure are mplemened: CAC: Th algorhm emae he load ncreae (boh n UL and DL) ha he accepance of a new moble wll caue n he rado nework. A Targe Load (TL) pecfed o ha when he nework load exceed TL, allow no more admon n 108

127 order o preerve a good qualy for moble whch are already n he nework. The reource are hared beween real-me (RT) and non-real-me (NRT) raffc wh dfferen mplemenaon: A reerved band allocaed o RT raffc and a econd one hared beween RT and NRT raffc, Two ndependen band are reerved for RT and NRT. Load conrol: To decde wheher he nework n congeon or no, a creron nroduced baed on a load hrehold defned above he TL. If he load of he nework ncreae above h hrehold for ceran duraon, new moble are blocked and ceran nerferng moble are dropped by he load conrol mechanm. Power conrol: The power conrol mplemened by lmng he power of he moble o he mnmum power requred o manan a gven SIR for requred yem performance. Therefore, he power of each moble calculaed o ha he ranmed power reache he arge SIR. Macro-dvery: he of handover mechanm mplemened n he mulaor baed on he 3 even: even1a, even1b and even1c. Selecon and re-elecon: he algorhm for he elecon/ reelecon procedure mplemened n he mulaor are hoe pecfed n he 3GPP andard for 3G nework and decrbed n he econd chaper of he he. Wh repec o WLAN ub-yem mulaor, bac RRM algorhm are mplemened. Oher mechanm are alo developed for he ner-yem handover,.e. WLAN-UMTS ner-yem mobly and admon conrol, n he cae of non real me raffc. The bac RRM algorhm are: Selecon of modulaon cheme: h algorhm, baed on he SNR value, elec he modulaon o ha he packe error rae reman below a gven hrehold. CAC: when a new MS eek acce o he nework, he algorhm emae he nework load and he hroughpu o be allocaed o afy h reque. Baed on h expeced hroughpu, he MS eher admed or blocked ou. Phycal b rae adapaon: afer admon o he yem, he rado condon of a MS vary due o mobly. Th een n erm of he flucuaon of he SNR value a each ranmon. To maxmze he yem performance and o avod he QoS degradaon of oher MS, he b rae adapaon algorhm execued o adap he phycal b rae of he MS,.e. he modulaon cheme. 109