THe recent development of wireless technologies has

Transcription

1 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION 1 Mathematcal Modelng for Network Selecton n Heterogeneous Wreless Networks A Tutoral Lusheng Wang and Geng-Sheng (G.S.) Kuo Abstract In heterogeneous wreless networks, an mportant task for moble termnals s to select the best network for varous communcatons at any tme anywhere, usually called network selecton. In recent years, ths topc has been wdely studed by usng varous mathematcal theores. The employed theory decdes the objectve of optmzaton, complexty and performance, so t s a must to understand the potental mathematcal theores and choose the approprate one for obtanng the best result. Therefore, ths paper systematcally studes the most mportant mathematcal theores used for modelng the network selecton problem n the lterature. Wth a carefully desgned unfed scenaro, we compare the schemes of varous mathematcal theores and dscuss the ways to beneft from combnng multple of them together. Furthermore, an ntegrated scheme usng multple attrbute decson makng as the core of the selecton procedure s proposed. Index Terms Network selecton, heterogeneous wreless networks (HWNs), utlty theory, multple attrbute decson makng (MADM), fuzzy logc, game theory, combnatoral optmzaton, Markov chan. I. INTRODUCTION THe recent development of wreless technologes has totally revolutonzed the world of communcatons. Multple technologes are evolvng smultaneously towards provdng users wth hgh-qualty servces of broadband access and seamless moblty. On one hand, wreless wde area networks (WWANs) evolve from GSM to UMTS and beyond 3G, provdng wde coverage and good moblty capabltes. On the other hand, a seres of standards of wreless local area networks (WLANs), ncludng IEEE a, IEEE b, IEEE g, IEEE n, etc., have been establshed for local-area hgh-speed economc wreless access. To complement them, wreless personal area networks (WPANs), e.g., Bluetooth and Zgbee, and wreless metropoltan area networks (WMANs), e.g., WMAX, are developed for short-range and metropoltan coverages, respectvely. All the above networks have been deployed wth coverage overlappng one another, hence formng a hybrd network for wreless access, whch s usually called heterogeneous wreless networks (HWNs). L. Wang s wth Moble Communcatons System Department, Insttute Eurecom, Sopha Antpols, France (e-mal: lswang.enst@gmal.com). G.S. Kuo s wth Natonal Chengch Unversty, Tape, Tawan (e-mal: gskuo@mal.com). Manuscrpt receved March 12, 2011; revsed September 19, 2011; accepted December 4, The work of the frst author was supported manly by France Telecom Orange Labs from EC FP7 EuroNF Project and partly by Insttute Eurecom. The work of the second author was supported by the Natonal Scence Councl of Tawan under Grant NSC H MY3. Dgtal Object Identfer /00$00.00 c 0000 IEEE To access the Internet through HWNs, current termnals, e.g., laptops and cellphones, are usually nstalled wth multple wreless access network nterfaces. One type of termnals wdely used nowadays s those wth multple nterfaces but no functonalty to support IP moblty or multhomng, called mult-mode moble termnals. The other s wth IP moblty and multhomng functonaltes, called mult-homed moble termnals. Moblty means that a termnal can swtch between networks wthout breakng on-gong communcatons. Multhomng means that a termnal has multple IP connectons to one or multple networks smultaneously. Mult-homed termnals use multple nterfaces to share load for the same sesson and support sesson contnuty wth low (or no) packet loss durng moblty or lnk break. By contrast, mult-mode termnals can only select and use one nterface for certan sesson at a tme. Both mult-mode and mult-homed termnals requre always to rank the access networks and select the best at any tme anywhere, whch s well known as always best connected (ABC). ABC brngs plenty of advantages to users. Wth ABC functonalty, termnals select approprate access networks to ft for varous QoS requrements of applcatons; termnals avod selectng a network wth hgh traffc load for avodng congeston; termnals predct networks avalablty so that they do not connect to networks whch dsappear soon; and termnals mnmze sgnallng costs by usng network selecton and handover decson strateges specfcally for ths purpose. Moreover, ABC benefts operators. Snce ABC has the feature of assstng the assgnment of traffc load to multple networks, operators maxmze the utlzaton rate of the resources of the networks they operated, hence maxmzng revenue. Accordng to network selecton strateges, operators analyze and decde the number of WF access ponts they should deploy to attract users to WLANs. Fnally, ABC s sutable to synthetcally consder users and operators benefts, so that a wn-wn partnershp can be acheved. ABC contans many necessary components [1], such as network dscovery, network selecton, handover executon, authentcaton, authorzaton and accountng (AAA), moblty management, profle handlng, content adaptaton, etc., n whch network selecton s a key component and wll be extensvely dscussed n ths paper. In recent years, a large number of research works have dscussed the selecton of the best network. Among them, dfferent mathematcal theores have been used for modelng ths problem. Although two survey papers on ths topc [2], [3] have been publshed, they were not focused on the mathematcal theores used to model

2 2 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION TABLE I NETWORKS AND SELECTED ATTRIBUTES IN THE UNIFIED SCENARIO Bandwdth Prce Cell radus Securty Power consumpton Traffc WWAN /100 X WMAN /100 X WLAN /50 X WPAN /1000 X TABLE II SELECTED PROPERTIES OF THE 16 USERS IN THE UNIFIED SCENARIO User No Conversatonal Applcaton User Termnal Streamng Interactve Background Money frst Qualty frst Battery frst Moblty frst ths problem. Based on our study, the mathematcal model used for representng the problem s the frst thng and the most mportant thng we should consder when desgnng a network selecton strategy. It decdes the am of optmzaton, the utlzaton of dfferent parameters, and the performance of the selecton strategy. Therefore, to fll out ths blank, we conduct a serous survey and provde a systematc tutoral on mathematcal theores for modelng the network selecton problem. Throughout ths paper, we use a unfed scenaro to help explan schemes usng dfferent mathematcal theores. On the network sde, we consder 4 types of avalable networks (.e., WWAN, WMAN, WLAN and WPAN) and 6 attrbutes (.e., bandwdth, prce, cell radus, securty, power consumpton and traffc), as gven n Table I. These attrbutes are carefully selected, so that there s upward attrbute e.g., bandwdth, downward attrbute e.g., prce, dynamc attrbute e.g., traffc, termnal-related attrbute e.g., power consumpton, applcaton-related attrbute e.g., securty and moblty-related attrbute e.g., cell radus. Note that one attrbute could have multple of these features. Moreover, we desgn WMAN as a domnant alternatve of WWAN, so that we could clearly see the load balancng feature of the schemes wth dfferent mathematcal theores. On the user sde, we consder 4 types of applcatons wth dfferent QoS requrements ncludng conversatonal, streamng, nteractve and background [4]. For each applcaton type, we consder 4 users wth dfferent user preferences (.e., money frst and qualty frst) and dfferent termnal propertes (.e., battery frst and moblty frst). Totally, there are 16 users wth dfferent user-sde features, as summarzed n Table II. VHO represents handover between dfferent types of access technologes, whch s needed not only for connectvty reason but also for other ones, such as user preference and network load balancng. In the lterature, VHO decson s sometmes confused wth the term network selecton, so n ths paper, we strctly dstngush the two terms: network selecton s to rank networks and fnd the best one, whle VHO decson s to decde whether t s worth the handover to the best network or a network better than the current one. VHO decson s not to smply check whether the dfference between the two networks s larger than the VHO cost. In fact, ths decson takes nto account the predcted nformaton of many parameters as long as they are predctable, ncludng the expected tme pont that a better network wll be avalable, the average duraton that a better network can last, the probablty densty functon of a better network s dwellng tme, the utltes of networks, etc. However, snce the subject of ths tutoral s network selecton, we are not gong to dscuss too much on VHO decson. The rest of ths paper s organzed as follows. From Sectons II to VII, we systematcally dscuss the exstng studes on network selecton usng utlty theory (cost functon), multple attrbute decson makng, fuzzy logc, game theory, combnatoral optmzaton, Markov chan, respectvely. In Secton VIII, we compare schemes usng dfferent mathematcal theores, dscuss the ways to combne multple of these theores together, and propose an ntegrated scheme n the end. Secton IX concludes the paper. Fnally, Secton X and Secton XI provdes the notatons and the glossary. II. UTILITY THEORY (COST FUNCTION) For makng a decson, utlty refers to the satsfacton that a goods or servce provdes to the decson maker [5]. An assocated term s utlty functon whch relates to the utlty derved by a consumer from a goods or servce. Dfferent consumers wth dfferent user preferences wll have dfferent utlty values

3 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 3 Utlty(x) Fg. 1. exponental 1 e ax, a=20 logarthmc ln(1+ax)/ln(1+a), a=5 lnear sgmodal (x/0.5) a /[1+(x/0.5) a ], a=20 sgmodal (x/0.5) a /[1+(x/0.5) a ], a=5 sgmodal (x/0.75) a /[1+(x/0.75) a ], a=20 exponental e a(x 1), a= x Typcal utlty functons. for the same product. Thus, the ndvdual preferences should be taken nto account n the utlty evaluaton. A. Utlty functons n network selecton Utltes can be classfed nto monotonc utltes and nonmonotonc ones. The utlty s sad to be monotonc f the measure of satsfacton assocated wth the attrbute shows a monotonc ncrease and decrease wth an ncrease n attrbute value. Otherwse, t s sad to be non-monotonc. Normally, monotonc utltes are used, except f the attrbute s consdered as the nomnal-the-best. For a nomnal-the-best attrbute, nstead of consderng the best (ether the largest or the smallest) as the most desred network, the one that s closest to the servce requrement s preferred [6]. When evaluatng the utlty of an attrbute, we should dstngush between the upward and downward attrbutes. The attrbutes of whch the hgher preference relaton s n favor of the hgher value are called upward attrbutes. Conversely, the downward attrbutes encompass varous costs. Gven an attrbute, ts utlty can be calculated based on certan utlty functon. And, the utlty functon of one attrbute could be dfferent from that of others. Some examples of common utlty functons are shown n Fg. 1. It s mportant to select the sutable utlty functon for each attrbute. Sgmodal utlty functon s consdered to be sutable for the network selecton problem [7], but the parameters n the sgmodal functon mght be dfferent to ft for dfferent attrbutes features. Durng the network selecton procedure, we consder multple attrbutes together, so the utltes of multple attrbutes are combned as a total utlty. It has been ponted out that a vald form to combne these attrbutes together satsfes the followng requrements [7]: U u j 0 lm U = 0, j = 1,..., M u j 0 (1) lm U = 1 u 1,...,u M 1 where U s the total utlty of all the attrbutes and u j s the utlty of attrbute j. M denotes the number of attrbutes throughout ths paper. Cost functon s a measurement of the cost caused by usng certan network. Usually, the cost of a network can be consdered as the nverse of ts utlty, but the form of ths nverson s related wth the way to combne multple attrbutes. For example, f these attrbutes are summed up, the total cost s calculated as the cost mnus the utlty. A general form of cost functon for the network selecton problem was gven n [8], whch ntegrates a large number of attrbutes, ther weghts, and furthermore, network elmnaton factors gven by F = k ( j ǫ k j ) j [fj k (wk j )N(uk j )], (2) where N(u k j ) represents the normalzed utlty of applcaton k n network n terms of attrbute j. fj k(wk j ) s the weghtng functon of attrbute j for applcaton k. ǫ k j s the network elmnaton factor, ether 1 or nfnte, to reflect whether current network condtons are sutable for requested applcatons. For example, f a network cannot guarantee the delay requrement of certan real-tme applcaton, ts correspondng elmnaton factor wll be set to nfnte. Thus, the correspondng cost becomes nfnte, whch elmnates ths network. One study that s worth mentonng s the usage of the consumer-surplus concept of mcroeconomcs n [9]. Users always search for cost effectve solutons to meet ther expectatons. If the prce s less than the value the user s wllng to pay, he saves money. Consumer-surplus represents the dfference between the monetary value of the data to the user and ts actual prce, so the network wth the best predcted consumer-surplus, whch s also predcted to meet the servce completon deadlne, wll be selected. B. Attrbutes n network selecton A lot of studes model the network selecton ssue wth cost or utlty functons, but they may consder dfferent attrbutes and measure them n dfferent manners. A summary of attrbutes and ther usage n dfferent papers s provded n Table III. For types of attrbutes, we frst classfy them nto upward and downward attrbutes, then statc, dynamc and sem-dynamc attrbute. Sem-dynamc attrbutes are those that are not totally statc but not qute dynamc ether. For example, bandwdth s sometmes used statcally as the total bandwdth of each network, but sometme used dynamcally as the average bandwdth a user obtans. Bt error rate (BER), jtter and servce completon tme are changeable along wth the envronment and the network condton, but t s dffcult to dynamcally evaluate ther nstantaneous values for network selecton, so they are classfed as sem-dynamc attrbutes. Moreover, we also consder some other features of attrbutes, such as moblty-related, QoS-related, termnal-related and nter-network. For lsts of references, consderng that every study on network selecton wll use one or multple attrbutes as decson crtera and some key attrbutes are even used by most studes on ths ssue, so t s tedous to provde complete lsts for all the attrbutes. Instead, Table III just ams to lst

4 4 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION TABLE III KEY ATTRIBUTES AND THEIR UTILITY FUNCTIONS Attrbute Types References Utlty functons Bandwdth upward/sem-dynamc/qos-related [7], [8], [10], [20], [21], [23], [24], [28], [30], [37], [38], [46], [51] lnear, logarthmc, sgmodal Cell radus (dameter) upward/statc/moblty-related [38] lnear Securty upward/statc/qos-related [10], [21], [23], [24], [51] lnear, sgmodal Battery upward/dynamc/termnal-related [21], [22], [28] lnear SNR/SIR upward/dynamc/qos-related [21], [22] lnear RSS upward/dynamc/qos-related [11] [13], [21], [28], [51] lnear Prce downward/statc [7] [10], [13], [21], [23], [24], [28], [34], [38] lnear, logarthmc VHO sgnalng cost downward/statc/moblty-related/nter-network [12], [24], [54] lnear VHO latency downward/statc/moblty-related/nter-network [12], [27] lnear HHO sgnalng cost downward/statc/moblty-related [12], [54] lnear HHO latency downward/statc/moblty-related [12], [38] lnear Handover falure probablty downward/statc/moblty-related [27] lnear Interrupton probablty downward/statc/moblty-related [27] lnear Sze of unsent messages downward/statc/moblty-related [27] lnear Traffc downward/dynamc [7], [11], [24], [34], [37] lnear, sgmodal Power consumpton downward/statc/termnal-related [24], [38], [51] lnear BER downward/sem-dynamc/qos-related [21], [23], [24] lnear, sgmodal Delay downward/sem-dynamc/qos-related [20], [21], [23], [51] lnear, sgmodal Packet loss downward/sem-dynamc/qos-related [20], [23] lnear, sgmodal Jtter downward/sem-dynamc/qos-related [20], [21], [23], [24], [51] lnear, sgmodal Response tme downward/sem-dynamc/qos-related [23] lnear Servce completon tme downward/sem-dynamc/qos-related [9] lnear, polynomal, exponental some most typcal examples of each attrbute. For utlty functons used n the lterature, most studes that do not specfcally dscuss utlty functons could be consdered as usng lnear utlty functons. Whle n some recent studes, polynomal, logarthmc, exponental and sgmodal utlty functons are utlzed for some attrbutes, whch are summarzed n ths table. In the above presentatons, we dscussed utlty functons for varous attrbutes. To avod a potental msunderstandng, we would lke to pont out that utlty functon for a certan attrbute could be totally dfferent n dfferent scenaros. For example, the utlty of bandwdth should jump to a fxed value after certan thresholds for voce and vdeo applcatons, but knd of lnearly ncrease for data applcaton [13]. If sgmodal functons are used, the parameter a, as shown n Fg. 1, should be large for voce and vdeo applcatons whle small for data applcatons. For voce and vdeo applcatons, the md values, correspondng to the thresholds, should be also dfferent. Moreover, t s mportant to state clear that other studes on the network selecton ssue could also evaluate networks based on utlty/cost functons whch combne multple attrbutes. However, those studes focus on other mathematcal models, whch wll be presented n later sectons. C. Case study We consder the unfed scenaro presented n Secton I wth Tables I and II. Snce t would be unfar by assumng dfferent networks wth dfferent traffc condtons, we assume that they have the same traffc condton, whch means that the attrbute traffc s not consdered n ths case study. Based on the above studes, sgmodal utlty functons wth dfferent confguratons of md value and parameter a, as shown n Fg. 1, are used for dfferent attrbutes under the cases of dfferent user-sde propertes. For example, user 5 requres streamng applcaton whle user 1 requres conversatonal applcaton, so the md value n the sgmodal utlty functon of bandwdth

5 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 5 TABLE IV OBJECTIVE AND SUBJECTIVE WEIGHTING METHODS Category Calculaton Entropy Objectve weghtng w j = 1 1 N [ ] xj ln(x ln N =1 j ) N Varance Objectve weghtng w j = =1 (x j x j ) 2 /N x j, x j = 1 N N =1 x j Egenvector Subjectve weghtng (B λi) w = 0 M M Weghted least square Subjectve weghtng mn Z = =1 j=1 (b jw j w ) 2 M, s.t. =1 w = 1 TRUST Subjectve weghtng w = e (d I) R s much larger for user 5 than for user 1; user 7 prefers better servce, so a n the sgmodal utlty functon of prce can be small but that of bandwdth should be large. In other words, sgmodal utlty functons could be dfferent for dfferent users and dfferent attrbutes, so there are 5 16 sgmodal utlty functons. For the sake of concseness, we are not gong to lst them. In order to promnently reflect the effect of the sgmodal utlty functons, we smply sum the utltes of these attrbutes wth equal weghts. Moreover, we use the Enhanced Max- Mn method n Table V to normalze the values of attrbutes for all the case studes throughout ths paper. We want to menton that, wth Enhanced Max-Mn method, the utltes of the best and the worst networks on any attrbute wll be stretched close to 1 and 0, respectve. Then, f the utltes are gong to be summed up wth equal weghts as we sad above, multple trval attrbutes could conceal the mportance of the key attrbute and domnant the fnal decson. To avod ths ptfall, we compress all the utltes from [0, 1] to [0.1, 0.9] and set the md value of sgmodal functon to 0.01 (or 0.99) when the attrbute s trval (or dramatcally mportant). Network selecton results of the 16 users are gven n Table VII, together wth the results from schemes usng other mathematcal theores for comparson. III. MULTIPLE ATTRIBUTE DECISION MAKING Multple attrbute decson makng (MADM) refers to makng preference decson over the avalable alternatves that are characterzed by multple (usually conflctng) attrbutes. MADM s a branch of multple crtera decson makng (MCDM) whch also ncludes multple objectve decson makng (MODM). MODM problems nvolve desgnng the best alternatve gven a set of conflctng objectves, whch creates a product n the desgn process. For example, automoble manufacturers want to desgn a car that maxmzes rdng comfort and fuel economy and mnmzes producton cost. Apparently, network selecton does not create any physcal product but only makes a decson, so MADM s more sutable for ths problem. A. MADM bascs MADM problems have several common characterstcs [14]: Alternatves: a fnte number of alternatves are screened, prortzed, selected and/or ranked for makng the fnal decson. The term alternatve s synonymous wth opton, polcy, acton, canddate, etc. Multple attrbutes: the decson maker does consder multple attrbutes of these alternatves. The term attrbute can be referred to as goal, crteron, property, characterstc, etc. Decson matrx: a MADM problem can be concsely expressed n a matrx format, where columns ndcate attrbutes and rows ndcate alternatves. Thus, a typcal element x j of the matrx ndcates the value of the th alternatve wth respect to the jth attrbute. Attrbute weghts: dfferent decson makers mght focus on dfferent aspects when rankng the alternatves, so weghts must be calculated to represent multple attrbutes relatve mportance. Table IV gves some common weghtng methods ncludng objectve and subjectve methods. The objectve weghts are calculated drectly based on the relatve dfference between attrbutes, gven by w j for attrbute j. Then, the objectve weghts are obtaned as the normalzed values of w j. By contrary, subjectve weghts w are usually calculated based on the decson maker s par-wse comparson between all the attrbutes, gven by b j as the comparson value between the th and jth attrbutes and B as the matrx contanng all the comparson values. Moreover, for the egenvector method n the table, λ s the egenvalue and I s an dentty matrx. N denotes the number of networks throughout ths paper. However, these tradtonal methods to calculate subjectve weghts do not work well for the network selecton problem snce ts par-wse comparson process s slow and not automatc. Therefore, we proposed a TRgger-based automatc Subjectve weghtng (TRUST) method [15] to calculate subjectve weghts, as shown n the weghtng module of Fg. 6. Snce some events can trgger the network selecton procedure, there should be some relatonshp between these events and selecton results. Our method uses a mappng pot to store ths relatonshp n order to calculate the subjectve weghts. Two parameters are stored n the mappng pot and used for the calculaton of subjectve weghts. One s a E-by-M matrx R representng the relatonshp between events and network attrbutes, where E s the number of events and r j n the matrx represents the strength of the effect from the th event to the jth attrbute, e.g., the event speed up ncreases the

6 6 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION TABLE V NORMALIZATION METHODS FOR ATTRIBUTES IN NETWORK SELECTION Normalzaton Functon Max-Mn v j = (x j mn(x j ))/(max(x j ) mn(x j )) N Square root v j = x j / =1 x2 j N Sum v j = x j / =1 x j 1 x j max(x j ) /(max(x j ) mn(x j )) for upward attrbutes Enhanced Max-Mn v j = 1 x j mn(x j ) /(max(x j ) mn(x j )) for downward attrbutes 1 x j Λ j /max{max(x j ) Λ j, Λ j mn(x j )} for nomnal-the-best attrbutes weght of moblty-related attrbutes. The other s a 1-by-E vector e representng the weghts of events, whch can be calculated n advance or obtaned from the operator durng the ntaton of the moble termnal. Fnally, the subjectve weghts of network attrbutes can be calculated as shown n Table IV, where d s a 1-by-E bnary vector denotng true or false of the trgger events. Normalzaton: dfferent attrbutes have dfferent measurement unts, so normalzaton s treated as a necessary step of network selecton. There are several methods of normalzaton, compared n Table V. For a gven attrbute j, x j represents the value of the th network n terms of ths attrbute, and v j represents ts normalzed value. The enhanced Max-Mn method consder three groups of network-sde attrbutes,.e., upward, downward and nomnal-the-best, where Λ j represents the nomnal value of attrbute j. There are two dfferences between Max-Mn and enhanced Max-Mn methods: frst, the latter consders the nomnal-the-best group; second, the latter adjusts downward attrbutes nto upward attrbutes. For the sake of the second dfference, the outputs of the enhanced Max-Mn method are all consdered as upward attrbutes, whle for the other three methods, we have to dstngush between upward and downward attrbutes whle combnng them together. For examples of the usages of these normalzaton methods, refer to [16] [19]. B. MADM algorthms n network selecton MADM algorthms can be dvded nto compensatory and non-compensatory ones [20]. Non-compensatory algorthms, e.g., domnance, conjunctve, dsjunctve or sequental elmnaton, are used to fnd acceptable alternatves whch satsfy the mnmum cutoff. By contrary, compensatory algorthms combne multple attrbutes to fnd the best alternatve. Most MADM algorthms that have been studed for the network selecton problem are compensatory algorthms, ncludng smple addtve weghtng (SAW), multplcatve exponental weghtng (MEW), gray relatonal analyss (GRA), Technque for Order Preference by Smlarty to an Ideal Soluton (TOP- SIS), ELmnaton Et Chox Tradusant la REalté (ELEC- TRE), etc. SAW s wdely used by most studes of the network selecton problem usng cost or utlty functons, generally gven by C SAW = M w j v j, (3) j=1 where w j represents the weght of the jth attrbute, and v j represents the adjusted value of the jth attrbute of the th network. MEW s to calculate the coeffcent by multplcatve operaton [7], [21], gven by C MEW = M j=1 v wj j. (4) (4) can be further modfed as CMEW = ln(c MEW) = M j=1 w j ln(v j ). Consderng the characterstc of the natural logarthm, the attrbute whose cost s close to 0 has larger mpact on the total cost than others. For example, Bluetooth s more often selected by MEW than by other algorthms due to ts low monetary and power costs. Another two MADM algorthms used for network selecton are TOPSIS [17], [22] and GRA [6], [23], whch both consder the dstance from the evaluated network to one or multple reference networks. Coeffcent of TOPSIS can be calculated as D α C TOPSIS = D β + Dα, (5) M where D α = j=1 w2 j (v j Vj α)2 and D β = M j=1 w2 j (v j V β j )2 represent the Eucldean dstances from the current network to the worst and best reference networks, respectvely. V α j and V β j represent the values of the jth attrbute of the worst and best reference networks, respectvely. Dfferent from TOPSIS, GRA uses only the best reference network to calculate the coeffcent, gven by 1 C GRA = M j=1 w j v j V β (6) j + 1. ELECTRE, another well-known MADM algorthm but dfferent from the above four algorthms, does not calculate certan coeffcent for network rankng. It contans the followng steps [16]: 1) dentfyng attrbutes of dfferent networks as a decson matrx;

7 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 7 AHP Structurng a herarchy of all the attrbutes Par-wse comparson of attrbutes and sub-attrbutes Calculatng weghts on each level of the herarchy Syntheszng the herarchy to get the weghts Informaton gatherng GRA Classfyng attrbutes Normalzaton of all the attrbutes Defnng an deal network Calculatng coeffcents of networks Fg. 2. An example of combnng MADM wth AHP-based subjectve weghtng. 2) defnng an deal network; 3) calculatng the absolute dfference between each network and the deal network; 4) normalzng the absolute dfference; 5) multplyng weghts of attrbutes; 6) calculatng concordance and dscordance matrces; and 7) makng decson based on concordance and dscordance matrces. Among them, the key step s 6), n whch concordance and dscordance matrces are calculated based on concordance and dscordance sets, denoted by C and D, respectvely. C kl contans the attrbutes on whch network k s better than network l, and D kl s nverse. Then, the elements n concordance and dscordance matrces are calculated as follows: c kl = j C kl w j v kj v lj j D d kl = kl (7) M. v kj v lj j=1 Among all the MADM algorthms, [7] ponted out that MEW s the only one that satsfes all the requrements ndcated by (1), whle [6] argued that GRA s more sutable than others n the scenaros when some attrbutes have nonmonotonc utltes. [21] showed that SAW, MEW and TOPSIS have smlar performance to all traffc classes, whle GRA provdes a slghtly hgher bandwdth and lower delay for nteractve and background traffc. [24] showed that MEW gves larger probablty to select WPAN than other algorthms due to ts multplcaton operaton. Moreover, t s easy to combne compensatory MADM algorthms wth the egenvector subjectve weghtng method based on analytcal herarchy process (AHP), such as the scheme shown n Fg. 2 [23]. AHP s a procedure to dvde a complex problem nto a number of decdng factors and ntegrate the relatve domnances of the factors wth the soluton alternatves to fnd the optmal one. For weghtng the attrbutes n a network selecton scheme, AHP structures attrbutes nto a herarchy. For example, [23] structures all the QoS-related attrbutes nto fve groups (.e., throughput, tmelness, relablty, securty and cost) and each group has one or multple attrbutes (e.g., delay, response tme and jtter are n the group of tmelness). Therefore, QoS s on the frst level, the fve groups are on the second level, whle attrbutes n each group are on the thrd level. Then, on each level n the herarchy, weghts are calculated based on certan weghtng method, e.g., those n Table IV. Fnally, weghts of dfferent levels are syntheszed to acheve the overall weght of each attrbute. Note that MADM s not the only mathematcal theory that combnes multple attrbutes together. Theores n the other sectons also prefer to combne multple attrbutes for decson, usng usually SAW. Moreover, weghtng and normalzaton are common operatons for schemes usng all knds of mathematcal theores, not only for MADM. We present them n ths secton snce they are manly studed n the scope of MADMbased network selecton. C. Case study We consder the unfed scenaro presented n Secton I wth Tables I and II. Smlar to the case study n Secton II, attrbute traffc s not consdered n ths case study. Based on the above studes, we choose the wdely used MADM algorthm, SAW, for ths case study. Enhanced Max-Mn method s used for normalzaton. Egenvector method s used for calculatng the subjectve weghts. For each user, a par-wse comparson matrx s obtaned by the decson maker based on user-sde propertes. For example, the par-wse comparson matrx of user 1 could be B = 1 1/ / / /7 1 1/ / Weghts are calculated as the egenvector of the above parwse comparson matrx correspondng to the largest egenvalue, gven by {0.0588, , , , }. Sometmes, the egenvector could be negatve, so we should always normalze the obtaned egenvector to avod treatng the worst network as the best. We can see from ths matrx that two attrbutes are key factors for the decson,.e., prce (as the user preference s money frst ) and power consumpton (as the termnal property s battery frst ). For the other three attrbutes, t s really dffcult for us to say whch one s the most mportant one, so we gve them equal weghts. For the sake of concseness, we are not gong to lst the par-wse comparson matrces for all the users, but we would lke to remnd that par-wse comparson matrces are dfferent from user to user and from scenaro to scenaro. Network selecton results of the 16 users are gven n Table VII, together wth the results from schemes usng other mathematcal theores for comparson. Notce that the selecton

8 8 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION Attrbutes Fuzzfer Hgh Low Large Small Hgh Low Fuzzy rule base Fuzzy nference engne Defuzzfer Rank Recurson module (neural network, kernal learnng, etc.) Fg. 3. A combned framework of fuzzy logc based network selecton. results by usng other MADM algorthms are qute close to SAW. For example, wth TOPSIS, the only dfference n the results s that user 6 selects WLAN nstead of WMAN. IV. FUZZY LOGIC Humans usually thnk n terms of lngustc descrptons, so gvng these descrptons some mathematcal form helps explot human knowledge. Fuzzy logc utlzes human knowledge by gvng the fuzzy or lngustc descrptons a defnte structure. A. Fuzzy logc bascs To understand well ths secton, t s necessary to know the followng concepts [25]: Fuzzy set: a fuzzy set s a class of objects wth a contnuum of grades of membershp, whch s characterzed by a membershp functon assgnng to each object a grade of membershp rangng between zero and one [26]. Fuzzy set s consdered as an extenson of the classcal noton of set. In the classcal set theory, the membershp of elements n a set s assessed n bnary terms, whch means ether belongs or does not belong to the set. By contrast, the fuzzy set theory permts the gradual assessment of membershp usng a membershp functon valued wthn [0, 1]. The classcal set s usually called crsp set n the fuzzy logc theory. Fuzzfer: the module to map a crsp pont nto a fuzzy set. Fuzzy rule base: the module consstng of a collecton of fuzzy IF-THEN rules. A typcal form of a rule s IF X 1 s F l 1 and... and X M s F l M, THEN Y s G l, (8) where l denotes the ndex of the rule n the fuzzy rule base, X j represents the jth nput varable, Y represents the output varable, and F l j and G l are correspondng fuzzy sets for X j and Y, respectvely. Fuzzy nference engne: the module whch uses fuzzy logc prncples to combne the fuzzy IF-THEN rules n the fuzzy rule base. Defuzzfer: the module to map a fuzzy set nto a crsp pont (the opposte of fuzzfer). Membershp functon: representng the degree of truth n fuzzy logc theory. B. Fuzzy logc n network selecton There are dfferent ways to use the fuzzy logc theory n a network selecton scheme: some studes use t as the core of the selecton scheme, some combne fuzzy logc wth MADM algorthms, whle some use the fuzzy logc wth recurson (neural network, kernel learnng, etc.). A very basc framework wthout combnng wth any other theory s used by [27] for fuzzy logc based network selecton, as shown n Fg. 3, elmnatng the recurson part. In ther scheme, three nput fuzzy varables are consdered (.e., the probablty of a short nterrupton, the falure probablty of handover to rado, and the sze of unsent messages), whle we could surely consder more attrbutes as nput fuzzy varables for network selecton. At the begnnng of the procedure, the fuzzy varables are fuzzfed and converted nto fuzzy sets by a sngleton fuzzfer. Then, based on the fuzzy rule base, the fuzzy nference engne maps the nput fuzzy sets nto output fuzzy sets by the algebrac product operaton. Fnally, the output fuzzy sets are defuzzfed nto a crsp decson pont. Many studes proposed schemes by combnng fuzzy logc wth MADM algorthms [2], [22], [28], coned fuzzy MADM. The dea s to use MADM for the fuzzy nterference engne and defuzzfer parts. Fuzzy MADM s partcularly nterestng for the case when some attrbutes cannot be precsely obtaned or when some attrbutes are better to be set wth fuzzness due to the complex HWNs envronment n an MADM scheme. Accordng to the data type of the alternatve s performance, fuzzy MADM can be categorzed nto three groups: data beng all fuzzy, all crsp, and ether fuzzy or crsp [22]. Snce some dynamc factors change frequently, the recurson s used to combne the latest nformaton wth prevous rankng result to obtan the latest rank. In the lterature, there are several proposals combnng fuzzy logc wth a recurson procedure. The recurson procedure can be a smple recurson wthout any further operaton or certan learnng procedure, such as neural network or kernel learnng, as shown n Fg. 3. [29] proposed a fuzzy logc based scheme usng smple recurson, whch consders the requrements of both operator and user. The rank produced by the fuzzy module s fed back to ths module, so that t could produce a new rank when some factors change. [30] combned the fuzzy logc wth neural network for network selecton. Elman neural network s used to predct the number of users usng certan network after the selecton and feeds t back to the fuzzfer. And, [31] proposed a scheme to combne the fuzzy logc wth kernel learnng for

9 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL Membershp Membershp Membershp Bandwdth Prce Cell radus Membershp 0.5 Membershp Securty Power consumpton Fg. 4. Membershp functons of dfferent attrbutes n the unfed scenaro. smlar purpose. C. Case study We consder the unfed scenaro presented n Secton I wth Tables I and II. For the same reason as the case studes n prevous sectons, attrbute traffc s not consdered n ths case study. We consder two fuzzy sets for each attrbute, e.g. bandwdth has large and small fuzzy sets. Thus, wth fve attrbutes, there are maxmum 2 5 fuzzy rules n the fuzzy rule base. For example, a basc fuzzy rule could be IF bandwdth s large & prce s low & cell radus s large & securty s hgh & power consumpton s low, THEN utlty s hgh. Membershp functon of each attrbute s carefully desgned based on the property of the attrbute, as shown n Fg. 4. For example, bandwdth s an attrbute whch has some knd of threshold to guarantee QoS, so the slope of ts membershp functon s large. In order to combne the user-sde propertes nto the scheme and to smplfy the fuzzy rule base, each user mantans hs/her own bunch of fuzzy rules and each fuzzy rule contans only some of the fve nputs. For example, user 1 uses conversatonal applcatons wth money frst and battery frst propertes, so one of hs fuzzy rules could be IF prce s low & power consumpton s low, THEN utlty s hgh. For the sake of concseness, we are not gong to lst all the fuzzy rules. For each network, the fuzzy nference engne combnes all the fuzzy rules n the user s fuzzy rule base and the defuzzfer transfers the fuzzy output nto a crsp value to represent the utlty of the network. In the end, the network wth the hghest utlty s selected. Network selecton results of the 16 users are gven n Table VII, together wth the results from schemes usng other mathematcal theores for comparson. Snce fuzzy logc module gnores trval dfference, there s a non-neglgble probablty that several networks mght have the same prorty. Therefore, n Table VII, we mark all the best networks when we could not dstngush them. V. GAME THEORY Game theory s related to the actons of decson makers who are conscous that ther actons affect each other. The essental elements of a game nclude [32]: Player: the ndvdual who makes the decson. The goal of each player s to maxmze hs/her own payoff by a choce of strategy. Strategy set: the set contanng all the strateges a player can choose. In each round, the player chooses one strategy from the set. Payoff : the utlty that a player can receve by takng certan strategy when all the other players strateges are decded. Equlbrum: the combnaton of strateges contanng the best strategy for every player. Nash equlbrum (NE) s the soluton of a game, n whch no player can acheve more payoffs by unlaterally changng hs own strategy. The technques of game theory are wdely adapted n resource management mechansms n HWNs. We categorze game theoretcal network selecton scheme nto three groups: game between users, game between networks and game between users and networks. A. Game between users The game between users consders the problem n whch users selfshly select ther beleved best network, hence causng network congeston and performance degradaton. [33] modeled the network selecton problem nto a non-cooperatve

10 10 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION game belongng to the class of congeston games between selfsh users. In ths game, the users are the players who take ther actons on selectng one network among the avalable ones. Analytcal upper bounds for the prce-of-anarchy and prce-of-stablty are derved, whch are consderably tghter than well known bounds for generc congeston games. The cost of each user depends on the congeston of the selected network, gven by c k (, l K η l), where ndcates that user k selects network. η k s a bnary varable representng whether user k selects network, so l K η l ndcates the total number of users selectng network. Ths game becomes a problem n whch all the users try to choose the network wth mnmum cost, whose NE can be ndcated as ζ k η k c k (, η l ) c k (, η l ),, N, k K, l K l K (9) where K and N represents the sets of users and networks, respectvely. ζ k s a bnary varable representng whether user k s wthn the coverage of network. Another game model used for network selecton s the evolutonary game, whch extends the formulaton of a noncooperatve game by ncludng the concept of populaton,.e. a group of players. In an evolutonary game, there could be a sngle or multple populatons, and the players from one populaton may choose strateges aganst players from another populaton. In a word, an evolutonary game defnes a foundaton to obtan equlbrum for the game of populatons. Besde the concept of populaton, there are two other mportant concepts n an evolutonary game: replcator and replcator dynamcs. A replcator s a player from a populaton who s able to replcate tself through the process of mutaton and selecton. Ths replcaton process can be modeled by a set of ordnary dfferental equatons, called replcator dynamcs, gven by ṗ (t) = p (t)[π (t) π(t)], (10) where p (t) = K /K denotes the proporton of players choosng strategy, wth K s the number of players choosng strategy and K s the total number of players n the game. π (t) s the payoff of the players choosng strategy and π(t) s the average payoff of the entre populaton. Based on the above replcator dynamcs, the evolutonary equlbrum s defned as the set of fxed ponts of the replcator dynamcs that are stable. In other words, none of the players wants to change ts strategy snce ts payoff s equal to the average payoff of the populaton. [34] studed the evolutonary game for network selecton. In ths game, users are players, users n a servce area forms a populaton, the selecton of one network s consdered as the strategy and utlty of a user s ts payoff. For servce area a, the evolutonary equlbrum s obtaned by solvng the set of equatons ndcated by {ṗ (a) = 0 = 1,..., N}, where N s the total number of canddate networks n servce area a and p (a) denotes the proporton of users choosng network n servce area a. The evolutonary equlbrum s stable f all the egenvalues of the Jacoban matrx correspondng to the replcator dynamcs have a negatve real part. [34] also studed a non-cooperatve game between users n dfferent servce areas. In ths game, users n the same area collaborate wth each other to compete for bandwdth wth other groups of users n other areas. A strategy s the proporton of users choosng network, denoted by p (a). The payoff of a player s the total utlty from all users n the group choosng all dfferent networks, denoted by π(p (a),p ( a) ), where p (a) denotes the vector of proporton of users choosng dfferent networks n servce area a, and p ( a) denotes a vector of the proporton of users n all servce areas except a. Ths game s smlar to the congeston game presented above, except t s a game between groups of users n dfferent servce areas, nstead of sngle users. Another dea s to model network selecton as a Bayesan game wth ncomplete nformaton snce t s usually dffcult to nform all the players about the requred nformaton from other users. In a Bayesan game, the ncomplete nformaton s consdered as prvate nformaton of players before the game begns, called the type of the player. [35] modeled network selecton nto a Bayesan game by defnng the type of player k as ts mnmum bandwdth requrement B k B k, where B k s the type space of player k. B k s a varable obeyng certan probablty dstrbuton functon. Then, the expected payoff π k s defned as bandwdth utlty mnus connecton fee, where bandwdth utlty s the beneft the user gets from selectng certan network, whch could be zero f the allocated bandwdth s smaller than B k. In a Bayesan game, for every type of player k, the best response can be obtaned by B k (q k, B k ) = arg max q k Q π k(q k,q k, B k ), (11) where Q s the set of Bayesan strateges. A NE s ndcated by strategy {qk,q k }, f and only f q k Q, k K, π k (qk,q k ) > π k(q k,q k ). Moreover, a combnaton of Bayesan game and evolutonary game s also tred for the network selecton ssue by [35]. In the above studes of game between users, they assume that multple users are watng for servce at the tme of decson. However, we all know that users usually come for servce one by one. [36] studed a WLAN access pont selecton case where selecton requrement of multple termnals are not comng concurrently and all the termnals n the WLAN coverage area are nformed mmedately wth the network selecton nformaton of each termnal. It was proved that the outcome of a one-by-one optmzaton process of these termnals corresponds to the NE of a one-shot game wth multple termnals concurrent selecton. One specal scenaro where multple users mght do network selecton at the same tme s called group handover n [37]. Ths happens when multple users move together, e.g., n a bus, or when certan network has some sudden problem. Three optons were proposed: 1) f each moble termnal knows the traffc loads of the other termnals, a NE based algorthm can be used. In ths algorthm, the selecton of each termnal s the correspondng strategy of the computed NE; 2) another algorthm s to separate termnals handovers by usng random delays, smlar to the algorthm avodng handover synchronzaton n [38]. In ths algorthm, each

11 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 11 termnal that has decded ts selecton should announce that to the others or to an ndependent functon entty, so that others know ts selecton; and 3) sometmes, a termnal decdes to select a target network and announces ts selecton to others, but t may not be able to really handover to t due to falure or rejecton by that network. In ths case, other termnals get ncorrect nformaton about the handover of ths termnal. Therefore, the thrd algorthm s to announce ts selecton after the termnal has already fnshed ts handover to the target network. B. Game between networks In an HWNs envronment, dfferent networks mght be managed by dfferent servce provders, so ther competton to attract and get more users become an mportant ssue. Game between networks does not provde us a network selecton scheme for users, but t ndrectly gudes users to thnk about ther correspondng schemes for network selecton under ths network competton envronment. One model s to consder prcng strateges as the strateges of networks. For non-cooperatve case, the problem s modeled as a Bertrand game [13], whch descrbes nteractons among sellers that consder ther prces and buyers that choose ther product at that prce. Assumng that each user chooses the network wth the maxmum performance-cost rato (PCR), each network chooses the prcng strategy that maxmzes ts own payoff (related to the prce of servce and the number of users choosng ths network), fxng the other networks prcng strateges, whch ndcates the NE. However, severe competton may result n low prce and shrnk total payoff n turn, whch s not acceptable for network operators. Therefore, cooperaton between several or all network operators may be establshed to provde the same QoS to users wth the coalton prce. Another model s to consder the strategy of a network as the selecton of a user for servce, n whch the users are totally passve and have no rght to decde whch network he wants to use. As an example, [39] descrbed such a mult-round game model as follows: 1) a bunch of users send servce requests to multple networks; 2) a centralzed entty gathers requests and put the users nto a watng lst. Networks calculate payoffs based on gathered nformaton; 3) n each round, each network selects one user for servce and ths user s removed from the watng lst; 4) multple rounds are performed untl all the users are served. In ths game, the best strategy for each network s to select the user wth the maxmum payoff from the watng lst of users that have not been served. C. Game between users and networks The set of users and the set of networks are consdered as two players. The users strateges are to select ther favorable networks to maxmze ther payoffs, such as qualty of servces and prce. Meanwhle, the networks strateges are to select ther favorable users to maxmze ther payoffs, such as the revenue [40]. If NE exsts, the users and the networks correspondngly select each other. Otherwse, a sub-optmal soluton wll be used. At the end of ths secton, we would lke to menton that, for studes usng game theory, t s mportant to not only ndcate NE but also study how to reach the NE. Studes on network selecton have utlsed dfferent approaches for ths purpose, such as a centralzed approach called populaton evoluton n [34], and some decentralzed approaches n whch users could ndependently adapt themselves to reach the equlbra, e.g., Q-learnng n [34] and no-regret learnng n [41]. Moreover, n the lterature of game theory, there are numerous algorthms for NE searchng, e.g., Lemke-Howson algorthm [42] searchng for one NE and Dckhaut-Kaplan algorthm [43] searchng for the support of all NE. However, explanaton of these algorthms s out of the scope of ths tutoral. D. Case study We consder the unfed scenaro presented n Secton I wth Tables I and II. Frst and foremost, we emphasze that the feature and result of a game s largely related to the defnton of the utlty n the game. If the utlty s defned hghly correlated to the average bandwdth obtaned by selectng certan network, the equlbrum of ths game has the trend to unformly dstrbute users nto dfferent networks. However, when networks are all wth enough resource at certan moment, ths knd of equlbrum s apparently not a good soluton. Therefore, we defne the utlty of the game as follows n ths case study: when the selected network could support all ts users, the utlty of each user s calculated as the total utlty of the fve normalzed attrbutes by SAW algorthm, smlar to the case study n Secton III; otherwse, we assume that congeston n ths network occurs, so the utlty of each user n ths network s zero. Wth the above utlty functon, the equlbrum provdes the same result as SAW algorthm when networks have enough resource. In order to show the dfference between ths game model and MADM, we consder the stuaton when networks capactes are qute lmted and we could not let all the users select ther favorte networks as n MADM-based schemes. We set each network a lmted capacty for these 16 users. In other words, you could magne that these networks capactes have already been largely occuped by other users at the moment of the comng of these 16 users. For farness, we assume that the 4 networks have the same lmted capacty, gven by 12, so that we could avod the case where the prevous traffc of networks domnates these users selecton. Moreover, n order to let all the users beng served by the end of the selecton procedure, we set the capacty cost of each user, based on ther applcatons, as {1, 1, 1, 1, 5, 5, 5, 5, 1, 1, 1, 1, 2, 2, 2, 2}. We ntentonally set the whole capacty of the 4 networks (.e. 48) larger than the total requred capacty of the 16 users (.e. 36), so as to see the possblty of some networks havng more users than others. Based on Nash s theorem n [44], ths game has at least one NE. We could defntely use certan algorthm mentoned

12 12 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION above to fnd the NE, but the usage of these algorthms could not show us the dfference between usng game theory of ths network selecton ssue or other mathematcal theores. In order to show an ntutonstc comparson between game theoretcal network selecton scheme and other schemes, e.g., MADM-based schemes, we use the followng method to smply fnd a pure strategy NE: Frst, we put all the users nto ther favorte networks based on the calculated utltes usng SAW. Second, we check f there are some networks gettng congested. If so, we choose the user wth mnmum capacty cost from ths network and put t nto the network wth maxmum utlty among all the networks wth enough capacty. We contnue ths procedure untl no network s under congeston. Thrd, n the obtaned allocaton state, we search and swtch for each user f there s a better network untl no user could ncrease ts utlty by unlaterally changng to another network. Fnally, we reach a pure strategy NE. We can see that the objectve of the frst and second steps n the above method s just to get to an ntal state for the thrd step. We use SAW n the frst step nstead of a random ntal state, so that we could compare the results wth MADMbased schemes. We fnd that the allocaton n the frst step s qute smlar to that of MADM-based schemes wthout traffc consderaton n Secton III. Network selecton results of the 16 users usng the above game theoretcal scheme are gven n Table VII, together wth the results from schemes usng other mathematcal theores for comparson. Wth the above confguraton of networks and users, these results are actually obtaned by the frst and the second steps. When we check for the possblty of any user could unlaterally ncrease ts own utlty by changng to another network, we fnd that the allocaton state obtaned by the frst two steps s concdently already a pure strategy NE. VI. COMBINATORIAL OPTIMIZATION Combnatoral optmzaton searches for an optmum object n a fnte collecton of objects. The number of objects grows exponentally n the sze of the collecton, so scannng all objects one by one and selectng the best one s not an opton [45]. Based on the tme complexty, combnatoral optmzaton problems can be classfed nto several groups, e.g., NP-hard problems whch are consdered at least as hard as NP problems. NP s short for non determnstc polynomal tme. A. Combnatoral optmzaton n network selecton Two NP hard models,.e., knapsack and bn packng, have been consdered for the network selecton problem. Knapsack problems are a famly of optmzaton problems that requre a subset of some gven tems to be chosen so that the correspondng proft sum s maxmzed wthout exceedng the capacty of the knapsack(s). A generalzed knapsack model fttng for the network selecton problem s a combnaton of the 0-1 knapsack model and the multple choce multple dmenson knapsack (MMKP) model [46], gven by N M N max U = ψ k z k, s.t. c k z k C, (12) k=1 =1 k=1 where U s the total proft, ψ k s the proft of tem k placed n knapsack, c k s the capacty cost of tem k placed n knapsack, z k s a bnary varable representng the placement (or not) of tem k n knapsack, and C s the capacty of knapsack. Mappngs between network selecton and the knapsack problem are gven as follows: 1) Applcatons map to the tems, 2) Networks map to the knapsacks, 3) Resource constrant of a network maps to the capacty of a knapsack, 4) Cost of an applcaton n a network maps to the cost of an tem n a knapsack, 5) User utlty maps to the profts, and 6) Utlty of an applcaton n a network maps to the proft of an tem n a knapsack. It s worth mentonng that the knapsack model fts for the case when networks capactes are qute lmted and load balancng s strongly demanded. When the capacty of networks s large enough for a comng applcaton, the above model becomes a SAW algorthm presented n Secton III. Another NP hard model used to solve the network selecton problem s bn packng. The classcal bn packng problem s a well studed optmzaton problem: gven K objects wth szes c 1,..., c K belongng to (0, 1], fnd a packng n unt-szed bns that mnmzes the number of bns used. In the off-lne verson of ths problem, t s possble to consder all the objects and choose the order of assgnment. In the onlne verson however, each object must be assgned n turn wthout knowledge of the next objects. That s, gven K 1 already packed objects wth szes c 1,..., c K 1 belongng to (0, 1], the new object K wth sze c K belongng to (0, 1] must be packed n such a manner that the number of used bns s mnmzed. Network selecton can be formulated as a bounded-space varable-sze onlne bn packng problem, n whch the number of avalable bns at any tme s a restrcted to a predefned number (.e., bounded-space) and the capactes of bns can be dfferent (.e., varable-sze). The objectve s to fnd the best way of allocatng applcatons nto the networks n order to mnmze the number of rejected applcatons,.e., the blockng probablty, hence maxmzng the whole system s capacty. Moreover, one obvous dfference from the classcal bn packng problem s that the bandwdth requred by one applcaton s determned by the selected network, so we use c k to denote the sze of applcaton k n network. In [47], the authors mapped the problem of network selecton nto the bn packng problem n ths way and compared fve algorthms, ncludng FrstFt, BestFt, WorstFt, LessVoce and Random. The selecton rules of these algorthms are summarzed as follows: FrstFt: the frst randomly selected network that has enough space for the applcaton. BestFt: the network wth mnmum free space left after servng the applcaton.

13 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 13 WorstFt: the network wth maxmum free space left after servng the applcaton. LessVoce: the network wth mnmum c k /c voce,. Random: a totally random network, rejectng to serve the applcaton when no enough space for t. Based on the above studes, [35] proposed a greedy heurstc algorthm to match between the users and the networks. For the case of K users allocatng to N networks, the algorthm starts wth an K N utlty-to-resource rato lst where a utlty-toresource rato s between the utlty of a user and the resource that a network could allocate to ths user. In each round of the algorthm, the user-network par wth the largest utltyto-resource rato s pcked and all the ratos for ths user are removed from the lst. The tme complexty of ths algorthm s bounded by O(K 2 N). Ths greedy heurstc algorthm was compared wth three bn-packng algorthms (ncludng FrstFt, BestFt and WorstFt) and was shown that t outperforms them on both total utlty and blockng probablty. B. Case study We consder the usage of the MMKP knapsack model n the unfed scenaro presented n Secton I wth Tables I and II. Smlar to the case study n Secton V, ths model also fts for the stuaton when networks capactes are qute lmted. Otherwse, t becomes a SAW algorthm of MADM, as explaned n the case study of Secton III. Therefore, n order to show the dfference between schemes wth ths mathematcal model and others, the capacty of networks and the capacty cost of users are set n the same way as explaned n the case study of Secton V. The proft of each user s obtaned as the combnaton of the normalzed values of the fve attrbutes based on ther weghts obtaned by the egenvector method, smlar to our confguraton n the case study of Secton III. Fnally, we use smulated annealng (SA) algorthm [48] to fnd a sub-optmal soluton for ths problem. We state the algorthm from an ntal state wth a total proft of 6.16, gven by {W, M, L, P, W, M, L, P, W, M, L, P, W, M, L, P }, n whch network serves one user of each applcaton. Wth 1, 000, 000 rounds, we fnally fnd a sub-optmal soluton wth a total proft of and the allocaton n Table VII. Based on the selecton results of MADM n Table VII, we predcted that users should frst occupy the capactes of WMAN and WPAN as much as possble, then choose WWAN or WLAN. Ths s proved true by the results, n whch the four networks capactes are occuped as {2, 12, 10, 12}. VII. MARKOV CHAIN Markov chan s a common tool for decson makng. In ths secton, we present three types of Markovan approaches for network selecton: Markov decson process (MDP) based scheme, permutaton-based scheme and rank aggregaton based scheme. A. MDP-based scheme In many stuatons n the optmzaton of dynamc systems, a sngle utlty for the optmzer mght not suffce to descrbe the real objectves nvolved n the sequental decson makng. A natural approach s to optmze each objectve wth constrants on others. MDP can be used to handle ths knd of mult-objectve dynamc decson makng problem [49]. In the lterature, several network selecton schemes based on MDP theory have been proposed. An MDP s defned through the followng objects [50]: a state space S, sets A (s) of avalable actons at states s S, transton probabltes ρ(y s, a) and reward functons r(s, a) denotng the one-step reward usng acton a n state s. The above objects ndcate a stochastc system wth a state space S. When the system s at state s S, a decson maker selects an acton a from the set of actons A (s). After an acton a s selected, the system moves to the next states accordng to the probablty dstrbuton ρ(y s, a) and the decson-maker collects a one-step reward r(s, a). The selecton of an acton a may depend on the current state of the system, the current tme, and the avalable nformaton about the hstory of the system. At each step, the decson maker may select a partcular acton or, n a more general way, a probablty dstrbuton on the set of avalable actons A (s), whch are called nonrandomzed and randomzed decsons, respectvely. An MDP s called dscrete f the state and acton sets are dscrete, whch s the case for network selecton. For dscrete MDP, we denote the transton probabltes by ρ(y s, a). [51], [52] provdes an dea for modelng the network selecton problem nto an MDP. They put many decson epochs durng the lfetme of a sesson wth ether equal or varable tme ntervals, represented by t = {1,..., T }, where T denotes the tme that the sesson termnates. At decson epoch t t, s t and a t are used to represent the current state and the chosen acton, respectvely. The state transton probablty s denoted by ρ(y s t, a t ). The reward s defned by r(s t, a t ) = f(s t, a t ) g(s t, a t ), where f(s t, a t ) represents the beneft from usng another network rather than the current one and g(s t, a t ) represents the sgnallng cost (may also consder packet loss) for handng-over to that network. For the whole sesson perod, a polcy θ = (δ 1,..., δ T ) Θ, Θ = A (s 1 )... A (s T ), s defned as a sequence of acton rules at all the decson epochs, where δ t, t {1,..., T } represents the acton rule at decson tme t. Gven an ntal state s 1, the objectve of ths MDP s to determne an optmal polcy θ to maxmze the expected total reward, denoted by v(s 1 ) = max θ Θ vθ (s 1 ). v θ (s 1 ) s calculated as the mean value of the total reward of all epochs wth respect to the polcy θ and the ntal state s 1. To satsfy the Bellman optmalty equaton, the above equaton could be further wrtten as v(s 1 ) = max a A (s 1) { r(s1, a) + γ y S [ ρ(y s1, a)v(y) ]} (13) where γ s the dscount factor mappng the future reward to the current state. The future reward s less relable and predctable, so t s less mportant than the current reward, denoted by γ 1.

14 14 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION One key feature of MDP model s that t consders a bunch of consecutve decson epochs and makes a combned decson at the begnnng, but ths also requres an ambtous assumpton that we need to predct, at the begnnng of a sesson, the state nformaton for all the future decson epochs durng ths sesson. Another feature s that MDP model solves network selecton and VHO decson at one tme by consderng both beneft f(s t, a t ) and handover cost g(s t, a t ). If we only consder f(s t, a t ), ths model tells us the best network at all the decson epochs. Moreover, [53] used MDP for user/operator negotaton after network rankng. State s defned by the number of ongong calls and the events, e.g., new call arrval, handover call arrval and call departure. Acton s defned as admttng a call, rejectng a call and no acton for call departure case. Reward s defned as the beneft for the operator from the acceptance of a call, whch s related to servce class. Based on these defntons, an operator could fnd the best strategy for a sequence of calls, whch satsfes the Bellman optmalty equaton. Due to the fact that [53] s manly about user/operator negotaton, not network selecton, we are not gong to dscuss more on t. B. Permutaton-based scheme To select the best network, an mportant task s to dstngush between networks. Snce we consder network selecton for moble termnals, one mportant type of attrbutes to dstngush between networks s the moblty-related attrbutes, such as cell radus, coverage percentage, VHO propertes, etc. Tradtonal attrbutes, e.g., prce, bandwdth, etc., usually lead to the dscovery of the best network, but mobltyrelated factors show us the prortes of networks. For example, notcng that certan nomadc termnal s VHO cost between 3G and WLAN s acceptably small, a strategy called WLAN frst for ths termnal should be used. Ths strategy does not mean the termnal always connects to WLAN, but WLAN has a hgher prorty than 3G. In ths tutoral, we use the concept permutaton to represent the prortes of all the networks, wthout consderng ther avalablty. At anytme and anywhere, the frst avalable network n the permutaton should be selected. When there are N networks, we have N factoral permutatons, so the network selecton ssue becomes the selecton of the best permutaton for usage, whle the defnton of the best permutaton s largely related to the VHO cost between networks. In our prevous work [54], the total cost of a permutaton was modeled as follows: Wth N networks and M attrbutes, we use v j to denote the value of the jth attrbute of the th network, σ to denote the probablty that network s avalable, w H to denote the weght of average handover cost and w to denote the weght of the th attrbute except the average handover cost, respectvely. The total cost of each permutaton can be wrtten as C PERM = (h H + h + V + h V ) w H+ N 1 [ R σ (1 σ j ) ] (1 w H ), =1 j=0 (14) where R = M j=1 v jw j s the combnaton of all the other attrbutes except VHO cost for network, h H s the average HHO cost, h + V and h V represent the average VHO cost of movng nto a network better than the current one and the average VHO cost of movng out of the frst avalable network, respectvely. Markov chan s used to help calculate h + V and h V. A state S( ) n the Markov chan s defned as the state of a termnal stayng n an area covered by a certan bunch of networks. For example, S({n 1 > n 2 > n 3 }) represents that the termnal s covered by network n 1, n 2 and n 3, whle S({n 2 > n 3 }) represents that the termnal s covered by network n 2 and network n 3. Symbol > represents the left-sde network s better than the rght-sde one. Therefore, when the termnal s movng from S({n 1 > n 2 > n 3 }) to S({n 2 > n 3 }), ths movement leads to a VHO, contrbutng to h V. Snce the number of permutatons s the factoral of the number of networks, a permutaton-based scheme could take too much tme on the calculaton of all the permutatons total costs, whch causes a problem of slow decson. One dea to smplfy the scheme s to dvde all the networks nto a few groups. As an example, [55] used sgmodal utlty functons for attrbute adjustment, hence dvdng all the networks nto two groups. One group s small-scale networks, whle the other group s large-scale networks. Usng the above model, a threshold could be obtaned for ths two-group case, gven by T (w H ) = R L S R L S + h {S>L} {L>S} /ρ S, (15) where the subscrpts L and S represents large-scale and smallscale networks, respectvely. Hence, R L S s the dfference between R L and R S, and h {S>L} {L>S} represents the dfference between average handover costs of the two permutatons {S > L} and {L > S}, respectvely. Seen from the above threshold, the decson s dependent on w H. If a scheme uses a weght smaller than T (w H ), {S > L} s the best permutaton. Otherwse, {L > S} s the best. Besde the consderaton of moblty-related factors, another key advantage of permutaton-based scheme s that t decreases the scheme trgger rate. When the best permutaton s obtaned, we do not have to trgger the scheme by termnal movement, but all the other schemes have to trgger network selecton when the termnal moves to a new place where network coverage s dfferent (.e. from state to state n the Markov chan of permutaton-based scheme). C. Rank aggregaton based scheme Network selecton can be formulated nto a rank aggregaton problem, n whch a better rank can be derved by combnng multple ranks of dfferent decson factors. [56] proposed a weghted Markov chan (WMC) scheme, fallng nto ths branch, whch fnds the best network wth the followng algorthm: 1) Based on each attrbute j, a rank of all the networks s obtaned, gven by τ j = {n j 1... nj N }, where nj represents the th network n the rank by ths attrbute and N represents the number of canddate networks. τ j () denotes the rank of network n τ j. w j denotes the weght of attrbute j.

15 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 15 2) An N N weghted Markov chan transton matrx Y s ntalzed and updated wth certan method below. 3) The statonary dstrbuton vector f = {f 1,..., f N }, where sd s the preference ndex of network, calculated by f = f Y. 4) The best network n θ s the one satsfyng θ = arg maxf. The key step of ths algorthm s step 2 to update the Y matrx. [56] proposed two methods for ths task: Method I: for each attrbute j and for each entry y kl n matrx Y, y kl = y kl + wj f τ τ j(n j(n j j k ) k ) τ j(n j l ). Method II: for each attrbute j and for each entry y kl n matrx Y, y kl = y kl + wj(n τj(nj k )+1) N y kl = y kl + wj N f τ j(n j k ) τ j(n j l ). f τ j (n j k ) = τ j(n j l ), or Another Markovan approach related to network selecton was proposed n [57]. State s defned based on the number of users of dfferent servces (e.g., voce and data) n dfferent canddate networks. Transtons between states wthn the Markov chan wll occur due to the arrval and departure of voce call or data sesson. Gvng the arrval dstrbutons of voce calls and data sessons, the transton rates between states n the Markov chan wll be decded by the network selecton polcy. The orgnal authors showed that ths model could be used to evaluate the performance of many types of network selecton schemes, e.g., random selecton and load balancng based selecton. However, based on our understandng, ths approach s more related to call admsson control and t s dffcult to be used as a scheme to dynamcally select the best network n varous scenaros. Therefore, we are not gong to dscuss more on ths model. D. Case study MDP s an mportant mathematcal model for decson makng. An mportant feature of studes n [51], [52] s that MDP enlarges the mportance of handover cost, so some state nformaton, e.g. the current used network, becomes very mportant for the decson. By gnorng VHO cost, these consequent decsons become totally ndependent, and ths model provdes actually an MADM-based network selecton. By consderng VHO cost, ths model provdes actually a VHO decson scheme not a network selecton scheme. However, snce MDP-based scheme becomes an MADM-based scheme by removng the VHO decson part, we are not gong to do any comparson between MDP-based scheme and other schemes. For smlar reason, we are gong to compare the permutaton-based scheme wth other schemes. Instead, we select the WMC-based scheme wth MC update method I for ths case study. We stll use the unfed scenaro presented n Secton I wth Tables I and II. Weghts are calculated by egenvector method, as explaned n Secton III. As we assumed n Table I, some features of dfferent networks are totally the same. If we gve them dfferent postons n the rank, t s unfar. For example, we assume cell radus of WWAN and WMAN are both 2000, f we gve WWAN the frst place n the rank and WMAN the second place n the rank, WWAN domnates WMAN based on the rank of cell radus for most moblty frst users, whch s wrong. Therefore, n our study, we specfcally check f some networks have qute smlar values for certan attrbute. If so, we gve them the same poston n the rank. For each user, the statonary dstrbuton vector s obtaned and the best network s selected as shown n Table VII. VIII. INTEGRATED SCHEME A. Comparson of usng dfferent mathematcal theores for network selecton A general comparson of usng the above mathematcal theores for the network selecton ssue s provded n Table VI. We compared eght aspects as follows: Objectve: dfferent mathematcal theores have dfferent functonaltes, whch lead to dfferent objectves for ther usage n network selecton. To sum up, utlty theory evaluates the utlty of the value of each attrbute. For example, a lttle change of the value of an attrbute, that passes some QoS threshold, leads to greatly change of ts utlty. MADM provdes a comprehensve theory for the combnaton of multple attrbutes for a decson, although most studes usng other theores also consder SAW by default. Fuzzy logc theory s especally helpful to adjust the values of dynamc attrbutes snce the nformaton of these attrbutes could be mprecsely collected. Game theory tells us the equlbrum between networks, between users, or between networks and users, whch helps us to balance benefts among multple enttes. Combnatoral optmzaton provdes us a sub-optmal allocaton of users to networks, whch could be qute close to the optmal soluton. For the three types of Markovan approaches, the functonaltes and objectves are totally dfferent. MDP-based scheme s to optmze a seres of consecutve decsons wth predcton, permutaton-based scheme provdes the prortes of networks nstead of the best network, whle rank aggregaton based scheme s to aggregate the ranks of networks obtaned by dfferent attrbutes. Decson speed: schemes usng utlty theory, MADM or fuzzy logc are all fast to make a decson. Schemes usng combnatoral optmzaton are really slow. For example, n our case study of the knapsack problem usng smulated annealng, t takes dozens of seconds to complete a search of 1,000,000 rounds, whch s defntely too late for makng the decson on the best network. For game theory, the learnng process takes some tme. For Markovan approaches, the combnaton of consecutve decsons n MDP-based scheme, the calculaton of the total costs of all the permutatons n permutaton-based scheme and the update of the MC matrx n rank aggregaton based scheme all take some tme. Therefore, schemes usng game theory and Markov chan are not as fast as the frst three theores, but defntely faster than combnatoral optmzaton. Implementaton complexty: schemes usng utlty theory, MADM or fuzzy logc are all smple to be mplemented. Schemes usng combnatoral optmzaton are complex. The complexty of Markovan approaches s between them due to the fact that the algorthms and calculatons n Markovan approaches are more complex than the frst three theores and defntely less complex than combnatoral optmzaton. For the mplementaton of a game-theoretc scheme, a dstrbuted algorthm by each player s usually used to get to the

16 16 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION TABLE VI COMPARISON OF USING DIFFERENT MATHEMATICAL THEORIES FOR NETWORK SELECTION Utlty theory MADM Fuzzy logc Game theory Combnatoral optmzaton Markov chan Objectve Utlty evaluaton Combnaton of multple attrbutes Imprecson handlng Equlbrum between multple enttes Allocaton of applcatons to networks Consecutve decsons / rank aggregaton / prorty evaluaton Decson speed Fast Fast Fast Mddle Slow Mddle Implementaton complexty Smple Smple Smple Complex Complex Mddle Precson Mddle Hgh Mddle Hgh Hgh Hgh (but Low for WMC) Decentralzed Yes Yes Yes Yes No Yes User-centrc Yes Yes Yes No No Yes Moblty-orented No No Yes No No Yes Traffc-orented No No No Yes Yes No equlbrum, whch s largely more complex than Markovan approaches. Precson: schemes usng MADM, game theory or combnatoral optmzaton are precse. For Markovan approaches, MDP-based scheme and permutaton-based scheme are precse, but rank aggregaton based scheme s really mprecse due to the fact that rank only provdes networks prortes not the exact dfference between ther quanttatve values. The precson of schemes usng utlty theory and fuzzy logc s dffcult to judge. Utlty functons n utlty theory and membershp functons n fuzzy logc both have the functonalty to adjust attrbutes,.e. enlarge or dmnsh the dfference between networks on certan attrbute, but ths adjustment could loss precson. For example, n our case study of fuzzy logc, we utlzed some smple membershp functons and some smple fuzzy rules, so some networks are found wth the same total utlty. Therefore, the precson of schemes usng utlty theory and fuzzy logc s lower than MADM, game theory and combnatoral optmzaton. Decentralzed: wthout consderng the problem of nformaton gatherng, all the theores could be used for decentralzed network selecton schemes except combnatoral optmzaton. Combnatoral optmzaton provdes centralzed algorthms to optmze the allocaton of applcatons to networks. User-centrc: schemes usng game theory or combnatoral optmzaton consder too much on the traffc load of networks, whch benefts operators a lot but degrades the user s beneft. Schemes usng the other theores do not have ths feature, whch are user-centrc. Moblty-orented: as we explaned n Secton VII, t s dffcult to take moblty-related attrbutes, especally VHO-related attrbutes, nto account for the decson of the best network. In the lterature, only schemes usng fuzzy logc and some Markovan approaches consdered VHO-related attrbutes for the decson. Among these schemes, only permutaton-based scheme used VHO-related attrbutes for network selecton, whle the others used them for VHO decson. In Table VI, we judge all these schemes as moblty-orented due to the fact that network selecton and VHO decson mght be processed together. Traffc-orented: smlar to our explanaton on whether the schemes are user-centrc, schemes usng game theory or combnatoral optmzaton take traffc load as a qute mportant factor for the decson, whch even degrades other attrbutes. Therefore, the two are consdered as traffc-orented, whle the others are not. B. Integraton of multple mathematcal theores We studed the usage of varous mathematcal theores n ths tutoral for the network selecton ssue. As you can see from the above studes, they have dfferent features and dfferent functonaltes. To get all ther benefts, we could thnk about combnng them n the way shown n Fg. 5 to acheve an ntegrated soluton: Utlty theory: network attrbutes, ncludng traffc load, are adjusted by utlty functons, but traffc load, whch s hghly related to combnatoral optmzaton and game n later operatons, may be adjusted n a dfferent way from others. Fuzzy logc: when there are many access networks, we classfy all the networks nto several groups to decrease the tme cost on the comparson of all the permutatons. Ths operaton s based on some key factors, such as cell radus, bandwdth and prce, usng fuzzy logc. MADM: after the adjustment of the network attrbutes, MADM algorthm s used to combne these attrbutes based on ther weghts. Combnatoral optmzaton: before MADM, we mght check whether many networks avalable capacty become lmted. If so, nstead of MADM, we could use certan algorthm of combnatoral optmzaton for the allocaton of new servces. Note that ths theory s used n a centrc manner on the network-sde, not by termnals. Markov chan: MDP mght be used n the tradeoff of VHO decson after the networks are ranked

17 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 17 Trgger & Informaton gather Adjustment of attrbutes Network rankng & user allocaton VHO decson wth predcton Equlbrum management Utlty theory MADM Network attrbutes & user-sde and network-sde nformaton Weghtng CCT MDP Game theory Fuzzy logc Markov chan based Recurson Fg. 5. Relatonshp between varous mathematcal theores for network selecton. Game theory: after the tradeoff of VHO decson, many smultaneous (or techncally consdered as smultaneous) handng-over termnals mght select the same best network, whch causes congeston. We mght use game theory for an opportunstc decson, so that these termnals could be dstrbuted nto dfferent networks. C. Case study of an ntegrated scheme For ths case study, we take our prevous desgn whch was very brefly presented n [24] as an example of the ntegrated scheme. As shown n Fg. 6, the soluton contans four steps: 1) montor the trggers and gather the requred nformaton. 2) preparatons before combnng all the attrbutes, ncludng weghtng procedure and attrbute adjustment procedure. 3) combne multple attrbutes as a sngle rank. The left part of ths step, whch could be any tradtonal network selecton scheme, gves the best network. The rght part of ths step gves the best permutaton, as explaned n Secton VII.B. Snce t takes more tme to get the best permutaton than to get the best network, we use the best network untl the best permutaton s obtaned. 4) make VHO decson. If the best network or the frst avalable network of the best permutaton s better than the current network, ths step makes a smple decson on whether the beneft s worth the one-shot VHO cost. We beleve that the network selecton procedure mplemented n the future termnals should be smple and fast, and the man goal of network selecton s to always select the best network for servng the gven applcaton, not to pay too much attenton to load balancng. A network selecton scheme payng too much attenton to ths attrbute degrades other attrbutes mportance. Takng two networks both wth low but totally dfferent traffc loads as an example, the normalzaton process wll gnore the two networks low traffc loads but retan only the relatve large dfference, whch leads to mmoderate traffc load balancng between the two networks and compromses the mportance of other attrbutes. Therefore, traffc loads are usually not requred to be strctly balanced among dfferent networks due to the fact that traffc load s only one of a number of attrbutes, usually even not a decsve attrbute, except when at least one network does not have much resource. Based on the above analyss, we do not use game theory or combnatoral optmzaton for a specfc load balancng. Instead, we adjust traffc load n a dfferent manner from other attrbutes n our ntegrated scheme. Frst, snce the real values are more mportant than the relatve dfference between the traffc loads of the two networks, we do not normalze them. Second, we use a specal sgmodal utlty functon wth md value equals 1 and a 2 to calculate the utlty. Based on our experence, a large value for a can be used to overcome mmoderate traffc load balancng. For ths case study, the unfed scenaro presented n Secton I wth Tables I and II s used. In order to provde a far comparson wth other schemes, we use the same attrbutes, the same sgmodal utlty functons and the same MADM algorthm.e., SAW. Moreover, we have the followng specfc confguraton to guarantee farness n ths comparson: fuzzy logc theory for network groupng s not consdered n ths study. Otherwse, some networks are gong to be totally the same after the adjustment of utlty functons and fuzzy membershp functons, whch hdes the load balancng feature of ths ntegrated scheme. other schemes do not consder VHO decson, so we are not gong to use VHO decson step n ths case study, ether. Otherwse, the comparson s unfar for other schemes. Therefore, the networks ndcated n Table VII are the best networks for those users. Whether those users wll handover to ther best networks stll depends on VHO decson. The dfference between the ntegrated scheme n ths study and other scheme presented n prevous sectons s as follows: our soluton combnes utlty theory and MADM. the sgmodal utlty functon for traffc load s specfcally desgned as explaned above. weghts are calculated based on our trgger-based method. Markov chan s used for best permutaton selecton.

18 18 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION Applcatons QoS Levels Applcatons QoS Levels Informaton Gatherng Objectve Weghtng Termnal Propertes Customer Preferences Operator Polces Networks Propertes VHO Propertes Trgger-based Subjectve Weghtng Combned Weghts Mappng Pot Termnal Propertes Customer Preferences Operator Polces Dynamc Network Propertes Avalablty of A Network Traffc Load Values Other Attrbutes Network Groupng Scheme Trggers Begnnng Step Pre-MADM Step Weghtng Adjustng Combne Multple Attrbutes Best Network Certan Permutaton Best Permutaton VHO Cost Total Costs of Permutatons MADM Step Best Network Best Permutaton VHO Decson Comparson lst = {Best Network, Current Network} Comparson lst = Best Permutaton Lst ndex = 1, VHO = False = + 1 = current network? No Enough resource? No Yes No Tradeoff between the current network and network? Yes VHO executon to network Fnal Decson Step Yes Fg. 6. An example of ntegrated scheme for case study. the best network s the frst network (because all the four networks are avalable) n the best permutaton obtaned by the rght sde of step 3, not from a best network selecton scheme. tradtonal network selecton scheme s ntegrated for a fast decson before the best permutaton s found. the dffculty of mplementaton comes from the calculaton of total costs of all permutatons. the precson s hgh as long as we do not use the network groupng functonalty. the soluton s decentralzed, user-centrc, mobltyorented and traffc-orented. Note that Fg. 6 s just one example of ntegratng multple theores, and the features and network selecton results could be totally dfferent f you combne multple theores n a dfferent way. In ths study, we also consder the case where each network has a lmted capacty for these 16 users, as explaned n the case study of game theory. We rank the 4 networks based on the ntegrated scheme as shown n Fg. 6, but at the begnnng of the last step, we check f the network has enough resource before VHO decson. We wll show that our ntegrated scheme could acheve a smlar load balancng functonalty wthout usng game theory or combnatoral optmzaton. Network selecton results of the 16 users are gven n Table VII, together wth the results from schemes usng other mathematcal theores for comparson.

19 WANG AND KUO: MATHEMATICAL MODELING FOR NETWORK SELECTION IN HETEROGENEOUS WIRELESS NETWORKS A TUTORIAL 19 TABLE VII SELECTION RESULTS OF DIFFERENT SCHEMES IN THE CASE STUDIES Sgmodal utlty M M M M L M L M M M M M P M P M SAW wth AHP P M P M P M P M P M P M P M P M Fuzzy logc L M L WM LP L M M P WM WMP WM P L P WM Game between users L L W W P M P M W W W W L L P M Knapsack wth SA P M P M L L P M P M M M P M P W WMC P M P M P M M M P M M M P M P M Integrated scheme P M P M L L P M M M M M L P P W Note: W = WWAN, M = WMAN, L = WLAN and P = WPAN D. Observatons on the selecton results of dfferent schemes For the selecton results of dfferent schemes n the case studes n ths secton and prevous sectons, summarzed n Table VII, we have the followng mportant observatons: dfferent types of users have some general preferences. For example, WLAN s selected by a lot of streamng users but not selected by nteractve users at all; nteractve users prefer WMAN and WWAN for securty reason; conversatonal users also prefer WMAN and WWAN but for contnuty reason; money-frst users prefer WPAN and WLAN; moblty-frst users prefer WMAN and WWAN; and battery-frst users prefer WPAN. snce we desgn WWAN as a domnated network by WMAN, users bascally prefer WMAN to WWAN. For example, wth the frst two schemes, no user selects WWAN at all. Wth schemes usng fuzzy logc, WMAN s better than WWAN for most users, but equally good as WWAN for some users for the sake of mprecson of fuzzy logc. Wth schemes usng the other four theores, traffc s consdered, so WWAN mght be selected when WMAN s full. f we consder battery low as an mportant event, WPAN s obvously preferred. A few exceptons wth utlty theory and fuzzy logc are due to the mprecson reason, whle a few exceptons wth the last four theores are due to the reason of traffc load balancng. SAW wth AHP, fuzzy logc, game between users, knapsack wth SA and WMC all defne total utlty n the same way,.e., summng up multple attrbutes based on lnear utlty functon. Among these fve schemes, SAW wth AHP provdes hgher utlty than fuzzy logc and WMC snce t s precse, whle knapsack wth SA provdes hgher utlty than game between users snce t takes much more tme to search for the network wth the maxmum utlty. However, t s unfar to compare the total utltes of all the schemes together snce they are actually sutable for dfferent stuatons: SAW wth AHP, fuzzy logc and WMC are sutable for the case when traffc s not a key factor, whle game between users and knapsack wth SA are sutable for the case when resource of some networks becomes tght. For the scheme sgmodal utlty and the ntegrated scheme, t s unfar to compare wth other schemes on the total utlty snce they use actually a totally dfferent way to evaluate the total utlty. Sgmodal utlty scheme uses sgmodal functons to adjust the utltes of attrbutes, so t assumes that the best network should be wth the maxmum adjusted utlty, not the maxmum unadjusted utlty. The ntegrated scheme combnes traffc nto the total utlty, so the defnton of the total utlty s dfferent from others. If we use ths defnton to evaluate the utlty of dfferent schemes, the ntegrated scheme s surely wth the maxmum utlty, but we feel t unfar for other schemes n ths knd of comparson. That s also why we provde general comparson of dfferent schemes total utltes, nstead of demonstratng them n fgures. wth the ntegrated scheme, traffc loads of dfferent networks are {2, 11, 12, 11}. Consderng that WWAN s domnated by WMAN, t s qute correct to not select WWAN untl there s not enough space n WMAN. Traffc loads of dfferent networks usng game between users and knapsack wth SA are {6, 12, 6, 12} and {2, 12, 10, 12}, respectvely. Therefore, consderng traffc load balancng, we can see that our ntegrated scheme s equally good as knapsack wth SA, whle we do not have to use a slow optmzaton algorthm, such as SA, n our ntegrated scheme. IX. CONCLUSION Network selecton has been wdely studed by usng varous mathematcal theores n the lterature. The employed theory s extremely mportant because t decdes the objectve of optmzaton, complexty and performance, but there lacks a tutoral on the mathematcal models used for the network selecton problem. Therefore, ths paper flled the blank by conductng a serous survey and provdng a systematc tutoral on the man mathematcal theores used for ths problem, ncludng utlty theory (cost functon), MADM, fuzzy logc, game theory, combnatoral optmzaton, Markov chan. A unfed scenaro was used to explan and compare selected network selecton schemes usng these theores. In the end, the ntegraton of multple of these theores was dscussed, and an ntegrated scheme combnng the advantages of several mathematcal theores was proposed and compared wth selected schemes.

20 20 IEEE COMMUNICATIONS SURVEYS & TUTORIALS, ACCEPTED FOR PUBLICATION X. NOTATIONS C: total coeffcent of combnng multple attrbutes K: number of users or applcatons M: number of attrbutes N: number of networks c k : capacty cost of applcaton k n network n : network v j : normalzed value of attrbute j n network wj k : weghts of attrbute j for applcaton k x j : value of attrbute j n network Utlty Theorey (Cost Functon): N( ): normalzaton of certan utlty F : total cost of network U: total utlty of all the attrbutes fj k ( ): weghtng functon of attrbute j for applcaton k u k j : utlty of applcaton k n network n terms of attrbute j ǫ k j : network elmnaton factor for applcaton k, network and attrbute j MADM: C kl : concordance set ncludng the attrbutes on whch network k s better than network l D kl : dscordance set ncludng the attrbutes on whch network k s worse than network l D α : Eucldean dstance from certan network to the worst reference network D β : Eucldean dstance from certan network to the best reference network : value of the jth attrbute of the worst reference network Vj α V β j : value of the jth attrbute of the best reference network B: par-wse comparson matrx between all the attrbutes R: relatonshp matrx between events and attrbutes d: bnary vector denotng true or false of events e: weghts of all the events I: dentty matrx w: weghts of all the attrbutes E: number of events b j : comparson value between the th and the jth attrbutes n B r j : strength of the effect from the th event to the jth attrbute x j : mean value of all the networks n terms of attrbute j λ: egenvalue of B Λ j : nomnal value of attrbute j Fuzzy Logc: Fj l : fuzzy set for the jth nput n fuzzy rule l G l : fuzzy set for the output n fuzzy rule l X j : the jth nput of a fuzzy logc system Y: output of a fuzzy logc system Game Theory: B k : type space of player k n Bayesan game K : set of users N : set of networks Q: set of Bayesan strateges B k (q k, B k ): best response of player k n Bayesan game K : number of users choosng network p (a) : vector of proporton of users choosng dfferent networks n servce area a q k : Bayesan strateges of all the players except k B k : mnmum bandwdth requrement as the type of player k n Bayesan game c k ( ): cost of user k n the congeston game p (t): proporton of users choosng network : proporton of users choosng network n servce area a q k : Bayesan strategy of player k ζ k : bnary varable representng whether user k s wthn the coverage of network η k : bnary varable representng whether user k selects network π (t): payoff of the users choosng network n the evolutonary game π(t): average payoff of the entre populaton π k : expected payoff of player k as bandwdth utlty mnus connecton fee p (a) Combnatoral Optmzaton: U: total proft C : capacty of network z k : bnary varable representng whether applcaton k selects network ψ k : proft of applcaton k selectng network Markov Chan: S: state space A (s): set of avalable actons at state s R : combnaton of all the other attrbutes except VHO cost S( ): state denoted by the area covered by a certan bunch of networks T : threshold between the selecton of dfferent permutatons Y: weghted Markov chan transton matrx f: statonary dstrbuton vector t: decson epochs T : number of epochs durng a sesson lfetme n an MDP a t : acton at epoch t f(s t, a t ): beneft of usng acton a t from state s t g(s t, a t ): cost of usng acton a t from state s t h H : average horzontal handover cost : average cost of vertcal handover to a better network h + V h V : average cost of vertcal handover from the current best network r(s t, a t ): one-step reward usng acton a t from state s t s t : state at epoch t n an MDP y kl : element n Y, representng the dfference between network k and l δ t : epoch t durng a sesson lfetme γ: dscount factor mappng the future reward to the current state σ : probablty that network s avalable ρ(y s, a): transton probablty from state s wth acton a n dscrete MDP ρ(y s, a): transton probablty from state s wth acton a n contnuous MDP