Reputtion mngement for distributed service-oriented rchitectures Article Accepted version C. Crpnzno, F. Milzzo, A. De Pol, G. Lo Re In Proceedings of the Fourth IEEE Interntionl Conference on Self- Adptive nd Self-Orgnizing Systems Workshop (SASOW), 2010, pp. 160-165 It is dvisble to refer to the publisher s version if you intend to cite from the work. Publisher: IEEE http://ieeexplore.ieee.org/stmp/stmp.jsp?tp=&rnumb er=5729615 NDS LAB - Networking nd Distributed Systems http://www.dicgim.unip.it/networks/
Reputtion Mngement for Distributed Service-Oriented Architectures Clogero Crpnzno, Fbrizio Milzzo, Alessndr De Pol, nd Giuseppe Lo Re DINFO - Diprtimento di Ingegneri Informtic Università degli Studi di Plermo, Itly e-mil: {depol, lore}@unip.it Abstrct Nowdys, severl network pplictions require tht consumer nodes cquire distributed services from unknown service providers on the Internet. The min gol of consumer nodes is the selection of the best services mong the huge multitude provided by the network. As bsic criteri for this choice, service cost nd Qulity-of-Service (QoS) cn be considered, provided tht the underlying Service-Oriented Architecture (SOA) be ugmented in order to support the declrtion of this informtion. The correct behvior of such new SOA pltforms, however, will depend on the presence of some mechnisms tht llow consumer nodes to evlute trustworthiness of service providers. This work proposes new methodology for discourging ntisocil behviors of mlicious service providers tht declre QoS higher thn the rel one. The rchitecture is fully distributed over the network nd emultes decentrlized hierrchicl trusting uthority cpble of mnging reputtion vlues nd of providing correct QoS ssessments. Index Terms Reputtion Mngement; Distributed SOA; QoS-bsed Service Selection. I. INTRODUCTION During the lst few yers, Internet pplictions hve been gretly influenced by the introduction of innovtive softwre rchitectures nd new communiction protocols for the construction of service-oriented network infrstructures. A Service-Oriented Architecture (SOA) is softwre pltform tht describes the structure of service-oriented networks. Over the Internet, SOAs re typiclly implemented through the use of web services stndrds, nd rely on centrlized pproch, tht requires the presence of mster node mintining relevnt informtion bout network services nd the reltive providers. This pproch suffers from well-known limits of centrlized systems, i.e. lck of sclbility nd presence of single point of filure. Distributed SOA (D-SOA) [1] represent n importnt evolution of clssic SOAs nd cn overcome their limits using hierrchicl network structure, for distributing worklod mong network nodes. This rchitecturl prdigm is well-suited in those scenrios in which trusting uthority is implicitly distributed, for instnce, s in Virtul Orgniztions, where gents belonging to different orgniztions interct by trding services. Ech orgniztion is responsible for its own resources nd it is not fesible to entrust the mngement of ll network resources to centrlized trusting uthority. In order to overcome this problem, we propose hierrchicl structure in which ech orgniztion represents trusting uthority for services held by its providers. However, the lck of centrlized trusting uthority my encourge ntisocil behviors, such s declring flse QoS vlues. This behvior is fully explined by gme theory, ccording to which, the nlysis of gent interctions in rel complex scenrio cnnot tke into ccount the qulity of being honest. On the contrry, ech gent selects its own ctions in order to chieve its mximum dvntge, to the best of its knowledge, even if they cuse dmges to other gents [2]. In our scenrio such n opportunistic behvior consists in the untruthful declrtion of QoS vlues higher thn the rel ones, in order to guiltily promote inferior services. This considertion imposes tht the trditionl centrlized trusting uthority is replced by distributed one tht offers equivlent functionlities. The min contribution of our work is the definition of distributed reputtion mngement system tht llows to set up distributed trusting uthority. The reputtion mngement system evlutes the relibility of the service provider declrtions nd supports service consumers in the selection of truthful providers, filtering declred QoS vlues in order to obtin the ctul ones by exploiting users feedbck. According to the txonomy presented in [3], our system cn be defined s personlized nd decentrlized. It is decentrlized becuse of the lck of centrl entity mnging informtion; rther, informtion on reputtion re spred over the network. It is personlized becuse different nodes cn hve different reputtion vlues for the sme service provider; similrly, different nodes cn receive different QoS ssessments for the sme service. Provider reputtion is mnged by exploiting consumers feedbck, relesed fter service usge. The smller the difference between declred nd ctul QoS, the greter the client stisfction, with consequent increse of the provider reputtion. After n initil trnsitory phse, the system will converge towrd n ccurte estimte of the ctul QoS vlues. Reputtion vlues will be used in mechnism of penlties nd incentives, in order to llow consumer gents to identify mlicious untruthful nodes. The rest of the pper is structured s follows. In Sec. II other works presented in literture re described, s they contin some key concepts exploited in this work; Sec. III describes the proposed rchitecture, while detils bout the dopted policies for the mngement of QoS nd reputtion re provided in Sec. IV; Sec. V describes the gossip protocol tht performs the reputtion diffusion. Finlly, Sec. VI reports the experimentl results, nd Sec. VII sttes some conclusions.
II. RELATED WORK The pproches presented in the scientific literture relted to the problem of seprting mlicious nd truthful nodes in SOAs involve different techniques, such s exploittion of users feedbck in order to produce single QoS estimte, single provider reputtion estimte, or finlly hybrid QoS nd reputtion estimte. In [4], the uthors propose model for reputtion mngement in peer-to-peer networks. Informtion regrding the peers reputtion is mnged using d hoc developed byesin networks tht periodiclly re exchnged mong ll peers. The reputtion is updted by mens of reinforcement lerning technique. This work, to the best of our knowledge, is one of the firsts which propose the doption of reinforcement lerning in order to model reputtion. The decentrlized system for reputtion mngement nd service selection proposed in [5] exploits monitor nodes for collecting QoS vlues of the service providers. QoS vlues re compred with users feedbck in order to filter out deceitful providers through clustering methods. This work presents distributed form of trusting uthority, however it is extremely inefficient becuse of the computtionl complexity of the dopted filtering lgorithm. Authors of [6] propose the use of Certifiction Authority (CA) in order to check tht declred QoSs mtch their rel vlues. Such pproch does not tke dvntge from users feedbck nd presents centrlized bottleneck tht prevents the full sclbility of the system. Also utors of [7] rely on certifictes to gurntee gents trustworthiness, but in this work none centrlized certificte uthority is proposed; on the contrry, uthors propose fully distributed solution tht does not require to revel gents identity. A system cpble of integrting users feedbck nd reputtion is proposed in [8]. Reputtion is computed by weighted sum of users feedbck, s function of the feedbck ge. The pproch is fully centrlized, since user s feedbck re stored in centrlized dtbse. It is ssumed tht truthful service providers updte QoS informtion nd this lst ssumption is fully unrelistic for rel scenrio. Authors of [9] proposes trust network for multi-gent system tht exploits feedbcks individully provided by gents. Ech gent provides its own belief of trustworthiness of the others, s function of their pst observed behviors. The underlying Dempster Shfter theory of evidence llows to merge informtion coming from vrious gents nd to cope with the lck of informtion. In [10] uthors introduce the concept of service broker. The tsk of broker is to seek those services in the network tht better mtch user requirements, in order to mximize the customer utility function. The utility function for service is computed under different conditions of lod. Although broker bsed pproch my be useful in some rchitectures, the service broker computtionl lod my be excessive in networks with high density of providers. Finlly, uthors of [11] propose system tht expnds trditionl SOA with three dditionl components: QoS registries, Universl QoS mtching nd Web Service Broker. Brokers monitor the invoked services nd compute QoS vlue. Universl QoS mtching compute the service QoS by weighted sum of declred QoS, feedbck QoS nd monitored QoS, in which weights re directly proportionl to the ge of the informtion. Such pproch, however, does not dopt ny reputtion mngement mechnism cpble of inducing service providers to declre rel QoS vlues. Works discussed here presented some key concepts, like trusting uthority, QoS estimte, reputtion mngement nd network monitors; nevertheless none of them merges ll these spects in comprehensive pproch. One of the contributions of our work is thus the integrtion of QoS estimte nd reputtion mngement in single distributed mechnism. The proposed system hs the dvntge of effectively detecting mlicious behviors, mintining the computtionl lod low thnks to the exploittion of hierrchy of uthorittive nodes. III. THE PROPOSED ARCHITECTURE Our work proposes system cpble both of mnging the providers reputtion nd of estimting rel QoS vlues in distributed SOA. The proposed rchitecture is well-suited for those scenrios in which trusting uthority is implicitly distributed, since it llows the chievement of high degrees of gurntee for the QoS, still mintining full utonomy for the locl resource mngement. Severl rel scenrios fll within this description, for instnce Virtul Orgniztions [12] nd Cloud Computing [13], whose min purpose is to dynmiclly coordinte different institutions in order to exchnge services nd dvertise new ones. Adopting our QoS-bsed rchitecture, ech institution cn select the best services it needs on the bsis of the QoS vlues estimtes nd discover mlicious provider exploiting the provided reputtion informtion. A. System Architecture - Overview From logicl point of view, the proposed D-SOA cn be seen s two-levels hierrchicl network. Top-level subsystem is constituted by set of Super Nodes forming n overly network nd cting s distributed trusting uthority. The low-level implements the service exchnge subsystem nd its components, clled Nodes, re service consumers nd providers. The min tsk of the distributed trusting uthority is the monitoring of service exchnge ctivity occurring t the low-level nd the provision of updted informtion on both QoS nd provider reputtion for supporting service selection. From physicl point of view, the whole network is prtitioned into smll clusters, clled domins, s shown in Fig. 1. Ech cluster is supervised by Super Node which is responsible for monitoring the ctivities of ll Nodes belonging to its domin. More precisely, Super Nodes re ctively involved in the initil service negotition phse nd in the finl phse of users feedbck mngement, whilst the ctul service exchnges occur through direct connections mong Nodes. B. System Architecture - Functionl View Super Nodes overly network implements the distributed trusting uthority by evluting the reputtion of service providers. Ech Super Node mintins reputtion vlue for
Fig. 1. Clustered hierrchy orgniztion of Super Nodes nd Nodes. ech Node in its domin nd lso, in order to mintin links to different domins, reputtion vlue for ny neighbor node in the overly network. The neighborhood reputtion llows Super Nodes to filter QoS informtion s function of the reputtion of the node which provides it. This filtering phse llows service consumers to correctly select the best services mtching with their requirements of qulity nd cost. In order to provide more detiled description of our rchitecture, the event flow generted by service request is nlyzed nd shown in Fig. 2. We distinguish four roles: Consumer Node: the service consumer (CN in Fig. 2); Provider Node: the service provider (PN); Seeker Super Node: the super node tht is the cluster-hed of the cluster hosting the Consumer Node (SSN); Mnger Super Node: the super node tht is the clusterhed of the cluster hosting the Provider Node (MSN). When CN looks for given service, it sends query to its SSN (1) tht, in turn, forwrds it to its Super Node neighbors in the overly network (2). We ssume, without loss of generlity, tht Super Nodes form fully connected overly network. Under this ssumption, ll Super Nodes in the network cn reply to the query. For the ske of simplicity, the ssumption lso llows us to disregrd problems relted to the query routing tht do not fll within the issues ddressed by this work. When Super Node receives query, it performs locl serch for the services provided by Nodes in its domin (3). Ech Node replies to the locl query declring the updted QoS vlues for the requested service, QoS decl (4). In this phse, these Nodes ct s PNs. The Super Node replies to the the SSN with list of services mtching the query, enriched lso by QoS informtion (5). In this phse, the queried Super Node cts s trusting uthority for QoS informtion, thus plying the role of MSN. In its gurntor role, ech MSN hs lso the cpbility of modifying QoS vlues. Hence, the SSN receives from certin number of MSN lists of services coupled with respective QoS vlues dvertised by respective MSNs, QoS dv. A comprehensive merged list is then forwrded by SSN to the CN (6). Here, lso the SSN hs the cpbility of fixing the received QoS Fig. 2. Event flow generted by service request. vlues thus producing the finl ccepted QoS vlues, QoS cc. The CN selects service out from the received list, nd s finl step closing the loop it determines feedbck vlue by estimting the perceived QoS, QoS percv, to be send bck to the SSN (7). As function of the received feedbck, the SSN is thus ble to updte its reputtion estimte of the MSN. This updte is performed ccording to function which tkes into ccount the similrity between dvertised QoS nd perceived QoS. Finlly, the SSN forwrds the sme feedbck vlue to the MSN (8), in order to enble it to pply the sme procedure to updte the reputtion of the PN in its domin. IV. QOS AND REPUTATION MANAGEMENT POLICY The system behvior hevily depends both on the policies dopted for the QoS nd reputtion mngement nd on the service selection methods. For this reson, in order to provide full specifiction of the system, we shll provide the description of the following functionlities: Service selection performed by CN fter receiving the service list (step 6); MSN reputtion mngement s performed by SSN fter receiving users feedbck (step 7); PN reputtion mngement s performed by the MSN fter receiving users feedbck (step 8). All the bove policies exploit the reinforcement lerning mechnism s their driving principle. A. The Adopted Reinforcement Lerning Model Reinforcement Lerning (RL) [14] is brnch of Mchine Lerning, modeling how gents lern which ctions to perform with the im of mximize score function, bsed on the results of pst interctions with the environment. The RL model ssumes tht, fter ech interction with the environment, the plying gent obtins rewrd for the performed ction. Such rewrds constitute the input dt of tril-nd-error lerning mechnism whose gol is the genertion of the best sitution-ction mpping to be considered for mximizing the verge rewrd. In order to select the ction to be performed, n optiml trde-off between the exploittion of the cquired knowledge nd the explortion of not-yet-evluted solutions must be chieved. The former criterion involves tht the gent
would choose the best ction given its current stte, wheres the ltter implies tht the gent would lso choose sub-optiml ctions in order to explore new outcomes. We identify the lerning gents with the network nodes, nd for ech gent its environment is represented by ll the other nodes. In order to perform the reputtion mngement, QoS estimte, nd service selection, we dopted the Q-lerning [15], mong ll the proposed RL techniques, becuse it considerbly simplifies the formliztion of the lerning lgorithm nd it comes with forml proof of its erly convergence. In such method, the verge utility of performing n ction in stte s, referred s Q(s t, t ), is updted s function of the pst estimte nd of the rewrd r t+1 obtined fter the gentenvironment interction, ccording to the following eqution: Q(s t, t ) (1 α)q(s t, t )+ + α[r t+1 + γmx t+1, )], where Q(s t, t ) is the current estimte of the utility obtined by performing the ction t in the stte s t, s t+1 is the new stte in which the gent trnsits fter the ction performnce, r t+1 is the obtined rewrd, nd mx Q(s t+1, ) is the mximum rewrd obtinble in the new stte. The α nd γ prmeters, both rnging in [0, 1], control the lerning mechnism, nd represent respectively the lerning rte nd the discount fctor. The former determines the weight of new informtion with respect to the pst history, nd the ltter determines the influence of future rewrds. Bsed on the Q(s t, t ) vlues, the gent selects the ction to be performed with technique known s reinforcement comprison, ccording to which, ech ction cn be selected with probbility π directly relted to its estimted verge rewrd, computed s follows: π t (, s) = P r{ t = s t = s} = eq(st,t)/τ (1) eq(st,)/τ. (2) Such selection mostly stresses the choice of the best ction thus enbling the exploittion; however, since the probbility to select sub-optiml ctions is never 0, it lso llows the explortion. High vlues for the τ prmeter, temperture in the Boltzmnn distribution, mke the ctions quite equiprobble, while low vlues mke tht smll difference in ction utility correspond to big difference in ction selection probbility. B. QoS Filtering nd Service Selection In response to the query for service, the Consumer Node receives list of services mtching the query prmeters. These services re ssocited to some QoS informtion, dvertised by MSNs nd filtered by the SSN. QoS informtion filtering is performed by the SSN on the bsis of its reputtion vlue ssocited to the MSN tht provided such informtion. If rep represents the reputtion vlue of the MSN providing the QoS declrtion, rep mx the mximum reputtion vlue of ll neighbor Super Nodes, nd QoS dv the QoS vlue dvertised by the MSN, the filtering rule which determines the ccepted QoS, QoS cc, cn be written s: rep QoS cc = QoS dv. (3) rep mx After its filtering ctivity, the SSN forwrds the modified service list to the CN, in order to support it in the selection of best services. Selection is performed through reinforcement comprison method described in eq. 2, where the selection of service corresponds to n ction, nd the ction rewrd corresponds to the QoS of the selected service. Through this selection mechnism, the CN cts with the direct gol of mximizing its own utility nd with the indirect effect of penlizing mlicious PNs. Nmely, low reputtion vlue will correspond to low ccepted QoS vlues nd definitely this will led to less sold services. C. Mnger Super Nodes Reputtion Mngement After CN uses service, it replies its SSN, with feedbck vlue expressing the perceived QoS. The SSN exploits this informtion in order to updte the MSN reputtion. This updte opertion tkes into ccount the gp between QoS cc nd QoS percv. In such phse the SSN my choose mong three possible ctions: it my increse, decrese, or confirm its MSN reputtion. Intuitively, if the ccepted QoS is similr enough to the QoS perceived by the CN, the current estimte of the MSN reputtion vlue cn be considered correct nd then confirmed. Vice vers, if the filtered QoS vlue does not correspond to the perceived one, it is more pproprite to updte the reputtion estimte. In order to select the best ction to be performed, n d-hoc reputtion-lerning subsystem ws designed. All the possible sttes of the subsystem represent the set of possible reputtion vlues for the MSNs in the neighborhood of the SSN; the subsystem gol is to lern the utility vlue of ech ction in ll possible sttes. In this context, the utility vlue is function of the similrity between filtered nd perceived QoS vlues. In the current stte, for ech possible ction (in short: incr, decr, conf), the SSN evlutes which QoS vlue would hve trnsferred to the CN, using eq. 3; for ech of these three hypotheticl vlues, the SSN evlutes the difference between perceived QoS nd hypotheticl filtered QoS, QoS cc hyp. Finlly, this hypotheticl error, err hyp, is compred to the ctul one, err ct, in order to obtin the rewrd r for ll the possible ctions, ccording to the following equtions: err ct = QoS percv QoS cc, err hyp = QoS percv QoS cc hyp, r = err ct err hyp. Obviously, the ction of confirming the reputtion of the MSN hs null rewrd. The verge utility for ll ctions is updted using the Q- Lerning method s described in Sec. IV-A, s function of the computed rewrds. As regrds the current stte, represented by the current reputtion of the MSN, the rule for updting ctions utility is the following: Q(rep, incr) (1 α)q(rep, incr)+ + α[r incr + γmxq(rep + 1, ))], Q(rep, decr) (1 α)q(rep, decr)+ + α[r decr + γmxq(rep 1, )], Q(rep, conf) (1 α)q(rep, conf) + α[r conf + γmxq(rept, )]. (4) (5)
In summry, when SSN receives feedbck from CN, it performs the following ctions: 1) Compute the rewrd vlues, r incr, r decr, r conf (eq. 4); 2) Evlute the effects of possible ctions for updting MSN reputtion, by computing their utility vlues, Q(rep, incr), Q(rep, decr), Q(rep, conf), (eq. 5); 3) Select the ction to be performed through the reinforcement comprison method (eq. 2); 4) Updte the reputtion of the MSN ccording to the selected ction. D. Provider Nodes Reputtion Mngement The fct tht SSN evlutes the reputtion of MSN by estimting the relibility of the QoS informtion they dvertise, represents, ultimtely, the reson why MSN tht does not correctly certify the reputtion of the PN in its own domin my experience reduction of reputtion, since it is not ble to discover mlicious behviors. In order to void other Super Nodes discredit, ech Super Node mintins the reputtion vlues of ll the PN belonging to its domin, with the im to penlize them whenever they declre incorrect QoS vlues. The informtion necessry to mnge the reputtion of the PN is obtined from users feedbck tht re forwrded by the SSN. Such mngement policy mirrors the mngement policy of MSN reputtion crried out by the SSN, nd described in Sec. IV-C. Reputtion vlues of the PN re here used by MSN to filter t the origin dvertised QoS vlues, with mechnism equivlent to tht described in Sec. IV-B. V. GOSSIP PROTOCOL FOR REPUTATION DIFFUSION Achieving homogeneous evlution criterion in the distributed trusting uthority represents n importnt gol of our work. A problem present in the Super Node overly network, is tht locl reputtion estimtes of Super Nodes my significntly differ mong the overly Nodes. This is minly due to the fct tht the locl reputtion estimte of Super Node is performed only fter the usge of services provided by its domin. This mens tht if Super Node SN i does not require services gurnteed by Super Node SN j, it will not updte its locl vlue of reputtion t ll. Therefore, r j i (t), i.e. the reputtion of Super Node SN j s estimted by Super Node SN i, my differ substntilly from the one estimted by Super Node SN k. In order to chieve substntil greement in the Super Node overly network, we propose gossip-bsed protocol, for reputtion diffusion, bsed on previous work [2]. Such protocol ims to diffuse informtion relibility, so s to obtin view of the network s more homogeneous s possible. According to this protocol, ech Super Node periodiclly sends to its neighbors its reputtion tble. A Super Node tht hs received the reputtion tbles from its neighbors, merges received informtion into its own reputtion estimte with weight proportionl to the reputtion of the source. The reputtion merging is thus performed only if the source is considered relible, tht mens, only if its reputtion vlue exceeds given threshold. Given the Super Node SN i, the set K i of its relible neighbors is subset of the djcent Super Nodes, nd is clculted ccording to the following eqution: K i = {k : r k i (t) ρ}. (6) Exploiting the reputtion tbles received from its relible neighbors K i, SN i updtes the reputtion vlue regrding Super Node SN j, ccording to the following eqution: r j k (t) rk i (t) r j i (t + 1) = (1 β) rj i (t) + β k K i. (7) ri k (t) k K i The β coefficient tunes the weight of the gossip informtion with respect to the locl estimte; the ρ threshold in eq. 6 expresses the trustiness degree of Super Node in its neighbors. VI. EXPERIMENTAL RESULTS In order to evlute the system behvior, we performed wide set of simultions, through n d-hoc developed simultor. We report the results of experiments devoted to highlight how the reputtion mngement system motivtes MSN to correctly dvertise the reputtion of their domin PNs, nd how PNs re compelled to declre true QoS vlues. A. Advntges from correct reputtion mngement The first experiment focuses on the usefulness deriving to MSNs from correct mngement of the PN reputtion. In our setting the simulted network is composed of 150 nodes spred over 11 domins, contining the sme number of honest PNs, mlicious PNs nd CNs. Ech SN correctly mnges the reputtion only for prt of the mlicious PNs belonging to its domin, nd msquerdes for the other prt. The experiment consists of 10 simultions of 1000 steps nd the results were verged over ll simultions. Fig. 3 shows tht MSNs tht correctly mnge the reputtion of greter percentges of PNs chieve cler dvntge, since they obtin high vlues of reputtion, wheres the reputtion of mlicious MSNs decreses quickly over the time. Nmely, since SSNs must provide the most ccurte QoS estimtes to CNs, they drmticlly reduce the MSN reputtion until the ccepted QoS vlues, filtered ccording to eq. 3, mtch the perceived ones. B. Detection of Mlicious Provider Nodes In order to effectively detect mlicious PNs, MSN should ssign different reputtion vlues to PNs s function of declred QoS vlues. The second experiment ims to prove tht the reputtion mngement policy provides MSNs with this cpbility. The setting differs from the previous experiment since ll MSNs correctly mnge the reputtion of PNs. Fig. 4 shows the verge vlue of reputtion PNs, ggregted by the probbility of declring flse vlues. The reputtion scores of mlicious PNs decrese over the time, for the sme reson dduced in the previous experiment. This leds to n importnt conclusion: mlicious behviors re lwys detected, either by the decrese of the PN reputtion in the opinion of its MSN, or
Fig. 3. Comprison between the reputtion vlues of MSNs tht msquerde for different percentges of mlicious PNs. Fig. 4. Comprison between the reputtion vlues of PNs tht provides flse QoS vlues with different probbilities. by the decrese of MSN reputtion when it does not correctly dvertise its PN reputtions. C. The Economic Drwbck for Mlicious Provider Nodes A reputtion mngement system ble to detect mlicious nodes cn ctully led PNs to declre true QoS vlues only if this detection corresponds to some economic drwbck for mlicious providers. Such deterrent cn originte only from reduction of sold services. In this experiment, we use the sme bsic setting of of the previous ones. Results shown in Fig. 5 prove tht, fter trnsitory phse, during which services provided both by mlicious nd truthful PNs re chosen with the sme percentge, drstic reduction of the percentge of selected services (tht do not rech 5% for PN declring flse vlues more often thn 40%) is determined by the decrese of reputtion vlues for mlicious PNs. VII. CONCLUSIONS D-SOAs represent novel rchitecturl prdigm wellsuited in scenrios in which limits of clssicl SOAs, cused by their intrinsiclly centrlized nture, constitute severe disdvntges. D-SOA cn exploit dditionl prmeters, such s QoS, in order to support consumers in the selection of the best services. Unfortuntely the lck of centrlized supervising entity fvors ntisocil behviors. A hierrchicl reputtion mngement system ws proposed in order to effectively detect nd penlize mlicious behviors. Our system is bsed on Fig. 5. Comprison between the verge percentge of selected services provided by PNs with different probbility of being dishonest. decentrlized Reinforcement Lerning pproch, nd llows Consumer Nodes to lern the best service in order to mximize the perceived QoS nd motivte Provider Nodes to honestly behve, declring true QoS. REFERENCES [1] F. Bnei-Kshni, C. Chen, nd C. Shhbi, WSPDS: Web Services Peer-to-peer Discovery Service, in Proceedings of the Interntionl Conference on Internet Computing, 2004, pp. 733 743. [2] A. De Pol nd A. Tmburo, Reputtion Mngement in Distributed Systems, in 3rd Interntionl Symposium on Communictions, Control nd Signl Processing (ISCCSP), 2008, pp. 666 670. [3] J. Vssilev nd Y. Wng, A Review on Trust nd Reputtion for Web Service Selection, in 27th Interntionl Conference on Distributed Computing Systems Workshops (ICDCSW 07), 2007. [4] Y. Wng nd J. Vssilev, Trust nd Reputtion Model in Peer-to-Peer Networks, in IEEE Conference on P2P Computing, 2003. [5] L.-H. Vu, M. Huswirth, nd K. Aberer, QoS-bsed Service Selection nd Rnking with Trust nd Reputtion Mngement, in On the Move to Meningful Internet Systems, 2005, pp. 466 483. [6] S. Rn, A model for web services discovery with QoS, ACM SIGecom Exchnges, vol. 4(1), pp. 1 10, 2004. [7] Y. Mss nd O. Shehory, Distributed trust in open multi-gent systems, Trust in Cyber-societies, pp. 159 174, 2001. [8] Z. Xu, P. Mrtin, W. Powley, nd F. Zulkernine, Reputtion-enhnced QoS-bsed web services discovery, in IEEE Interntionl Conference on Web Services, 2007. ICWS 2007, 2007, pp. 249 256. [9] B. Yu nd M. Singh, An evidentil model of distributed reputtion mngement, in Proceedings of the first interntionl joint conference on Autonomous gents nd multigent systems (AAMAS). ACM, July 2002, pp. 294 301. [10] D. A. Mensce nd V. Dubey, Utility-bsed QoS Brokering in Service Oriented Architectures, in IEEE Interntionl Conference on Web Services, 2007, pp. 422 430. [11] G.-Q. Liu, Z.-L. Zhu, Y.-Q. Li, D.-C. Li, nd J.-C. Cui, A New Web Service Model Bsed On QoS, in Interntionl Symposium on Intelligent Ubiquitous Computing nd Eduction, 2009. [12] T. Normn, A. Preece, S. Chlmers, N. Jennings, M. Luck, V. Dng, T. Nguyen, V. Deor, J. Sho, nd W. Gry, Agent-bsed formtion of virtul orgnistions, Knowledge-Bsed Systems, vol. 17, no. 2-4, pp. 103 111, 2004. [13] R. Buyy, C. Yeo, S. Venugopl, J. Broberg, nd I. Brndic, Cloud computing nd emerging IT pltforms: Vision, hype, nd relity for delivering computing s the 5th utility, Future Genertion Computer Systems, vol. 25, no. 6, pp. 599 616, 2009. [14] R. Sutton nd A. Brto, Reinforcement Lerning: An Introduction. The MIT press, 1998. [15] C. Wtkins nd P. Dyn, Q-Lerning, Mchine Lerning, vol. 8, no. 3, pp. 279 292, 1992.