Article received on April 23, 2007; accepted on October 18, 2007

Transcription

1 A Renforcement Learnng Soluton for Allocatng Replcated Fragments n a Dstrbuted Database Una solucón de Aprendzae Reforzado para ubcar fragmentos replcados en Bases de Datos Dstrbudas Abel Rodríguez Morff 1, Daren Rosa Paz 1, Marsela Manegra Hng 2 and Lusa Manuela González González 1 1 Departamento de Cenca de la Computacón, 2 Departamento de Matemátca Unversdad Central Marta Abreu de Las Vllas, Carretera a Camauaní km. 5.5, C.P , Santa Clara, Cuba. Tel: +(53)(42) [email protected], [email protected], [email protected], [email protected] Artcle receved on Aprl 23, 2007; accepted on October 18, 2007 Abstract Due to the complexty of the data dstrbuton problem n Dstrbuted Database Systems, most of the proposed solutons dvde the desgn process nto two parts: the fragmentaton and the allocaton of fragments to the locatons n the network. Here we consder the allocaton problem wth the possblty to replcate fragments, mnmzng the total cost, whch s n general NP-complete, and propose a method based on Q-learnng to solve the allocaton of fragments n the desgn of a dstrbuted database. As a result we obtan for several cases, logcal allocaton of fragments n a practcal tme. Keywords: Dstrbuted database desgn, allocaton, replcaton, renforcement learnng, Q-Learnng. Resumen Debdo a la compledad del problema de la dstrbucón de los datos, la mayoría de las propuestas de solucón presentadas hasta la fecha han concddo en dvdr el proceso de dseño de la dstrbucón en dos fases seradas: la fragmentacón y la ubcacón de los fragmentos en los stos de la red. Este trabao aborda el problema de ubcacón de fragmentos partendo de un modelo matemátco que en su forma general es NP-Completo y propone un método metaheurístco basado en Q-Learnng de Aprendzae Reforzado que mnmza el costo total en un tempo aceptable. Esta propuesta ntegra la replcacón de fragmentos. Palabras claves: Dseño de bases de datos dstrbudas, ubcacón, replcacón, aprendzae reforzado, Q- Learnng. 1 Introducton In the last years there has been an enormous outgrown of dstrbuted nformaton systems. These systems have better adaptablty to the necessty of the decentralzed organzatons due to ther capacty to smulate the physcal structure of such organzatons. Though more advantageous than centralzed systems, the dstrbuted ones have some more complexty and ther desgn and mplementaton represent a challenge (Özsu and Valdurez, 1999). The desgn should be so that for the user the dstrbuton s transparent (Cer and Pelagatt, 1984). The desgner must defne the dstrbuton of the nformaton n the network maxmzng the localty dstrbutng the workload. Ths problem s known as dstrbuton desgn and nvolves two man tasks: fragmentng and allocatng (Özsu and Valdurez, 1999). Varous approaches have already been descrbed for the fragment allocaton n dstrbuted database systems (Hababeh et al., 2004; Ma et al., 2006). The allocaton of the data nfluences the performance of the dstrbuted systems gven by the processng tme and overall costs requred for applcatons runnng n the network. Some allocaton methods are lmted n ther theoretcal and mplementaton parts. Other strateges are gnorng the optmzaton of the transacton response tme. The other approaches present exponental tme of complexty and test ther performance on specfc types of network connectvty. Most of these methods are qute complex, not well understood and dffcult to use n a real lfe. Many assumptons have been done n order to smplfy the problem, so

2 118 Abel Rodríguez Morff, et al. the solutons are applcable n specfc condtons. For example, most of the proposed solutons consder that each fragment must be allocated n only one locaton smplfyng the soluton space but mssng the advantages of replcaton. In ths paper, we propose a method for allocatng database fragments to locatons usng as a reference the general model by (Özsu and Valdurez, 1999). Fndng an optmal fragment allocaton n ths model s a NP-complete problem snce gven n fragments and m locatons, there wll be ( 2 m 1) n dfferent combnatons. In our case, t s would very dffcult to reach and optmal usng an exact method because the problem s computatonally very complex. Therefore, ths approach proposes a metaheurstc algorthm to ad allocaton decson based on Q-Learnng from Renforcement Learnng technques (Watkns, 1999). 2 The allocaton problem Accordng to (Özsu and Valdurez, 1999), the allocaton problem consders a set of fragments F = {f1, f 2,..., f n }, a set of locatons L = {l1, l 2,..., l m } n a network, and a set of applcatons A = {a 1, a 2,..., a q } placed at L. These applcatons need to access the fragments whch should be allocated n the locatons of a network. The allocaton problem conssts on fndng an optmal dstrbuton of F over L. We consder the allocaton problem that mnmzes the overall cost subect to some constrants as n (Özsu and Valdurez, 1999). Furthermore, here we consder the replcaton opton whch makes ths problem more complex and t s known to be NP-complete, thus there s not a polynomal tme algorthm to solve t. Let us defne the problem n some more detals. The decson varable x = 1 f f s stored at locaton l k ; else t s 0. Before we derve the cost formulas, some nformaton must be analyzed n advance. That s, the quanttatve data about the database, the applcatons behavor, the locatons and network nformaton. Database nformaton: - sel (f ) s the number of tuples n f that need to be accessed by applcaton a. - sze( f ) = card(f ) length(f ) corresponds to the sze of fragment f. Applcaton nformaton: - RR s the number of read accesses of applcaton a to fragment f. - UR s the number of update accesses of applcaton a to fragment f. Two access matrces, UM and RM, that descrbe the retreval and update behavors of all the applcatons are also needed. The elements u and r are specfed as follows: - u = 1 f a updates f ; else t s 0. - r = 1 f a reads f ; else t s 0. - o( ) s the locaton where applcaton a orgnates. Locaton nformaton: - USCk s the untary cost of storng data at locaton l k. - SPC k s the untary cost of processng at locaton l k. Network nformaton: - g s the communcaton cost per frame between locatons l and l. - fsze s the frame sze measured n bytes. The total cost functon has two components: the applcatons processng cost and the storage cost. It s expressed as follows:

3 A Renforcement Learnng Soluton for Allocatng Replcated Fragments n a Dstrbuted Database 119 TOC = QPC + STC, a A lk L f F where QPC s the processng cost of applcaton a, and STC s the storage cost of fragment f at locaton l. The storage cost s gven by: STC = USC sze f x k ( ) For each applcaton a, the processng cost s calculated as the cost of processng (PC) plus the transmsson cost (TC). Processng costs contans three factors: access costs, ntegrty enforcement (IE) costs, and concurrency control (CC) costs: PC = AC + IE + CC The specfcaton of each cost depends on the used algorthm to make these tasks. The detaled specfcaton of AC would be: AC = u UR + r RR x SPC ( ) lk L f F The operatonal costs of data transmsson for update and read-only applcatons are dfferent. For update applcatons t s necessary to update all exstng replcas and read-only applcatons ust need to access to one of the copes. In addton, at the end of an update operaton there s only an update confrmaton message, and read-only applcatons may cause a large amount of data transmsson. The update component of the transmsson functon s: TCU = u x g + u x g l L F F o(),k l L F F k k k k,o() The cost of the read-only applcatons s: sel (f ) TCR = mn(r x go(), k + r x length(f ) gk,o() ) f F l L k fsze The constrants are: - Response tme constrant: The executon tme of an applcaton must be less or equal than the maxmum allowable response tme for that applcaton. - Storage capacty constrant: For each locaton, the total storage cost of fragments assgned to ths locaton must be less or equal than the capacty of the locaton. - Processng capacty constrants: For each locaton, the total processng cost for all the applcatons executed at ths locaton must be less or equal than the processng capacty of the locaton. 3 Soluton approaches There are dverse ways to attack combnatoral optmzaton problems (Huang and Chen, 2001; Ln and Orlowska, 1995; Ma et al., 2006; March and Rho, 1995; Pérez et al., 2005; Pérez et al., 2004; Pérez et al., 2003; Pérez et al., 2003; Pérez et al., 2002; Wolfson and Jaoda, 1995). These methods nclude exact and heurstcs approaches whch have been very useful n solvng real lfe problems. In ths secton we menton some of them and analyze ther applcablty. 3.1 Exact methods The total enumeraton s the method that chooses the best soluton out of all the possble solutons. A more sophstcated way s partal enumeraton, leavng out certan areas of the soluton space that for certan do not nclude any optmal soluton, here we can menton Branch and Bound, Cuttng planes and Dynamc programmng methods. The man problem wth these methods s ther applcablty to large problems, specfcally for the type of NPcomplete problems for whch there s no guarantee to fnd an optmal soluton n a polynomal tme (Ln et al., 1993). k

4 120 Abel Rodríguez Morff, et al. 3.2 Heurstcs methods A good alternatve for NP-complete combnatoral optmzaton problems of large sze s to fnd a reasonable soluton n a reasonable tme (Papadmtrou, 1997). Ths s the dea of the heurstc methods whch are n general qute smple and based on ntutve and common sense deas. The general problem wth many heurstcs s that they may get stuck n local optmal solutons. More recently a number of metaheurstcs have evolved that defne ways to escape local optma. Metaheurstcs are hgher level heurstcs desgned to gude other processes towards achevng reasonable solutons. The most wdely used metaheurstcs are Smulated Annealng, Genetc Algorthms, Tabu Search and GRASP. These methods do not guarantee n general that one wll fnsh wth an optmal soluton, though some of them present convergence theores. However, they have been successfully appled to many problems. Here we explore a much more recently approach, Renforcement Learnng. Ths approach may be nterpreted as a conuncton between machne learnng and decson makng problems. 4 Renforcement Learnng Renforcement Learnng (RL) s an approach to solve sequental decson makng problems that can be modeled by Markov Decsons Processes. The man dea of RL s the contnuous nteracton between an agent and ts envronment. Through ths nteracton the agent tres control actons for the current state of the envronment, nfluencng the next state; dependng on the chosen acton and the new state, the agent receves a reward/penalzaton sgnal. Ths way, the agent should learn to behave n order to acheve ts goal. Ths approach have been successfully appled to several decson and optmzaton problems (Abe et al., 2003; Cho et al., 2004; Ida et al., 2004; Morales and Sammut, 2004). 4.1 Basc elements of Renforcement Learnng An RL-system has manly two components: the Envronment where the process to be studed takes place, and an agent (RL-agent) who should be able to learn to control the process. The Envronment n general, s assumed to be a Markov decson process (MDP) whch s characterzed by states, rewards and transtons (Puterman, 1994). An RLagent s characterzed by the goal, a knowledge structure, a learnng method to update ts knowledge and a specfc behavor (polcy). Fgure 1 summarzes the communcaton between the agent and ts envronment. At each decson moment, the agent observes the current state (1) of the envronment and performs an acton (2) selected accordng to ts decson polcy. As a result of the receved acton n the envronment (3), a transton to a new state takes place and a renforcement or reward sgnal s generated (4). The reward sgnal and the new state are receved by the agent (5) and can be used through the agent's learnng method n order to update ts knowledge about the envronment, and consequently t can update ts polcy (6). Rewards and state transton functons may be n general stochastc, and the underlyng probablty dstrbutons are assumed not to be known to the agent. The problem conssts of fndng a prescrptve rule to choose an acton for each state such that a certan crteron s optmzed (the goal); here we consder the mnmzaton of the total expected dscounted reward. A dscount factor 0<γ<1 corresponds to the dea that rewards are less attractve n the far future.

5 A Renforcement Learnng Soluton for Allocatng Replcated Fragments n a Dstrbuted Database 121 Envronment current state, acton 4 reward sgnal, next state current state 1 current state Agent 2 acton reward sgnal, next state acton 3 reward sgnal, next state 5 6 Updatng knowledge and behavor Fg. 1. Agent-Envronment nteracton 4.2 Q-Learnng Our focus s on RL control usng Q-Learnng (QL) methods that try to learn the optmal acton value functon Q as a way to obtan an optmal polcy (Watkns, 1999). Ths functon s defned for each state-acton par (s, a) as the total expected dscounted reward startng n state s, takng acton a and thereafter followng the optmal polcy. Here the RL-agent's knowledge s an estmaton of the optmal acton-value functon Q. The classcal representaton for the estmaton of the acton-value functon s a lookup table. In ths case, for each state-acton par (s, a) there s an entry n the table whch s the correspondng approxmated acton value Q(s, a). Ths estmaton s updated durng the learnng process. The agent starts wth some estmaton and at each decson moment n whch the envronment s n state s the agent chooses an acton a accordng to ts behavor. The envronment reacts to the taken acton by gvng a reward r to the agent and changng to a new state s n the next decson moment. Wth ths new nformaton the agent updates the Q-values. The update rule for ths method, gven the experence tuple < s, a, r, s >, s as follows: Q(s,a) Q(s,a)+α[r+γ max a Q(s,a )-Q(s,a)] (1) Ths method converges to the optmal acton values wth probablty 1 as long as all pars (s, a) are vsted nfntely often and the learnng rate s reduced over tme accordng to the usual stochastc condtons for the learnng (Sutton and Barto, 1998). However, t s surprsng how many successful applcatons usually do not use step-szes satsfyng these condtons. The behavor defnes how the agent chooses the actons. The deal acton n a gven state s the one whch maxmzes the acton-value functon, the greedy acton. However, f we always choose the greedy acton based on the actual knowledge, many relevant state-acton pars may never be vsted due to the naccurate estmaton of the acton-value functon. Effcent exploraton s fundamental for learnng. Too much exploraton can cause nearly random behavor and too lttle can lead to non-optmal solutons. Ths s known as the explotaton-exploraton tradeoff. Explotaton deals wth the use of the avalable knowledge for example by choosng the greedy actons. Exploraton ncreases experence for example by choosng actons at random. Here we use the ε-greedy exploraton rule wth a parameter ε decreasng over tme. Wth probablty 1-ε one chooses a greedy acton wth respect to the current estmate of the Q-values and wth probablty ε a random acton.

6 122 Abel Rodríguez Morff, et al. 5 Implementaton of the Q-Learnng method for allocatng fragments Here we descrbe the elements of the RL-system to solve the allocaton problem presented n Secton 2. The envronment should nclude all the necessary nformaton needed by the MDP model to defne the dynamcs of the system: states, actons, mmedate rewards, and transtons. It takes nto account the restrctons of the problem. Makng an analogy between the allocaton problem and an MDP problem, we consder the soluton of the allocaton problem as the optmal polcy for the MDP problem. Ths soluton/polcy ndcates for each possble par of fragment and locaton whether the fragment should be located n the locaton or not. That s why the states should be all the pars (fragment, locaton). The acton space has only two actons: allocate the fragment n the locaton or do not allocate. The mmedate reward evaluates the cost of the taken acton for the current state. The problem here s that, as explaned n Secton 2, the allocaton cost of a fragment depends on several costs related to the other locatons where ths fragment s also allocated (remember we are consderng replcaton) and the applcatons accessng those locatons. Thus, all ths nformaton s related to prevous states and prevous actons. Ths breaks the Markov property;.e., the reward functon only depends on the current state and the acton taken n that state. In our approach, we save the necessary nformaton n the current soluton, whch s updated every tme an acton s taken, and n the data from the applcatons, ncludng ths nformaton n the state representaton mght create a very complcated structure. We choose nstead a nave strategy whch s to gnore the nformaton for the state, thus the states are lke actual observatons. Ths strategy have been used n dealng wth Partally Observed MDPs, small volatons of the Markovan propertes are well handled by Q-learnng algorthms (Kaelblng et al., 1996). Summarzng, the envronment has the followng elements. State Space: Is the set {( f, lk ) = 1...n, k = 1...m}. Each state represents a par (fragment, locaton). Actons: At each state (fragment, locaton) the agent could choose between two actons: allocate the fragment n the locaton (acton=1) or not (acton=0). Immedate reward: Gven the current state s= f,l ) and acton a, the current soluton s updated makng x = a, ( k and the mmedate reward ra evaluates the cost of allocatng the fragment f takng nto account the updated current soluton. Accordng to the problem descrpton n Secton 2, ths cost s gven by: TOC = AC + IE + TC + STC where: AC = a A ( u UR + r RR ) sk S TC = TCU + TCR TCU = u x g sk S o(),k + sk S x u x sk S SPC g k k,o() sel(f ) TCR = mn(r x go(),k + r x length(f) gk, o() ) sk S fsze Thus, we consder an epsodc task. At each epsode there are n tmes m transtons, gong through the whole state space. In our experments, we use 200 epsodes. The RL-Agent should learn to allocate database fragments n the locatons of the network n such a way that the total costs are mnmzed. It s characterzed by the knowledge structure, the polcy and the learnng method. The learnng process s epsodc; at each epsode the RL-Agent vsts all the states and updates ts knowledge. The knowledge structure: The Q acton-value functon represented by the Q(s, a) matrx s updated usng equaton (1) at each transton. The transtons are determnstc; actually the next state only depends on the prevous state, see table 1. In a determnstc world ths matrx may be ntalzed arbtrarly and we consder a zero matrx.

7 A Renforcement Learnng Soluton for Allocatng Replcated Fragments n a Dstrbuted Database 123 Table 1. State transtons The polcy: The agent follows an ε-greedy polcy based on the current acton-value functon gven by the Q matrx. Next, we summarze the algorthm. Besdes ntalzng the Q matrx as a zero matrx, we need an ntal soluton satsfyng all the constrants for the problem, excludng the storage capacty constrant that was dsregarded. Ths soluton s used to evaluate the costs of each acton and t s ntalzed allocatng each fragment randomly n only one locaton. Algorthm 1. A Q-Learnng method for allocatng fragments Durng the learnng process, after an acton s chosen, ths soluton s updated takng the acton nto account. At the end of each epsode the soluton matrx s updated consderng a greedy polcy wth respect to the current Q matrx, that s, assgnng to each state the acton wth the smallest value n the Q matrx. Thus, any fragment can be allocated to more tan one locaton snce t s possble to fnd for one fragment n the state space the smallest value n the Q matrx correspondng to the same acton=1. If a fragment s not allocated n any locaton (unfeasble soluton) we force the fragment to be n the locaton wth the smallest dfference n the acton-values for that fragment. In the algorthm we keep record of the best feasble found soluton (btsol), whch s ntally the same ntal soluton, not necessarly feasble (goodsoluton false). A parameter tunng was completed as a result of the tests appled to the

8 124 Abel Rodríguez Morff, et al. algorthm 1. Parameters α, ε and γ were fxed wth values 0.5, 0.99 and 0.8 respectvely. For the mplementaton we used Vsual C# from Vsual Studo 2005 by Mcrosoft. 6 Expermental cases In order to test the ntellgent RL-Agent from Secton 5 n the allocaton problem, we generate 15 random test cases gven the rank of values for the amount of fragments (FRG.), locatons (LOC.) and applcatons (APP.). In the frst three cases we obtaned the optmal soluton usng an exact method. In the rest of the cases ( ), we report the best soluton found by usng heurstcs lke Smulated Annealng and Genetc Algorthms (Rosa, 2006). Table 2 shows these cases and the lesser tme elapsed to fnd the soluton for the total cost functon (TOC) as n secton 5, namely the best soluton costs (BSC) measured n mllseconds. Table 2. Expermental cases 7 Expermental results Here we present the expermental results of our approach n order to evaluate the qualty of the soluton and the processng tme. Tmes are also consdered n mllseconds. Table 3 shows these results. Each case has been solved 20 tmes and the average cost was found (AVE). The table also shows the tmes the best known soluton was found (TB), the cost of the worst found soluton (WORST) and the absolute value of the relatve error (RE) calculated as follows: AVE BEST RE = 100 BEST The best known soluton obtaned by the Q-Learnng s guaranteed to converge to the optmal due to the use of a lookup table to store the Q-values. Every state-acton par contnues to be vsted, and the learnng rate s decreased approprately over tme. It has been assumed our functon approxmator s a lookup table. Ths s normally the case n classcal dynamc programmng. However, ths assumpton lmts the sze and complexty of the problems solvable.

9 A Renforcement Learnng Soluton for Allocatng Replcated Fragments n a Dstrbuted Database 125 Table 3. Expermental results Fg. 2. Executon tme for comparng methods Fgure 2 depcts the tme elapsed to fnd the optmal soluton for every case by usng a Genetc Algorthm n (Rosa, 2006) and the Q-Learnng method proposed here. Both methods have been developed to fnd a near-optmal allocaton such that the total cost (TOC) s mnmzed as much as possble. These methods use the same quanttatve data about the database, the applcatons behavor, the locatons and network nformaton. Both methods also use the same cost functon. As a result, we fnd that the Q-Learnng method performed better than Genetc Algorthm n (Rosa, 2006) for smaller cases. The relatve error values for both methods are farly smlar n most cases. 8 Conclusons We consder the allocaton problem n a dstrbuted database and propose an ntellgent RL-Agent to solve t by usng the Q-Learnng algorthm. The mprovements are sought by mantanng copes of the fragments. The decson regardng replcaton s a trade-off whch depends on the rato of the read-only applcatons to the update applcatons. The problem s then modeled usng the Renforcement Learnng approach and the algorthm s adapted to the problem. The parameters are chosen through expermental experence and are by no means optmzed. Ths method

10 126 Abel Rodríguez Morff, et al. neglects ntegrty enforcement and concurrency control costs as well as the storage capacty constrant, as an attempt to reduce the complexty of the problem. The results show that ths approach could obtan practcal solutons, even for larger cases. We beleve the RL approach s a very flexble method that can be appled to obtan good solutons n complex problems. 9 Recommendatons and future work Further research could study the tunng of the parameters nvolved n the algorthm. Specfcally we would lke to recommend the tunng of the number of epsodes accordng to the sze of the problem,.e., fragments tmes locatons. Many real-world problems have extremely large or even contnuous state spaces. In practce t s not possble to represent the value functon for such problems usng a lookup table. Hence, we recommend usng a functon approxmator that can generalze and nterpolate values of states never before seen as an extenson to classcal value teraton. For example, one mght use a neural network for the approxmaton. At ths moment we work on the ntegraton of several algorthms for the allocaton problem, specfcally Q- Learnng and SARSA from RL, Genetc Algorthms, Brd Flocks and some other tools developed by our research group. The man goal s to help n the desgn of Dstrbuted Databases n a more effcent way by usng less effort and tme. 10 Acknowledgements Ths work s part of a research and development Proect Database Technologes appled to problems, Number from the Natonal Program of Informaton Technology supported by the Cuban Mnstry of Scence, Technology and Envronment. The authors would lke to thanks to the Flemsh Interunversty Councl (Vlaamse InterUnverstare Raad) for ther support through the IUC VLIR-UCLV Program. References 1. Abe, N.; Bermann, A. W. and Long, P. M. (2003). "Renforcement Learnng wth Immedate Rewards and Lnear Hypotheses." Algorthmca 37(4): Cer, S. and Pelagatt, G. (1984). Dstrbuted Databases: Prncples and Systems, McGraw-Hll Book Company. 3. Cho, S. P. M.; Zhang, N. L. and Yeung, D.-Y. (2004). Renforcement Learnng n Epsodc Non-statonary Markovan Envronments. Proceedngs of the Internatonal Conference on Artfcal Intellgence, IC-AI '04. Proceedngs of the Internatonal Conference on Machne Learnng; Models, Technologes and Applcatons, MLMTA '04, Las Vegas, Nevada, USA, CSREA Press. 4. Hababeh, I. O.; Bowrng, N. and Ramachandran, M. (2004). A Method for Fragment Allocaton n Dstrbuted Obect Orented Database Systems. Proceedngs of the 5th Annual PostGraduate Symposum on The Convergence of Telecommuncatons, Networkng & Broadcastng (PGNet), Lverpool, UK. 5. Huang, Y.-F. and Chen, J.-H. (2001). "Fragment Allocaton n Dstrbuted Database Desgn." Journal of Informaton Scence and Engneerng 17(3): Ida, S.; Kuwayama, K.; Kanoh, M.; Kato, S. and Itoh, H. (2004). A Dynamc Allocaton Method of Bass Functons n Renforcement Learnng. Proceedngs of the 17th Australan Jont Conference on Artfcal Intellgence AI 2004: Advances n Artfcal Intellgence, Carns, Australa, Sprnger. 7. Kaelblng, L.; Lttman, M. and Moore, A. (1996). "Renforcement learnng: A survey." Journal of Artfcal Intellgence Research 4: Ln, X. and Orlowska, M. E. (1995). "An Integer Lnear Programmng Approach to Data Allocaton wth the Mnmum Total Communcaton Cost n Dstrbuted Database Systems." Informaton Scences 85(1-3): Ln, X.; Orlowska, M. E. and Zhang, Y. (1993). On Data Allocaton wth the Mnmum Overall Communcaton Costs n Dstrbuted Database Desgn. Proceedngs of the Ffth Internatonal Conference on Computng and Informaton - ICCI'93, Sudbury, Ontaro, Canada, IEEE Computer Socety.

11 A Renforcement Learnng Soluton for Allocatng Replcated Fragments n a Dstrbuted Database Ma, H.; Schewe, K.-D. and Wang, Q. (2006). A Heurstc Approach to Cost-Effcent Fragmentaton and Allocaton of Complex Value Databases. The 17th Australasan Database Conference (ADC2006) Hobart, Australa, ACM Internatonal Conference Archve Proceedng Seres vol Australan Computer Socety, Inc. 11. March, S. T. and Rho, S. (1995). "Allocatng Data and Operatons to Nodes n Dstrbuted Database Desgn." IEEE Trans. Knowl. Data Eng. 7(2): Morales, E. F. and Sammut, C. (2004). Learnng to fly by combnng renforcement learnng wth behavoural clonng, ACM. 13. Özsu, M. T. and Valdurez, P. (1999). Prncples of Dstrbuted Database Systems, Second Edton, Prentce- Hall. 14. Papadmtrou, C. H. (1997). NP-Completeness: A Retrospectve. Proceedngs of the 24th Internatonal Colloquum on Automata, Languages and Programmng, ICALP'97, Bologna, Italy, Sprnger. 15. Pérez, J.; Pazos, R. A.; Frausto-Sols, J.; Reyes, G.; Santaolaya, R.; Frare, H. J. and Cruz, L. (2005). An approach for solvng very large scale nstances of the desgn dstrbuton problem for dstrbuted database systems. Proceedngs of the 4th Internatonal School and Symposum on Advanced Dstrbuted Systems (ISSADS2005). Lecture Notes n Computer Scence, Sprnger. 16. Pérez, J.; Pazos, R. A.; Mora, G.; Castlla, G.; Martínez, J. A.; Landero, V.; Frare, H. J. and González, J. J. (2004). Dynamc Allocaton of Data-Obects n the Web, Usng Self-tunng Genetc Algorthms. Proceedngs of the 17th Brazlan Symposum on Artfcal Intellgence- SBIA 2004, Advances n Artfcal Intellgence São Lus, Maranhão, Brazl, Sprnger. 17. Pérez, J.; Pazos, R. A.; Romero, D.; Santaolaya-Salgado, R.; Rodríguez, G. and Sosa-Sosa, V. J. (2003). Adaptve and Scalable Allocaton of Data-Obects n the Web. Proceedngs of the Internatonal Conference on Computatonal Scence and Its Applcatons - ICCSA 2003, Part I., Montreal, Canada, Sprnger. 18. Pérez, J.; Pazos, R. A.; Santaolaya-Salgado, R.; Frausto-Solís, J.; Rodríguez, G.; Cruz, L. and Bravo, M. (2003). Data-Obect Replcaton, Dstrbuton, and Moblty n Network Envronments. Revsed Papers of the 5th Internatonal Andre Ershov Memoral Conference, Perspectves of Systems Informatcs, PSI 2003, Akademgorodok, Novosbrsk, Russa, Sprnger. 19. Pérez, J.; Pazos, R. A.; Velez, L. and Rodríguez, G. (2002). Automatc Generaton of Control Parameters for the Threshold Acceptng Algorthm. MICAI 2002: Advances n Artfcal Intellgence, Second Mexcan Internatonal Conference on Artfcal Intellgence, Merda, Yucatan, Mexco, Sprnger. 20. Puterman, M. L. (1994). Markov Decson Processes: Dscrete Stochastc Dynamc Programmng. New York, NY John Wley & Sons. 21. Rosa, D. (2006). Aplcacón de técncas de Intelgenca Artfcal en la solucón del problema de ubcacón en el dseño de bases de datos dstrbudas. Departamento de Cenca de la Computacón. Santa Clara, Unversdad Central "Marta Abreu" de Las Vllas. BSc.Thess. 22. Sutton, R. and Barto, A. (1998). Renforcement Learnng: An Introducton. Cambrdge, Massachussets, USA., MIT Press. 23. Watkns, C. (1999). Learnng from Delayed Rewards. Cambrdge, UK, Kng s College. 24. Wolfson, O. and Jaoda, S. (1995). "An Algorthm for Dynamc Data Allocaton n Dstrbuted Systems." Inf. Process. Lett. 53(2):

12 128 Abel Rodríguez Morff, et al. Abel Rodríguez Morff was born on January 25, 1969 n Santa Clara, Cuba. In 1993 he obtaned hs B.Sc. n Computer Scence at the Central Unversty of Las Vllas, Santa Clara, Cuba, and n 1996 obtaned hs M.Sc. n Informaton Management at the Unversty of Havana, Havana Cty, Cuba. In 2007 he obtaned hs Ph.D. at the Central Unversty of Las Vllas on a research proect Integrated tools to desgn dstrbuted databases". He s a professor n the Department of Computer Scence, Central Unversty of Las Vllas. Daren Rosa Paz was born on September 18, 1982 n Vlla Clara, Cuba. In 2006 he obtaned hs B.Sc. n Computer Scence at the Central Unversty of Las Vllas, Santa Clara, Cuba. Hs current research nterests are Operatons Research and Dstrbuted Databases Desgn. He s assstant professor at the Central Unversty of Las Vllas. Marsela Manegra Hng was born on Aprl 15, 1972 n Santago de Cuba, Cuba. In 1995 she obtaned her B.Sc. n Computer Scence and n 1997 her M.Sc. n Appled Mathematcs at the Central Unversty of Las Vllas, Santa Clara, Cuba. In 1998 she graduated from the master class program n Operatons Research organzed by the Mathematcal Research Insttute n The Netherlands. In 2006 she obtaned her Ph.D. at the Unversty of Twente on a research proect Intellgent Computatonal Technques supportng Learnng Organzatons". She s assstant professor at Central Unversty of Las Vllas. Lusa Manuela González González was born on Aprl 25, 1954 n Sanct Sprtus, Cuba. In 1973 she obtaned her B.Sc. n Mathematcs at the Central Unversty of Las Vllas, Santa Clara, Cuba, and n 1995 she obtaned her Ph.D. on a research proect Integrated tool for engneerng desgn". Snce 1975, she has been a professor n the Department of Computer Scence, Central Unversty of Las Vllas. For almost twenty years, she has been the manager of the Database Research Group. Her current research nterests are data modelng, decson support systems, and ontologes.