MULTIDIMENSIONAL FUNCTION APPROXIMATION USING NEURAL NETWORKS

Size: px

Start display at page:

Download "MULTIDIMENSIONAL FUNCTION APPROXIMATION USING NEURAL NETWORKS"

Jasper Holland
7 years ago
Views:

1 MULTIDIMESIOAL FUCTIO APPROXIMATIO USIG EURAL ETWORS Enăchescu Căln Petru Maor Unversty of Targu Mures, ROMAIA Abstract: Solvng a problem wth a neural network a prmordal task s establshng the network topology. Generally neural network topology determnaton s a complex problem and cannot be easly solved. When the number of tranable layers and processor unts s too low, the network s not able to learn the proposed problem. When the number of layers and neurons s too hgh, the learnng process becomes too slow. Learnng from examples means beng able to nfer the functonal dependence between nput and output spaces X and Z, gven the knowledge of the set of examples T. It means that, after we have learned examples, when a new nput varable x comes n, we need to be able to estmate, accordng to some crteron that we wll specfy, a correspondng value of z. From ths pont of vew learnng s equvalent to a functon approxmaton. ey words: eural networks, approxmaton, learnng I. ITRODUCTIO From 986 the most popular neural network model s the Mult Layer Perceptron (MLP) and the most popular learnng algorthm s the Back-propagaton (BP) method [7]. In spte of the fact that the classcal MLP networks have many advantageous propertes, they have some dsadvantages, too. The most mportant dsadvantage s the slowness of the tranng procedure, caused by the hgh number of the tranable layers and the necessty of error back-propagaton. The tranng process could be faster f the number of tranable layers can be dmnshed. That was the motvaton for developng neural networks wth a sngle tranable layer. Radal bass functons were frst ntroduced n the soluton of the real multvarate nterpolaton problem [0]. Broomhead and Lowe (988) [] were the frst to explot the use of radal bass functons n desgnng neural networks. Other major contrbutons to the theory, desgn and applcaton of RBF networks nclude papers by Moody and Darken (989) [8] and Poggo and Gros (990) [9]. II. RBF EURAL ETWORS TOPOLOGY. RBF s a feed-forward neural network wth an nput layer (made up of source nodes: sensory unts), a sngle hdden layer and an output layer []. The network s desgned to VI-7-

2 perform a nonlnear mappng from the nput space to the hdden space, followed by a lnear mappng from the hdden space to the output space. The processor unts of the hdden layer are dfferent from the processor unts of the MLP networks. The actvaton functons are radal bass functons (for example Gaussan functons). These functons generally have two parameters: the center and the wdth [3]. The output layer s composed by processor unts, whch are creatng normal, smple lnear-weghted value, every unt producng an output. The network has a typcal property: the value of weghts between the nput and the hdden layer s. The archtecture of the RBF neural network s presented n Fgure. [4]. g x w x g w x n w g Fgure : RBF neural network topology. x x n s the nput vector, g () s the Radal Bass Functon and c s the center parameter for the functon correspondng to neuron, then the output created by the network wll be: If = ( x,..., ) y = = Generally Gaussan functon s used: w g (x) = w g( x c ) () = g x c σ ( x) e = () where σ s the scale parameter for functon correspondng to neuron. There are some methods to select the parameters ( c, σ ) of the actvaton functon. If few tranng ponts are present, then all of them could be used as center parameter. In ths case the number of the processor unts n the hdden layer s equal wth the number of tranng ponts. If the number of tranng ponts s hgh, then not all of them mght be used. In ths VI-7-

3 stuaton a sngle neuron for a group of smlar tranng ponts can be consdered. These groups of smlar tranng ponts can be dentfed usng clusterng methods [5]. III. LEAIG STRATEGIES FOR RBF EURAL ETWORS The hdden layer of the RBF eural etworks may be traned wth a supervsed learnng algorthm. A descendent gradent-based algorthm can be consdered. The am s to establsh the synaptc weghts w, =,,, of the network. Let T n {(, z ) x R, z, =,, } = x R (3), be the set of the tranng samples. A clusterng algorthm s used on the ponts of the set T. The cluster centers c, =,..., are consdered (n ths way the number of the neurons n the hdden layer s ). Parametersσ R, =,..., can be determned correspondng to the dameter of clusters. Ths step s not executed when s equal wth (=), because n ths case c = x, =,..., (every tranng pont s a cluster center too and the value of the wdth parameters s σ = /). If Gaussan functon s used as actvaton functon, then at the l th step the global learnng error s E l = = ( z y ) (4) where y = j= w e j ( x c j ) σ j, =,..., (5) Let us to note: w E = η, w =,..., (6) where η s the learnng rate and E s the global learnng error. Weghts updatng s based on the followng correcton rule: w = w + w, =,..., (7) When the learnng process s fnshed, M ponts, whch are not from the tranng set T, are randomly generated. The correspondng generalzaton error s defned by the expresson [5]: VI-7-3

4 E g = M M = ( z y ) (8) IV. APPROXIMATIO AD ITERPOLATIO WITH RBF EURAL ETWORS The nterpolaton problem, n ts strct sense, may be stated as follows: R,..., and a correspondng set of real numbers { d R =,..., }, fnd a functon F : R p R that satsfes the nterpolaton condton [], [6]: p Gven a set of dfferent ponts { x = } F ( x ) d, = = (9),..., The RBF technque conssts of choosng a functon F that has the followng form [0]: F ( x) = w g( x x ) = (0) where { g( x x ) =,..., } () s a set of arbtrary radal bass functons. The known data ponts taken to be the centers of the radal bass functons. p x R, =,..., are A RBF network s consdered, wth a sngle processor unt n the output layer, and processor unts n the hdden layer, where { g( x x ) =,..., } s the set of the actvaton functons for the hdden processor unts. The nterpolaton problem s reduced to the determnaton of weghts (learnng process) [3]. In an overall fashon, the network represents a map from the p-dmensonal nput space to the sngle-dmensonal output space, wrtten as: s : R p R () The map s could be consdered as a hypersurface multdmensonal plot of the output as a functon of the nput. Γ R p+. The surface Γ s a In a practcal stuaton, the surface Γ s unknown and the tranng data are usually affected by nose. Accordngly, the tranng phase and generalzaton phase of the learnng process may be respectvely vewed as follows [], [4]: - The tranng phase consttutes the optmzaton of a fttng procedure for the surface Γ, based on known data ponts presented to the network n the form of nput-output examples. - The generalzaton phase s a synonymous wth nterpolaton between the data ponts, wth the nterpolaton beng performed along the constraned surface generated by the fttng procedure as the optmal approxmaton to the true surface Γ. VI-7-4

V. UMERICAL EXPERIMETS. In ths secton some experments and the obtaned results are presented. Standard nterpolaton problems are consdered. RBF neural networks are used for approxmatng functons.

5 V. UMERICAL EXPERIMETS. In ths secton some experments and the obtaned results are presented. Standard nterpolaton problems are consdered. RBF neural networks are used for approxmatng functons. The generalzed k-means clusterng algorthm s used for data clusterng and some comparsons are presented [3]. Experment: In order to study the propertes of the RBF networks obtaned as a theoretcal result, we have mplemented ths type of neural network and studyng the learnng capabltes and the generalzaton capabltes. We have taken n consderaton as target functon, to be approxmated, the followng functon: f : R R, f ( x, y ) = 0 cos( x ) sn ( y ) (3) Fg..: 400 tranng data, 0 learnng epochs. Fg. 3.: 400 tranng data, 50 learnng epochs. Fg. 4.: 400 tranng data, 00 learnng epochs. Fg. 4.: 400 tranng data, 500 learnng epochs. VI-7-5

6 umber 400 tranng data of epochs Learnng Error Generalzaton Error Table: Results of the smulatons, descrbng number of epochs, learnng error and generalzaton error. VI. COCLUSIOS Experments descrbed n ths chapter demonstrate that RBF neural networks can be successfully used for multdmensonal functon approxmaton. REFERECES [] Broomhead D.S., Lowe D., (988), Multvarable functonal nterpolaton and adaptve networks, Complex Systems, [] Enăchescu, C. (995), Propertes of eural etworks Learnng, 5th Internatonal Symposum on Automatc Control and Computer Scence, SACCS'95, Vol., 73-78, Techncal Unversty "Gh. Asach" of Ias, Romana. [3] Enăchescu, C. (996), eural etworks as approxmaton methods. Internatonal Conference on Approxmaton and Optmsaton Methods, ICAOR'96, " Babes-Bolya Unversty ", Vol.., 83-9, Cluj-apoca. [4] Enăchescu, C., (995), Learnng the eural etworks from the Approxmaton Theory Perspectve. Intellgent Computer Communcaton ICC'95 Proceedngs, 84-87, Techncal Unversty of Cluj-apoca, Romana. [5] Enăchescu, C. (998), The Theoretcal fundamentals of eural Computng, Casa Cărţ de Ştnţă, Cluj-apoca. (n Romanan). [6] Gros, F., T. Pogo (990), etworks and the Best Approxmaton Property. Bologcal Cybernetcs, 63, [7] Haykn, S., (994), eural etworks: A Comprehensve Foundaton, Macmllan College Publshng Company, ew York, Y. [8] Moody J., Darken C., (989), Fast learnng n networks of locally tuned processng unts, eural Computaton,, [9] Poggo T., Gros F., (990), etworks for approxmaton and learnng, Proceedngs of the IEEE 78, [0] Powell M.J.D., (988), Radal bass functon approxmatons to polynomals, umercal Analyss 987 Proceedngs, VI-7-6

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network 700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School