Internet Engineering. Jacek Mazurkiewicz, PhD Softcomputing. Part 1: Introduction, Elementary ANNs

Transcription

1 Inerne Engineering Jacek azurkieicz, PhD Sofcompuing Par : Inroducion, Elemenary As

2 Formal Inroducion conac hours, room o. 5 building C-3: onday: :45-5:5, Friday: 4:30-6:00, slides:.zsk.ic.pr.roc.pl Professor Wikor Zin es: during lecure - sofcompuing: - lecure + laboraory - laboraory mark 0% of final mark - bonus quesion!

3 Program Idea of inelligen processing Fuzzy ses and approimae reasoning Eper sysems - knoledge base organizaion Eper sysems - reasoning rules creaion Eper sysems: ypical organizaion and applicaions Arificial neural neorks: learning and rerieving algorihms ulilayer percepron ohonen neural neork Hopfield neural neork Hamming neural neork Arificial neural neorks: applicaions Geneic algorihms: descripion and classificaion Geneic algorihms: basic mechanisms and soluions

4 SUBJEC OBJECIVES C. noledge of arificial neural neorks in paern recogniion, digial signals and daa processing: opology of neorks, influence of parameers for neork behavior. C. noledge of geneic algorihms used for daa pre- and posprocessing. C3. noledge of eper sysems reasoning rules and knoledge base creaion for differen asks. C4. Skills of special environmen usage for projec phase, modeling and simulaion of sofcompuing sysems in case of differen scienific problems. SUBJEC EDUCAIOAL EFFECS relaing o knoledge: PE_W0 knos he rules and he idea of inelligen processing. PE_W0 defines he fuzzy ses and undersands he idea of approimae reasoning. PE_W03 defines he knoledge base and reasoning rules, knos he eper sysems consrucion. PE_W04 knos he archiecure of ypical arificial neural neorks srucures, learning and rerieving algorihms, applicaions. PE_W05 knos he descripion, classificaion, eamples of applicaions of geneic algorihms relaing o skills: PE_U0 can use he environmens for projec phase, modeling and simulaion of arificial neural neorks as ell as geneic algorihms in differen asks abou paern digial signals recogniion. PE_U0 can use he environmens for projec phase, modeling and implemenaion of eper sysems o dedicaed fields of knoledge. PE_U03 can use he environmens for projec phase, modeling and implemenaion of fuzzy ses and fuzzy reasoning o dedicaed fields of knoledge.

5 Lieraure B. Bouchon eunier, Fuzzy Logic and Sof Compuing O. Casilo, A. Bonarini, Sof Compuing Applicaions. Caudill, Ch. Buler, Undersanding eural eorks E. Damiani, Sof Compuing in Sofare Engineering R. Hech-ielsen, eurocompuing S. Y. ung, Digial eural eorks D.. Praihar, Sof Compuing S.. Sivanandam, S.. Deepa, Principles of Sof Compuing A.. Srivasava, Sof Compuing D. A. Waerman, A Guide o Eper Sysems D. Zhang, Parallel VLSI eural Sysem Design

6 Why eural eorks and Company? Sill in acive use o chance o solve some problems in oher ay Human abiliy vs. classical programs Works as primiive human s brain Arificial inelligence has poer! A + Fuzzy Logic + Eper Sysems + Rough Ses + An Algorihms = SofCompuing

7 he Sory 943 cculloch & Pis model of arificial neuron 949 Hebb informaion sored by biological neural nes 958 Rosenbla percepron model 960 Widro & Hoff firs neurocompuer - adaline 969 insky & Paper XOR problem single-layer percepron limiaions 986 ccleland & Rumelhar backpropagaion algorihm

8 Where Sofcompuing is in Use? Leers, signs, characers, digis recogniion Recogniion of ship ypes daa from sonar Elecric poer predicion Differen kinds of simulaors and compuer games Engine diagnosic in planes, vehicles Rock-ype idenificaion Bomb searching devices

9 eural eorks Realisaion Se of conneced idenical neurons Arificial neuron based on a biological neuron Hardare realisaion digial device Sofare realisaion simulaors Arificial neural neork idea, algorihm, mahemaical formulas Works in parallel o programming learning process necessary

10 Learning auczyciel eacher Wih a eacher Wekor Learning cech (dane vecor nauki) Wynik Resul of klasyfikacji learning Parameers lasyfikaor Weighs Wihou a eacher Wekor Learning cech (dane vecor esoe) Parameers lasyfikaor Weighs Wynik Resul of klasyfikacji learning

11 Sofcompuing vs. Classical Compuer Differen limiaions of sofcompuing mehods o sofcompuing: operaions based on symbols: ediors, algebraic equaions calculaions ih a high level of precision Sofcompuing is very nice, bu no as universal as compuer

12 brain brain sern cerebellum Anaomy Foundaions () prolonged cord ervous Sysem -ays, symmerical se of srucures, divided ino 4 pars: Spinal Cord receiving and ransmission of daa spinal cord Prolonged Cord breahing, blood sysem, digesion Cerebellum movemen conrol nervous sysem Brain (ca..3 kg) hemispheres feeling, hinking, movemen

13 Anaomy Foundaions ()

14 Anaomy Foundaions (3) Cerebral core hickness: mm, area: ca..5 m Cerebral core divided ino 4 par lobes Each lobe is corrugaed Each hemisphere is responsible for half par of body: righ for lef par, lef for righ par Hemispheres are idenical in case of a srucure, bu heir funcions are differen

15 Anaomy Foundaions (4) Brain composed by fibres ih large number of branches o ypes of cells in nervous issue: neurons and gley cells here are more gley cells: no daa ransfer among neurons caering funcions Ca. 0 milliard neurons in cerebral core Ca. 00 milliard neurons in hole brain euron: dendries inpus, aon oupu, body of neuron euron: housands of synapses connecions o oher neurons

16 Anaomy Foundaions (5) eurons in ork: chemical-elecrical signal ransferring cell generaes elecrical signals elecric pulse is changed ino a chemical signal a he end of aon chemical info passed by neuroransmiers 50 differen ypes of neurons neurons driven by a frequency of hundreds of Hz neurons are raher lo devices!

17 Anaomy Foundaions (6)

18 Biological and Arificial eural es Arificial neural neorks are a good soluion for: esing already idenified biological sysems paern recogniion alernaive configuraions o find he basic feaures of hem Arificial neural neorks are primiive brohers of biological nes Biological nes have sophisicaed inernal feaures imporan for heir normal ork Biological nes have sophisicaed ime dependences ignored in mos arificial neorks Biological connecions among neurons are differen and complicaed os archiecures of arificial nes are unrealisic from he biology poin of vie os learning rules for arificial neorks are unreal in biology poin of vie os biological nes e can compare o already learned arificial nes o realise funcion described in a very deailed ay

19 Linear A - ADALIE (ADAive Linear euron) 0 single neuron s anser: y j j j 0 y ~ ) ( j j ~ j0 scalar descripion vecor descripion... + y ~ col (,,..., 0 muli-oupu ne: ) col ( 0,,..., ) 0 number of inpu neurons number of oupu neurons

20 Single-Layer uli-oupu eork y y y k-neuron s anser: y () j0 kj j Inpu neuron W kj Oupu neuron y() column y(x) WX W 0 0 0

21 Learning Procedure eperimenal daa: - series,...,,,...,, learning daa required ansers, funcion implemened by ne error funcion mean-square error: n k n y k k W E ) ( n k j n k n j jk W E 0 ) ( looking for a minimum of E(W) funcion: 0 ) (, kj j k W E

22 Pseudoinverse Algorihm n n j j n k n j kj kj E W 0 ' ' 0 ) ( n j n n j n k n j n j kj j k 0 ' ', here: X W finally: X X W X XW X X) (X W pseudoinverse τ, X W τ