Fuzzy Signature Neural Network Final presentation for COMP8780 IHCC Project Supervisor: Professor Tom GEDEON Presented by:
Outline Background Neural Network Fuzzy Logic, Fuzzy Rule Based System and Fuzzy Signature Fuzzy Signature Neural Network Previous work Design & Implementation Construct Fuzzy Signature Neural Network Implement testing suite Experiment Experiment 1: with no missing data Experiment 2: with 20% missing data Experiment 3: with less fuzzy Conclusion & Future work
Background Neural Network: A mathematical model that is inspired by biological neural network. Fig1. Example of a single neuron Source: Kun, H. Fuzzy Signature Neural Network Fig2. Example of a feed-forward back propagate neural network Source: Chandra, P. Fuzzy Signature Neural Networks for Rule Discovery
Background Fuzzy Logic: Represent knowledge based on degrees of membership Fig 3: Difference between crisp set and fuzzy set Source: Gedeon, T.D. 2013, Bio- inspired Compu8ng COMP8420 Lecture Notes, Research School of Computer Science, Australian Na8onal University. Fuzzy Rule Based System: Rule: If A THEN B (A, B: collections of propositions containing linguistic variables) e.g. Rule: IF x is A3 OR y is B1 THEN z is C1 Problem: Number of inputs Number of terms in the input => Rule Explosion
Background Fuzzy Signature: Structure data into vectors of fuzzy values, each of which can be a further vector A solution for rule explosion Fig4: Two structures of fuzzy signature Source: Gedeon, T.D. 2013, Bio- inspired Compu8ng COMP8420 Lecture Notes, Research School of Computer Science, Australian Na8onal University.
Background Fuzzy Signature: Aggregate: Fig5: Example of aggregation Source: Gedeon, T.D. 2013, Bio- inspired Compu8ng COMP8420 Lecture Notes, Research School of Computer Science, Australian Na8onal University. GPLAB a Gene8c Programming toolbox for MATLAB Produce fuzzy signatures based on their inner- structures and intra- rela8ons
Background Fuzzy Signature Neural Network Fig6: Example of Fuzzy Signature Neural Network
Background Previous work Similar neural network has been created by Kun HE. Semi-randomly created fuzzy signatures. Number of fuzzy signatures is determined by users. Our approach Data-driven way to create fuzzy signatures Self-determined fuzzy signatures number Improve HE s fuzzy signature neural network More automatic Reduce risks caused by manual selection of fuzzy signatures number
Design & Implementation Construct fuzzy signature neural network Fig7: Steps of constructing fuzzy signature neural network Implement testing suite Fig8: Steps of implementing testing suite
Design & Implementation Damage input Randomly remove some values Cluster input Agglomerative hierarchical clustering Advantages: Do not need users to specify number of clusters More informative Deterministic Fig9: Example of agglomerative hierarchical clustering
Design & Implementation Obtain fuzzy signatures Generate fuzzy signatures Obtain membership values Create & Train neural network Initialize weights Receive input Update weights Get membership value Fig6: Example of Fuzzy Signature Neural Network Compare with desired output Generate actual output Fig10: Steps to create and train neural network
Design & Implementation Implement testing suite Test and collect results K-fold cross validation -> split dataset into training and testing datasets Map function membership value membership value 1 1 0.5 0.5 0 1 2 3 4 5 Class 0 Extract network information 1 2 3 4 5 Fig11(a): actual output Fig11(b): desired output Fig11: Example of actual output and desired output Class
Experiment Experiment 1: with no missing data Table 1: Results of our approach with no missing values and five fuzzy signatures Table 2: Results of HE s approach with no missing values and five fuzzy signatures
Experiment Experiment 2: with 20% missing data Decreased percentage 40 35 30 25 20 15 10 5 0-5 - 10 cancer diabetes high salary medium salary low salary This project 34.88170445 4.92186359-1.840490798-5.421686747 6.650860993 Kun's approach 0.308510638 5.324141977-1.923076923 12.42937853 19.34968791 Fig 12: Decreased percentage of testing accuracy for HE s and our approach Cancer dataset: missing one attribute
Experiment Experiment 3: with fewer fuzzy Accuracy Accuracy 100 100 95 95 90 90 85 85 80 80 75 75 70 70 65 65 60 60 55 55 50 6 fuzzy 5 fuzzy 4 fuzzy 3 fuzzy 2 fuzzy 50 6 fuzzy 5 fuzzy 4 fuzzy 3 fuzzy 2 fuzzy Fig 13(a): Testing accuracy for KUN s approach as fuzzy neuron numbers decrease Fig 13(a): Testing accuracy for our approach as fuzzy neuron numbers decrease Fig 13: Testing accuracy for KUN s and our approach as fuzzy neuron numbers decrease
Conclusion & Future work Conclusion This approach achieves stable and robust good results in extreme situations With missing values With fewer fuzzy signatures Data-oriented VS semi-random Future work Find a more consistent and less timeconsuming fuzzy signature generation method. Implement an another mapping function.