A Multiobjective Genetic Fuzzy System for Obtaining Compact and Accurate Fuzzy Classifiers with Transparent Fuzzy Partitions

Size: px

Start display at page:

Download "A Multiobjective Genetic Fuzzy System for Obtaining Compact and Accurate Fuzzy Classifiers with Transparent Fuzzy Partitions"

Eric Glenn
8 years ago
Views:

1 A Multiobjective Genetic Fuzzy System for Obtaining Compact and Accurate Fuzzy Classifiers with Transparent Fuzzy Partitions Pietari Pulkkinen Tampere University of Technology Department of Automation Science and Engineering Finland Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... /5

of Technology Department of Automation Science and Engineering Finland Pietari

2 Contents Fuzzy classifiers (FCs): What are they and what are their benefits? An example application of FCs as a reasoning mechanism in a bioaerosol detector Interpretability accuracy trade-off Components of the proposed multiobjective genetic fuzzy system (GFS) Results Conclusions Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 2/5

accuracy trade-off Components of the proposed multiobjective genetic fuzzy system (GFS) Results

3 Fuzzy Classifiers (FCs) Classification is based on if-then fuzzy rules. An example rule: If temperature is high and humidity is high, then climate is tropical Intuitive way of reasoning Before applying an FC in practice, it is possible to verify that: the FC is accurate enough 2 that the fuzzy rules are reasonable Interpretability Complex FCs with large number of rules are hard to interpret No reasonable linguistic labels for highly overlapping fuzzy sets Compact rule bases and transparent fuzzy partitions are preferred! Transparent fuzzy partitions x x x 3 Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 3/5

FC is accurate enough 2 that the fuzzy rules are reasonable Interpretability Complex FCs with large number of rules are hard to interpret No reasonable linguistic labels for highly

4 An FC as a Reasoning Mechanism in a Bioaerosol Detector Bioaerosol detector was developed by: Dekati, Environics, and TUT / Department of Physics Reasoning mechanism was developed by: TUT / Department of Automation Science and Engineering Pulkkinen, P., Hytönen, J. and Koivisto, H.: Developing a bioaerosol detector using hybrid genetic fuzzy systems. Engineering Applications of Artificial Intelligence, vol. 2, no 8, pp , December 28 Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 4/5

and Koivisto, H.: Developing a bioaerosol detector using hybrid genetic fuzzy systems.

5 Interpretability Accuracy Trade-off The purpose is to minimize the number of misclassifications and to minimize the complexity of FCs These are conflicting objectives! Improving one objective, deteriorates the other. Search for Pareto-optimal FCs Top: training set, bottom: testing set Error rate on train set Error rate on test set Number of fuzzy rules Number of fuzzy rules Error rate on train set Error rate on test set Total rule length Total rule length Training set: Complex FCs are the most accurate Testing set: Some simpler FCs seem to be very accurate in this example P. Pulkkinen and H. Koivisto, Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms, Int. J. Approx. Reason., vol. 48, no. 2, pp , June 28. Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 5/5

6 Searching for FCs Involves a Large Search Space A simple FC with 3 rules can be presented as: functions Jäsenyysaste Small Large 5 5 x Small Large 5 5 y Denote small with and large with 2 Rule : If x is and y is then class is 3 Rule 2 : If x is and y is 2 then class is 2 Rule 3 : If x is 2 then class is The antecedents of rules: A = (,,, 2, 2, ). }{{} }{{} }{{} Rule Rule 2 Rule 3 4 gbell membership functions: P = (P,, P,2, P,3, P,4, P 2,, P 2,2, P 2,3, P 2,4, } {{ } } {{ } Parameter a Parameter b P 3,, P 3,2, P 3,3, P 3,4 ). } {{ } Parameter c Rule consequent (i.e. class number): S = ( }{{} 3, }{{} 2, }{{} Rule Rule 2 Rule 3 ) Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 6/5

}{{} }{{} }{{} Rule Rule 2 Rule 3 4 gbell membership functions: P = (P,, P,2, P,3, P,4, P 2,, P 2,2, P 2,3, P 2,4, } {{ } } {{ } Parameter a Parameter b P 3,, P 3,2, P 3,3, P 3,4 ).

7 Multiobjective Genetic Fuzzy System Initial population of FCs is further optimized by NSGA-II developed by Kalyanmoy Deb et al. Purpose: to minimize the number of misclassifications and to minimize the number of rule conditions MF parameters are adjusted and rules are learnt Granularity, i.e., the number of fuzzy sets in each partition is also learnt Dynamic constraints keep the fuzzy partitions always transparent. No need to minimize any transparency index More efficient search Result: A Pareto optimal set of compact and accurate FCs All of them have transparent fuzzy partitions Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 7/5

No need to minimize any transparency index More efficient search Result: A Pareto optimal set of compact and accurate FCs All of them have transparent fuzzy partitions

8 Transparency Conditions MFs tuning usually improves the accuracy, but may deteriorate the transparency of fuzzy partitions α-condition: At any intersection point of two MFs, the membership value is at most α. 2 γ-condition: At the center of each MF, no other MF receives membership value larger than γ. 3 β-condition: At each point of universe of discourse at least one MF has membership value at least β. β =.5, γ =.25 and α =.8 are used α γ β Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 8/5

2 γ-condition: At the center of each MF, no other MF receives membership value larger than γ.

9 Dynamic Tuning of Functions Dynamically constrained 3-parameter MFs tuning strategy is used: Start from a transparent fuzzy partition and modify one of the gbell MF parameter a, b, or c. µ(x; a, b, c) = + x c a Only one parameter is modified at a time 2b If number of MFs is altered, a simple partition algorithm is used to create a new transparent partition Every partition in each FC is always transparent! More details available in: P. Pulkkinen and H. Koivisto. A dynamically constrained multiobjective genetic fuzzy system for regression problems. IEEE Transactions on Fuzzy Systems (accepted) Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 9/5

µ(x; a, b, c) = + x c a Only one parameter is modified at a time 2b If number of MFs is altered, a simple partition algorithm is used to create a new transparent partition

10 Two Simple Partition Algorithms Algorithms are used to: provide a transparent starting point for MFs tuning to find good partitions during further optimization Partitions are always transparent.9 α γ.3.2. β An evenly distributed uniformly shaped partition.9.8 α γ.2. β (a) An unevenly distributed non-uniformly shaped partition Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... /5

11 Dynamic Tuning of Functions: an Example Modifying MF 2 (a) Original partition (b) decrease its width (c) alter its shape (d) move it towards right The original and modified partitions Degree of membership Degree of membership α.8 α γ.3 γ β β (a) Original partition (b) Parameter a of MF 2 set to its minimum value Degree of membership α.8 α γ.3 γ β β (c) Parameter b of MF 2 set to its minimum value (d) Parameter c of MF 2 set to its maximum value Degree of membership Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... /5

6.4.4.3 γ.3 γ.2.2.. β β.2.4.6.8.2.4.6.8 (a) Original partition (b) Parameter a of MF 2 set to its minimum value Degree of membership.9 2 3 4 5.9 2 3 4 5.8 α.8 α.7.7.6.6.4.4.3 γ.3 γ.2.2.. β β.2.4.6.8.2.4.6.8 (c) Parameter b of MF 2 set to its minimum value (d) Parameter c of MF 2 set to its maximum value Degree of membership Pietari Pulkkinen: Tampere Univ.

12 Experiments Two well known classification problem Wine and Glass were studied. -fold cross-validation was repeated times for both problems. (altogether 2 runs) Wine is a problem with three different classes and 3 input variables Glass is a problem with six different classes and 9 input variables Results compared to our former approach 2 : It also utilizes NSGA-II It does not apply dynamic constraints and partitions are not always transparent Expected to have better accuracy than the proposed method due to trade-off between accuracy and transparency of fuzzy partitions. 2 P. Pulkkinen and H. Koivisto, Fuzzy classifier identification using decision tree and multiobjective evolutionary algorithms, Int. J. Approx. Reason., vol. 48, no. 2, pp , June 28. Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 2/5

: It also utilizes NSGA-II It does not apply dynamic constraints and partitions are not always transparent Expected to have better accuracy than the proposed method due to trade-off between accuracy

13 Results According to T-test, no statistical difference in test accuracy in both of the problems!.4.4 Method [] (train) Method [] (test) This paper (test) This paper (train).3.3 Error rate.2 Error rate Rules Rule conditions Glass problem: Comparison of the averaged Pareto fronts: It was expected that the former approach should be more accurate Surprisingly, especially test accuracy is almost the same for both approaches Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 3/5

14 Glass problem: comparison of the fuzzy partitions An example FC by the former approach Some partitions are not transparent An example FC by the proposed approach All partitions are transparent x x 2 x x x x x x x x x x x x 9 Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 4/5

75.25 2 3 4 x 3.75.25 x 2 2 3 x 4.75.25 2 3 4 x 3.75.25 2 3 x 4.75.25 7 72 74 x 5.75.25 2 3 x 8.75.25 2 4 6 x 6.75.25..2.3 x 9.75.25 7 72 74 x 5.75.25.2.4 x 9 Pietari Pulkkinen: Tampere Univ.

15 Conclusions A multiobjective genetic fuzzy system which searches for compact and accurate FCs with transparent fuzzy partitions was developed. Its strengths are: Number of input variables is reduced already in the initialization phase The number of fuzzy sets is learnt and MFs are tuned and resulting partitions are always transparent Accuracy and compactness was comparable to our former approach even though that approach does not always lead to transparent fuzzy partitions The proposed approach is not limited only to classification problems. Regression problem can be handled with some modifications 3. 3 P. Pulkkinen and H. Koivisto. A dynamically constrained multiobjective genetic fuzzy system for regression problems. IEEE Transactions on Fuzzy Systems (accepted) Pietari Pulkkinen: Tampere Univ. of Tech., Finland ICMLA 29: A Multiobjective Genetic Fuzzy System... 5/5

Accuracy and compactness was comparable to our former approach even though that approach does not always lead to transparent fuzzy partitions The proposed approach is not limited only to

Predictive Knowledge Discovery by Multiobjective Genetic Fuzzy Systems for Estimating Consumer Behavior Models

Predictive Knowledge Discovery by Multiobjective Genetic Fuzzy Systems for Estimating Consumer Behavior Models Jorge Casillas, Oscar Delgado Dept. Computer Science and Artif. Intell. Univ. Granada, 18071