Introduction to Fuzzy Logic Control

Transcription

1 Introduction to Fuzzy Logic Control 1

2 Outline General Definition Applications Operations Rules Fuzzy Logic Toolbox FIS Editor Tipping Problem: Fuzzy Approach Defining Inputs & Outputs Defining MFs Defining Fuzzy Rules 2

3 General Definition Fuzzy Logic Lotfi Zadeh, Berkely superset of conventional (Boolean) logic that has been extended to handle the concept of partial truth central notion of fuzzy systems is that truth values (in fuzzy logic) or membership values (in fuzzy sets) are indicated by a value on the range [0.0, 1.0], with 0.0 representing absolute Falseness and 1.0 representing absolute Truth. deals with real world vagueness

4 Applications Expert Systems Control Units Bullet train between Tokyo and Osaka Video Cameras Automatic Transmissions

5 Operations A B A B A B A

6 Controller Structure Fuzzification Scales and maps input variables to fuzzy sets Inference Mechanism Approximate reasoning Deduces the control action Defuzzification Convert fuzzy output values to control signals

7 MATLAB fuzzy logic toolbox MATLAB fuzzy logic toolbox facilitates the development of fuzzy-logic systems using: graphical user interface (GUI) tools command line functionality The tool can be used for building Fuzzy Expert Systems Adaptive Neuro-Fuzzy Inference Systems (ANFIS) 7

8 Graphical User Interface (GUI) Tools There are five primary GUI tools for building, editing, and observing fuzzy inference systems in the Fuzzy Logic Toolbox: Fuzzy Inference System (FIS) Editor Membership Function Editor Rule Editor Rule Viewer Surface Viewer 8

9 MATLAB: Fuzzy Logic Toolbox 9

10 MATLAB: Fuzzy Logic Toolbox 10

11 Fuzzy Inference system Two type of inference system Mamdni inference method Sugeno inference method *Mamdani's fuzzy inference method, the most common methodology 11

12 FIS Editor: Mamdani s inference system 12

13 Fuzzy Logic Examples using Matlab To control the speed of a motor by changing the input voltage When a set point is defined, if for some reason, the motor runs faster, we need to slow it down by reducing the input voltage. If the motor slows below the set point, the input voltage must be increased so that the motor speed reaches the set point. 13

14 Input/Output Input status words be: Too slow Just right Too fast output action words be: Less voltage (Slow down) No change More voltage (Speed up) 14

15 FIS Editor: Adding Input / Output 15

16 FIS Editor: Adding Input / Output 16

17 Membership Function Editor 17

18 Input Membership Function 18

19 Output Membership Function 19

20 Membership Functions 20

21 Rules Define the rule-base: 1) If the motor is running too slow, then more voltage. 2) If motor speed is about right, then no change. 3) If motor speed is to fast, then less voltage. 21

22 Member function Editor: Adding Rules 22

23 Rule Base 23

24 Rule Viewer 24

25 Surface Viewer 25

26 Save the file as one.fis. Now type in the commend window to get the result: >>fis = readfis('one'); out=evalfis(2437.4,fis) >>out =

27 Sugeno-Type Fuzzy Inference Takagi-Sugeno-Kang, method of fuzzy inference similar to the Mamdani method in many respects Fuzzifying the inputs and applying the fuzzy operator, are exactly the same. The main difference between Mamdani and Sugeno is that the Sugeno output membership functions are either linear or constant. 27

28 FIS Editor: Sugeno inference system 28

29 Add Input/output variables 29

30 Define Input/output variables 30

31 Add Input MF 31

32 Define Input MF 32

33 Add output MF 33

34 Define output MF 34

35 Add rules 35

36 Define Rule Base 36

37 View rules 37

38 Rules viewer 38

39 Surface viewer 39

40 Advantages of the Sugeno Method Sugeno is a more compact and computationally efficient representation than a Mamdani system. It is computationally efficient. It works well with linear techniques (e.g., PID control). It works well with optimization and adaptive techniques. It has guaranteed continuity of the output surface. It is well suited to mathematical analysis. 40

41 Advantages of the Mamdani Method It is intuitive. It has widespread acceptance. It is well suited to human input. 41

42 Support Vector Machine & Its Applications

43 Overview Introduction to Support Vector Machines (SVM) Properties of SVM Applications Gene Expression Data Classification Text Categorization if time permits Discussion

44 Support Vector Machine(SVM) The fundamental principle of classification using the SVM is to separate the two categories of patterns Map data x into a higher dimensional feature space via a nonlinear mapping. The linear classification (regression) in the high dimensional space is equivalent to the nonlinear classification (regression) in the low dimensional space

45 Linear Classifiers w x+ b>0 denotes +1 denotes -1 x α f y est f(x,w,b) = sign(w x + b) w x+ b<0 How would you classify this data?

46 denotes +1 denotes -1 Linear Classifiers x α f f(x,w,b) = sign(w x +b) y est How would you classify this data?

47 Linear Classifiers denotes +1 denotes -1 x α f f(x,w,b) = sign(w x + b) y est How would you classify this data?

48 Linear Classifiers denotes +1 denotes -1 x α f y est f(x,w,b) = sign(w x + b) Any of these would be fine....but which is best?

49 Linear Classifiers α denotes +1 denotes -1 x f f(x,w,b) = sign(w x +b) y est How would you classify this data? Misclassified to +1 class

50 Classifier Margin α denotes +1 denotes -1 x f f(x,w,b) = sign(w x +b) y est Define the margin of a linear classifier as the width that the boundary could be increased by before hitting a datapoint.

51 Maximum Margin denotes +1 denotes -1 Support Vectors are those datapoints that the margin pushes up against 1. Maximizing the margin is good according f(x,w,b) to intuition = sign(w and PAC x theory + b) 2. Implies that only support vectors are important; other The training maximum examples are ignorable. Linear SVM margin linear classifier is the linear classifier with the, um, maximum margin. 3. Empirically it works very very well. This is the simplest kind of SVM (Called an LSVM)

52 Linear SVM Mathematically x + M=Margin Width X - What we know: w. x + + b = +1 w. x - + b = -1 M = ( x + x w ) w = 2 w w. (x + -x -) = 2

53 Linear SVM Mathematically Goal: 1) Correctly classify all training data 2) Maximize the Margin same as minimize if y i = +1 if y i = -1 We can formulate a Quadratic Optimization Problem and solve for w and b Minimize subject to y wx i + b 1 wx i + b 1 i Φ( w) = y ( wx + b) 1 i i 1 2 t w w ( wx + b) 1 i for all i M = 2 w w t w 1 2 i

54 Solving the Optimization Problem Find w and b such that Φ(w) =½ w T w is minimized; and for all {(x i,y i )}: y i (w T x i + b) 1 Need to optimize a quadratic function subject to linear constraints. Quadratic optimization problems are a well known class of mathematical programming problems, and many (rather intricate) algorithms exist for solving them. The solution involves constructing a dual problem where a Lagrange multiplier α i is associated with every constraint in the primary problem: Find α 1 α N such that Q(α) =Σα i - ½ΣΣα i α j y i y j x it x j is maximized and (1) Σα i y i = 0 (2) α i 0 for all α i

55 The Optimization Problem Solution The solution has the form: w =Σα i y i x i b= y k - w T x k for any x k such that α k 0 Each non-zero α i indicates that corresponding x i is a support vector. Then the classifying function will have the form: f(x) = Σα i y i x it x + b Notice that it relies on an inner product between the test point x and the support vectors x i we will return to this later. Also keep in mind that solving the optimization problem involved computing the inner products x it x j between all pairs of training points.

56 Dataset with noise denotes +1 denotes -1 Hard Margin: So far we require all data points be classified correctly - No training error What if the training set is noisy? - Solution 1: use very powerful kernels OVERFITTING!

57 Soft Margin Classification Slack variables ξi can be added to allow misclassification of difficult or noisy examples. ε 2 ε 11 ε 7 What should our quadratic optimization criterion be? Minimize 1 w. w + C 2 R ε k k= 1

58 Hard Margin v.s. Soft Margin The old formulation: Find w and b such that Φ(w) =½ w T w is minimized and for all {(x i,y i )} y i (w T x i + b) 1 The new formulation incorporating slack variables: Find w and b such that Φ(w) =½ w T w + CΣξ i is minimized and for all {(x i,y i )} y i (w T x i + b) 1- ξ i and ξ i 0 for all i Parameter C can be viewed as a way to control overfitting.

59 Linear SVMs:Overview The classifier is a separating hyperplane. Most important training points are support vectors; they define the hyperplane. Quadratic optimization algorithms can identify which training points x i are support vectors with non-zero Lagrangian multipliers α i. Both in the dual formulation of the problem and in the solution training points appear only inside dot products: Find α 1 α N such that Q(α) =Σα i -½ΣΣα i α j y i y j x it x j is maximized and (1) Σα i y i = 0 (2) 0 α i C for all α i f(x) = Σα i y i x it x + b

60 Non-linear SVMs Datasets that are linearly separable with some noise work out great: But what are we going to do if the dataset is just too hard? 0 x How about mapping data to a higher-dimensional space: x 2 0 x 0 x

61 Non-linear SVMs: Feature spaces General idea: the original input space can always be mapped to some higher-dimensional feature space where the training set is separable: Φ: x φ(x)

62 The Kernel Trick The linear classifier relies on dot product between vectors K(x i,x j )=x it x j If every data point is mapped into high-dimensional space via some transformation Φ: x φ(x), the dot product becomes: K(x i,x j )= φ(x i ) T φ(x j ) A kernel function is some function that corresponds to an inner product in some expanded feature space. Example: 2-dimensional vectors x=[x 1 x 2 ]; let K(x i,x j )=(1 + x it x j ) 2, Need to show that K(x i,x j )= φ(x i ) T φ(x j ): K(x i,x j )=(1 + x it x j ) 2, = 1+ x i12 x 2 j1 + 2 x i1 x j1 x i2 x j2 + x i22 x 2 j2 + 2x i1 x j1 + 2x i2 x j2 = [1 x 2 i1 2 x i1 x i2 x 2 i2 2x i1 2x i2 ] T [1 x 2 j1 2 x j1 x j2 x 2 j2 2x j1 2x j2 ] = φ(x i ) T φ(x j ), where φ(x) = [1 x x 1 x 2 x 2 2 2x 1 2x 2 ]

63 What Functions are Kernels? For some functions K(x i,x j ) checking that K(x i,x j )= φ(x i ) T φ(x j ) can be cumbersome. Mercer s theorem: Every semi-positive definite symmetric function is a kernel Semi-positive definite symmetric functions correspond to a semi-positive definite symmetric Gram matrix: K= K(x 1,x 1 ) K(x 1,x 2 ) K(x 1,x 3 ) K(x 1,x N ) K(x 2,x 1 ) K(x 2,x 2 ) K(x 2,x 3 ) K(x 2,x N ) K(x N,x 1 ) K(x N,x 2 ) K(x N,x 3 ) K(x N,x N )

64 Examples of Kernel Functions Linear: K(x i,x j )= x i T x j Polynomial of power p: K(x i,x j )= (1+ x i T x j ) p Gaussian (radial-basis function network): xi x K( xi, x j) = exp( 2 2σ j 2 ) Sigmoid: K(x i,x j )= tanh(β 0 x i T x j + β 1 )

65 Non-linear SVMs Mathematically Dual problem formulation: Find α 1 α N such that Q(α) =Σα i -½ΣΣα i α j y i y j K(x i,x j ) is maximized and (1) Σα i y i = 0 (2) α i 0 for all α i The solution is: f(x) = Σα i y i K(x i,x j )+ b Optimization techniques for finding α i s remain the same!

66 Nonlinear SVM - Overview SVM locates a separating hyper plane in the feature space and classify points in that space It does not need to represent the space explicitly, simply by defining a kernel function The kernel function plays the role of the dot product in the feature space.

67 Properties of SVM Flexibility in choosing a similarity function Sparseness of solution when dealing with large data sets - only support vectors are used to specify the separating hyper plane Ability to handle large feature spaces - complexity does not depend on the dimensionality of the feature space Over fitting can be controlled by soft margin approach Nice math property: a simple convex optimization problem which is guaranteed to converge to a single global solution Feature Selection

68 SVM Applications SVM has been used successfully in many real-world problems - text (and hypertext) categorization - image classification - bioinformatics (Protein classification, Cancer classification) - hand-written character recognition

69 Weakness of SVM It is sensitive to noise - A relatively small number of mislabeled examples can dramatically decrease the performance It only considers two classes - how to do multi-class classification with SVM? -Answer: 1) with output arity m, learn m SVM s SVM 1 learns Output==1 vs Output!= 1 SVM 2 learns Output==2 vs Output!= 2 : SVM m learns Output==m vs Output!= m 2)To predict the output for a new input, just predict with each SVM and find out which one puts the prediction the furthest into the positive region.

70 Some Issues Choice of kernel Gaussian or polynomial kernel is default if ineffective, more elaborate kernels are needed domain experts can give assistance in formulating appropriate similarity measures Choice of kernel parameters σ in Gaussian kernel σ is the distance between closest points with different classifications Optimization criterion Hard margin vs. Soft margin A lengthy series of experiments in which various parameters are tested

71 Wind Power Forecasting(WPF) WPF is a technique which provides the information of how much wind power can be expected at a given point of time. Due to the increasing penetration of wind power into the electric power grid. A good short term forecasting will ensure grid stability and a favorable trading performance on the electricity markets.

72 Ɛ-SVM The objective function of the ε SVM is based on a ε insensitive loss function. The formula for the ε SVM is given as follows:

73 Structure of SVM

74 Data Resolution The resolution of the dataset is 10 minutes. Each data represents the average wind speed and power within one hour. The data values between xjtwo adjacent samples are linearly changed, that is: ) x j ( t) = x i + x i+ 1 dt + x i i. t 0 t dt Where is the time interval between x i and x i+ 1. dt i i

75 Data Value The average value of the data within T s can be calculated as ) x j ( t) t + T i s 1 ) = x j ( t) dt T s t i where T s = 60 minutes is used in the very short term forecasting (less than 6 hours) and T s = 2 hours is used for short term forecasting.

76 Fixed Step Prediction Scheme Prediction horizon of h steps fixed step forecasting means only the value of the next h sample is predicted by using the historical th data. ŷ(t + h) = f (yt, yt-1,,yt-d) Where f is a nonlinear function generated by SVM y + t h is predicted with the data before (the red blocks), is y t+h 1 predicted with the data before (the green blocks) y t y t 1

77 Wind speed normalization

78 Autocorrelations of the wind speed samples

79 SVM model and the RBF model

80 1h-ahead wind power prediction using the SVM model.

81 CONCLUSIONS The SVM has been successfully applied to the problems of pattern classification, particularly the classification of two different categories of patterns. SVM model is more suitable for very short term and short term WPF Provides a powerful tool for enhancing the WPF accuracy.