University, Samsun, Turkey b Statistics Department, Faculty of Science, Ankara University,

Transcription

1 This article was downloaded by: [Marmara Universitesi] On: 01 April 2015, At: 02:06 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK Click for updates Communications in Statistics - Theory and Methods Publication details, including instructions for authors and subscription information: A New Robust Regression Method Based on Particle Swarm Optimization Ozge Cagcag a, Ufuk Yolcu b & Erol Egrioglu c a Statistics Department, Faculty of Arts and Science, Ondokuz Mayis University, Samsun, Turkey b Statistics Department, Faculty of Science, Ankara University, Ankara, Turkey c Statistics Department, Faculty of Arts and Science, Marmara University, Istanbul, Turkey Accepted author version posted online: 29 May To cite this article: Ozge Cagcag, Ufuk Yolcu & Erol Egrioglu (2015) A New Robust Regression Method Based on Particle Swarm Optimization, Communications in Statistics - Theory and Methods, 44:6, , DOI: / To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content ) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &

2 Conditions of access and use can be found at

3 Communications in Statistics Theory and Methods, 44: , 2015 Copyright Taylor & Francis Group, LLC ISSN: print / X online DOI: / A New Robust Regression Method Based on Particle Swarm Optimization OZGE CAGCAG, 1 UFUK YOLCU, 2 AND EROL EGRIOGLU 3 1 Statistics Department, Faculty of Arts and Science, Ondokuz Mayis University, Samsun, Turkey 2 Statistics Department, Faculty of Science, Ankara University, Ankara, Turkey 3 Statistics Department, Faculty of Arts and Science, Marmara University, Istanbul, Turkey Regression analysis is one of methods widely used in prediction problems. Although there are many methods used for parameter estimation in regression analysis, ordinary least squares (OLS) technique is the most commonly used one among them. However, this technique is highly sensitive to outlier observation. Therefore, in literature, robust techniques are suggested when data set includes outlier observation. Besides, in prediction a problem, using the techniques that reduce the effectiveness of outlier and using the median as a target function rather than an error mean will be more successful in modeling these kinds of data. In this study, a new parameter estimation method using the median of absolute rate obtained by division of the difference between observation values and predicted values by the observation value and based on particle swarm optimization was proposed. The performance of the proposed method was evaluated with a simulation study by comparing it with OLS and some other robust methods in the literature. Keywords Linear model; Particle swarm optimization; Robust regression estimator; Simulation. Mathematics Subject Classification 1. Introduction Due to its easy calculation, ordinary least squares (OLS) technique is the most common method used in regression analysis which is used for eliciting unknown values of dependent variable and modeling its behavior. OLS method which is used to estimate the unknown parameters of regression model focuses on minimizing difference between predicted and observation values. This makes OLS technique very sensitive to the outliers in data set and may cause deviations in the model. Received March 6, 2012; Accepted July 30, Address correspondence to Ufuk Yolcu, Statistics Department, Faculty of Arts and Science, Ankara University, 06100, Ankara, Turkey; uyolcu@ankara.edu.tr 1270

4 New Robust Regression Method Based on Particle Swarm Optimization 1271 In cases where data set has outliers, the detection of existing outlier observation and exclusion of the observation from the data set can be considered as a way of analyzing the model properly. However, as each observation in data set may affect the model significantly, exclusion of them from the data set will affect the result substantially and thus will change the model. Therefore, each observation in the data set should be used to reach the appropriate solution. For this purpose, in cases where data set has outliers, robust methods which are less sensitive than OLS technique are used. In literature, several studies have been carried out for this purpose. Techniques, known as M estimators, aim to minimize the function of residuals rather than the sum of the squares of the residuals. Hogg (1979), Huber (1981), Huynh (1982), Hampel et al. (1986), Dave and Krishnapuram (1997) proposed robust methods using M estimators. Besides, Rousseuw and Leroy (1997) and Candan (1995) proposed robust methods which make residual squares as median least squares. Akbilgic and Akinci (2009) proposed a new approach based on least squares ratio (LSR). Sanli (2005) proposed an approach that extends multiple regression analysis to simple linear regression analysis and tried to reduce the weights of outliers in model forecasting. In this study, a new parameter estimation method using median of absolute rate obtained by division of difference between observation values and predicted values by the observation value and based on particle swarm optimization (PSO) was proposed. The performance of the proposed method was evaluated with a simulation study by comparing it with OLS and some other robust methods in the literature. In the second section of the study, simple linear regression model and OLS technique is introduced. Third section deals with the proposed method. In the fourth section, implementation of the proposed method is introduced with a simulation study, and obtained results are discussed. 2. Multiple Linear Regression Model and OLS A linear regression model can be shown as expressed in (1). y i = β 0 + p x ji β j + e i i = 1, 2,...,n j = 1, 2,...,p (1) j=1 Here, y i represents dependent variable for the i th observation, x i1,x i2,,x ip represents the number independent variable value that equalsp, e i represents error, and β 0,β 1,β 2,,β p represents regression coefficients. The expression of this model with matrix notation can be expressed as: Y = Xβ + ε (2) where Y is an n 1 dimensional vector of dependent variable observation, X is an n (p+1) dimensional matrix of dependent variable observation, β; (p + 1) 1 dimensional vector of regression coefficient and ε is an n 1 dimensional error vector. OLS technique, used in forecasting model parameters (β), is based on minimization of Eq. (3). (Y Ŷ ) = ε ε = (Y Xβ) (Y Xβ) (3)

5 1272 Cagcag et al. In order to make ˆβ estimator blue (best linear unbiased) which was obtained from OLS technique, error should be 0 mean or σ 2 variance. Unbiased estimator of σ 2 is as in Eq. (4). ˆσ 2 = 1 n p (Y X ˆβ) (Y X ˆβ) (4) Residual analysis, widely used in detecting outlier in regression analysis, is an effective method. In the model, the residual of i th observation is calculated as: e i = y i ŷ i i = 1, 2,...,n (5) 3. Particle Swarm Optimization In this study, in cases where there is an outlier observation in data set, PSO algorithm is used for the purpose of parameter estimation. PSO technique was proposed by Kennedy and Eberhart (1995). PSO is a population-based optimization algorithm. The most important feature of this algorithm is its ability to reach the optimum point from the several points at the same time. Because of this feature, local optimum does not block it and it has a better chance to reach global optimum. Modified PSO method, whose algorithm given below, is used in the study. In the given algorithm, inertia weight was taken as time-varying as stated in Shi and Eberhart (1999). Similarly, time-varying acceleration coefficients were employed as in the study by Ma et al. (2006). Algorithm 1. Modified PSO Algorithm Step 1. Optimization of particles (xi k,i = 1, 2,...,d; k = 1, 2,...,pn) are selected randomly and stored in X. X = { x k 1,xk 2,...,xk d},k = 1, 2,...,pn (6) Here, p n represents the number of particle, whereas d shows the number of position in each particle. Step 2. Velocities are determined randomly and stored in V. V = { v1 k },vk 2,...,vk d (7) Step 3. pbest and gbest are generated based on performance function. pbest i = (p i1,p i2,...,p id ) i = 1, 2,...,d (8) pbest g = gbest = (p g1,p g2,...,p gd ) (9) Here, pbest are the best positions of particle in separate iterations whereas gbest are the best positions of all particles in a population. Step 4. Possible intervals are determined for, where w is inertia parameter, c 1 and c 2 are cognitive and social coefficients. Inertia parameter (w), cognitive (c 1 ) and social (c 2 ) coefficients are calculated in each iteration in accordance with the following formula. t c 1 = (c 1f c 1i ) maxt + c 1i (10)

6 New Robust Regression Method Based on Particle Swarm Optimization 1273 Figure 1. Presentation of a particle in PSO. t c 2 = (c 2f c 2i ) maxt + c 2i (11) w = (w 2 w 1 ) maxt t + w 1 maxt (12) Here, (c 1i,c 1f ) are possible intervals for cognitive coefficient, (c 2i,c 2f )forsocial coefficient, and (w 1,w 2 ) for inertia parameters, respectively. maxt represents maximal number of iteration, and t represents valid iteration number. Step 5. New velocities and positions are calculated according to the formula given below. v k+1 id = [ w vid k + c 1 rand 1 (pbest id x id ) + c 2 rand 2 (pbest gd x id ) ] x k+1 id = x id + v k+1 id where rand 1 and rand 2 are random numbers between 0 and 1. Step 6. Steps 1 5 are repeated until a predetermined number of iteration (maxt) is reached. 4. The Proposed Method OLS method, widely used in regression analysis for parameter estimation, is highly affected by the presence of outlier observations in data set. In literature, parameter estimations are calculated by giving outliers lower weights in M-estimators. Using a median rather than error mean may decrease the effect of outliers without need for weight values. In literature, the mean absolute error (y i ŷ i ) is minimized for parameter estimation by using least median squares approach. Additionally, when compared with the absolute error, proportional error is considered to be a more valid error criterion. Akbilgic and Akinci (2009) proposed a new approach that minimizes mean proportional error. In this study, a method based on PSO which cannot be affected by outliers in data set in forecasting model parameter was proposed. The proposed method relies on minimizing the median of proportional error differently from M-estimator, least median squares and Akbilgic and Akinci s method. The proposed method uses the expression given in (15) as an objective function. ( min median β y i ŷ i y i ) (13) (14) (15) Here, ŷ i is y i β 0 β 1 x 1i β p x pi and β is [β 0 β 1 β p ]. As this objective function is based on median of absolute proportional error, it enables obtaining parameter estimations without taking outliers into consideration. As it is difficult to create normal equations in the optimization of given in (15), the optimization can be achieved with PSO. In prediction p + 1 parameter in PSO, the state of particle is expressed as in Fig. 1.

7 1274 Cagcag et al. gbest values obtained by employing algorithm 1 for the objective function in (15) are regression estimators. These estimators are express as follows. ˆβ PSO = [ ˆβ 0 ˆβ 1 ˆβ p ] (16) In the proposed method, the estimation of σ 2 is calculated as in Eq. (17). As this estimator relies on median, it will be less affected by the outlier in comparison with the OLS estimator given in (4) formula. ˆσ 2 PSO = Median( (Y Ŷ ) 2) (17) 5. Simulation Study for the Performance Evaluation of the Proposed Method A simulation study was designed to compare the proposed method with those found in the literature. In the simulation study, the performance of the methods was investigated for different values of the observation number and for σ 2. While evaluating the performance of the method, proximity to the true parameter value for β.dσ 2 parameter is calculated by the following criteria. MSE (β) = MSE (σ 2 ) = 1 tkr 1 (p + 1) tkr tkr j=1 p tkr (β ij ˆβ ij ) 2 (18) i=0 j=1 ( σ 2 j ˆσ j 2 ) 2 (19) The parameter of the method with small MSE value makes more accurate estimates. In Eqs (18) and (19), β.dσ 2 are real parameter values, ˆβ and ˆσ 2 are the parameter estimations obtained from simulation data. tkr represents the number of simulation data. To evaluate the performance of the method, below MSE values and the percentages of the method giving minimum error were used. MSE j (β) = 1 (p + 1) p (β i ˆβ i ) 2 j = 1, 2,...,tkr (20) i=0 MSE j (σ 2 ) = (σ 2 ˆσ 2 ) 2 j = 1, 2,...,tkr (21) For simulation, bivariate multiple regression model was designed. This model can be expressed by expressions (22) and (23). y i = β 0 + β 1 x 1i + β 2 x 2i + e i i = 1, 2,...,n (22) y i = 1 + 1x 1i + 1x 2i + e i i = 1, 2,...,n (23) Independent variables in the model were derived from a normal distribution with a mean of 100 and a variance of 100. Additionally, four different error data were created from normal distribution with a mean of 0 and a variance of 1, 9, 25, and 100, respectively. Moreover, four different data having these features with a sample size of 30, 50, 100, and 500 were created. Besides, data sets are transformed into data set with outliers by creating one, two, and three outliers with a value of 500. Simulation was performed with 10,000 iterations for each situation. At the end of the simulation, obtained results for the data sets

8 New Robust Regression Method Based on Particle Swarm Optimization 1275 Table 1 n = 30 and y 15 = 500 σ ˆβ 0 ˆβ 1 ˆβ 2 ˆσ 2 MSE (β) MSE (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% which are n = 30 and n = 50, and the success rates (SRs) of the method within the 10,000 iterations are shown in Tables 1 6. For the other situations, the SR of the methods is given in Tables 7and 8. A total of 48 different situations were analyzed in the simulation study. When the tables that summarize the results of the analyzed situations were considered, little estimation error was showed in proposed method for both model coefficients (β 0,β 1,β 2,,β p ) and variance of error, another parameter of the model, (σ 2 ) (see columns MSE(β) and MSE ( σ 2) of Tables 1 6). It can be seen that most of the 10,000 iterations of model coefficients (β 0,β 1,β 2,,β p ) in each situations are estimated with less error. Moreover, different simulation study was performed. We used chi-squared and F distributions that are known as skewed distributions. The aim of these simulations gives more evidence to test the efficiency of the proposed method. In the simulation, model can be expressed by (22) and (23) expressions. y i = β 0 + β 1 x 1i + β 2 x 2i + e i i = 1, 2,...,n (24) y i = 1 + 1x 1i + 1x 2i + e i i = 1, 2,...,n (25) Independent variables in the model were derived from a chi-squared distribution with two degrees of freedom and F distribution with 5 and five degrees of freedom. Additionally, four different error data were created from normal distribution with a mean of 0 and a Table 2 n = 50 and y 25 = 500 σ ˆβ 0 ˆβ 1 ˆβ 2 ˆσ 2 MSE (β) MSE (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00%

9 1276 Cagcag et al. Table 3 n = 30 and y 5, y 25 = 500 σ ˆβ 0 ˆβ 1 ˆβ 2 ˆσ 2 MSE (β) MSE (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% Table 4 n = 50 and y 10,y 40 = 500 σ ˆβ 0 ˆβ 1 ˆβ 2 ˆσ 2 MSE (β) MSE (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% Table 5 n = 30 and y 5,y 15,y 25 = 500 σ ˆβ 0 ˆβ 1 ˆβ 2 ˆσ 2 MSE (β) MSE (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00%

10 New Robust Regression Method Based on Particle Swarm Optimization 1277 Table 6 n = 50 and y 10,y 25,y 40 = 500 σ ˆβ 0 ˆβ 1 ˆβ 2 ˆσ 2 MSE (β) MSE (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% PROPOSED % % OLS % 0.00% LSR % 0.00% Table 7 Proportion of achievement in case of n = 100 y 50 = 500 y 25,y 75 = 500 y 25,y 50,y 75 = 500 σ SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % 90.90% % 93.10% % OLS 1.30% 0.00% 0.70% 0.00% 0.60% 0.00% LSR 12.10% 0.00% 8.40% 0.00% 6.30% 0.00% PROPOSED % % 91.30% % 92.30% % OLS 1.80% 0.00% 0.60% 0.00% 0.60% 0.00% LSR 9.40% 0.00% 8.10% 0.00% 7.10% 0.00% PROPOSED % % 93.60% % 94.40% % OLS 2.00% 0.00% 0.70% 0.00% 0.60% 0.00% LSR 6.90% 0.00% 5.70% 0.00% 5.00% 0.00% PROPOSED % % 93.80% % 95.70% % OLS 1.60% 0.00% 1.60% 0.00% 0.60% 0.00% LSR 4.60% 0.00% 4.60% 0.00% 3.70% 0.00% Table 8 Proportion of achievement in case of n = 500 y 250 = 500 y 100,y 400 = 500 y 100,y 250,y 400 = 500 σ SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % 61.00% % 68.40% % OLS 3.30% 0.00% 3.20% 0.00% 3.70% 0.00% LSR 47.00% 0.00% 35.80% 0.00% 27.90% 0.00% PROPOSED % % 68.40% % 72.90% % OLS 7.10% 0.00% 5.00% 0.00% 3.40% 0.00% LSR 29.70% 0.00% 26.60% 0.00% 23.70% 0.00% PROPOSED % % 77.30% % 75.90% % OLS 8.10% 0.00% 5.30% 0.00% 4.50% 0.00% LSR 19.50% 0.00% 17.40% 0.00% 19.60% 0.00% PROPOSED % % 83.60% % 86.70% % OLS 7.30% 0.00% 6.50% 0.00% 4.50% 0.00% LSR 10.30% 0.00% 9.90% 0.00% 8.80% 0.00%

11 1278 Cagcag et al. Table 9 Proportion of achievement in case of independent variables with chi-squared distribution One outlier Two outliers Three outliers σ SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % % % % % OLS 15.10% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 0.10% 0.00% 0.00% 0.00% 0.00% 0.00% PROPOSED % % 99.70% % 99.20% % OLS 34.00% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 0.10% 0.00% 0.30% 0.00% 0.80% 0.00% PROPOSED % % 94.40% % 93.30% % OLS 40.70% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 0.40% 0.00% 5.60% 0.00% 6.70% 0.00% PROPOSED % % 71.40% % 71.30% % OLS 41.50% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 8.70% 0.00% 28.60% 0.00% 28.70% 0.00% Table 10 Proportion of achievement in case of independent variables with F distribution One outlier Two outliers Three outliers σ SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) SR (β) SR (σ 2 ) PROPOSED % % % % % % OLS 2.70% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% PROPOSED % % 98.70% % 99.30% % OLS 22.60% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 0.50% 0.00% 1.30% 0.00% 0.70% 0.00% PROPOSED % % 92.10% % 90.40% % OLS 35.80% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 2.80% 0.00% 7.90% 0.00% 9.60% 0.00% PROPOSED % % 69.40% % 71.10% % OLS 43.80% 0.00% 0.00% 0.00% 0.00% 0.00% LSR 11.10% 0.00% 30.60% 0.00% 28.90% 0.00% Table 11 Sample data containing outlier in its dependent variable n i x 1i x 2i x 3i y i n i x 1i x 2i x 3i y i

12 New Robust Regression Method Based on Particle Swarm Optimization 1279 Table 12 Results of various regression methods for sample data ˆβ 0 ˆβ 1 ˆβ 2 ˆβ 3 MDAPE OLS MESTIMATOR Huber Hampel Tukey Andrews Sanli LSR Proposed variance of 1, 9, 25, and 100, respectively. Moreover, data having these features with a sample size of 500 were created. Besides, data sets are transformed into data set with outliers by creating one, two, and three outliers. Outlier values were obtained by multiplying the maximum observation value of the original data by 10. Then, 250th, 100th and 400th, 100th, 250th and 400th observation of the original data, which is the maximum observation value, were changed by these outliers. Simulation was performed with 10,000 iterations for each situation. At the end of the simulation, obtained results for the data sets are shown in Tables 9 and 10. To strengthen the superior performance of the proposed method, in addition to the simulation study, another analysis was done on a single data that was given in Table 11, and results were compared with the other robust methods. where X 1 (μ = 20,σ = 3), X 2 (μ = 50,σ = 12), X 3 (μ = 32,σ = 13). Dependent variables are created as below but 15th observation in dependent variable was transformed into outlier to be as y y i = 1 + 1x 1i + 1x 2i + 1x 3i + e i i = 1, 2,...,n (26) Additionally, the error data was created from normal distribution with a mean of 0 and a variance of 9. The results of the proposed method as well as the results of some robust methods were summarized in Table 12. In this application, MDAPE criterion was used in the comparison of methods. As MDAPE criterion is based on median, its use will be more reliable in comparing methods which are used for the analysis of data containing outlier. ( ) y i ŷ i MDAPE = median, i = 1, 2,,n (27) y i When Table 12 was analyzed, it was evident that the proposed method was superior to other methods in terms of MDAPE criterion. 6. Conclusions and Discussion OLS is one of the most frequently used methods in regression analysis which are most frequently referred in estimation problems. But, being sensitive to the outliers in the data

13 1280 Cagcag et al. led to the researchers to put forward alternative methods known as robust methods. M- estimators, one of the robust methods, focus on minimizing a function of residuals rather than the squares of residuals. Additionally, those methods which are based on simple fuzzy regression analysis and minimizing the proportional error rather than absolute error are also available in literature. In this study, a new method minimizing the median of proportional error with PSO in analyzing data containing outlier was proposed. The performance of the proposed method was applied to a simulation study, and a derived single data as well and obtained results were evaluated. When the results obtained from the application were analyzed, it was seen that proportional error was minimized with PSO and the method displayed superior performance when compared with the some other robust methods. The study results also revealed that estimators in the proposed method were more effective and unbiased in comparison with the OLS method. Although, the proposed method provides more biased results in comparison with the LSR method, it is seen that the proposed method is a better estimator. Therefore, it can be concluded that this method provides better estimation results in regard to both LSR and OLS methods. References Akbilgic, O., Akinci, E. D. (2009). A novel regression approach: Least squares ratio. Commun. Stat. Theory Methods 38: Candan, M. (1995). Robust estimator in linear regression. Ph.D. Thesis, Hacettepe University Ankara. Dave, R.N., Krishnapuram, R. (1997). Robust clustering methods: A unified view. IEEE Trans. Fuzzy Syst. 5(2): Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., Shatel, W. A. (1986). Robust Statistics (pp , 502 p). New York: John Wiley & Sons. Hogg, R. V. (1979). Statistical robustness: One view of its use in applications today. Am. Stat. 33(3): Huber, P. J. (1981). Robust Statistics (pp. 1 20, , 308 p). New York: John Wiley & Sons. Huynh, H. (1982). A comparison of four approaches to robust regression. Psychol. Bull. 92(2): Kennedy, J., Eberhart, R. (1995). Particle swarm optimization. In Proceedings of IEEE International Conference on Neural Networks, pp , Piscataway, NJ: IEEE Press. Ma, Y., Jiang, C., Hou, Z., Wang, C. (2006). The formulation of the optimal strategies for the electricity producers based on the particle swarm optimization algorithm. IEEE Trans. Power Syst. 21(4): Rousseuw, P. J., Leroy, A. M. (1997). Robust Regression and Outlier Detection (pp , 329 p.). New York: John Wiley & Sons. Sanli, K. (2005). Fuzzy robust regression analysis, Ph.D. Thesis. Ankara University, Ankara. Shi, Y., Eberhart, R. C. (1999). Empirical study of particle swarm optimization. Proc. IEEE Int. Congr. Evol. Comput. 3: