Available online at www.sciencedirect.com Systems Engineering Procedia 00 (2011) 000 000 Systems Engineering Procedia 4 (2012) 275 281 Systems Engineering Procedia www.elsevier.com/locate/procedia 2 nd International Conference on Complexity Science & Information Engineering Research on the Comprehensive Evaluation of Business Intelligence System Based on BP Neural Network Su-Li Yan *, Ying Wang, Ji-Cheng Liu North China Electric Power University, Being 102206,China Abstract In this paper, the current situation of comprehensive evaluation of business intelligence system (BIS) is analyzed, and the objectives and principles of establishing the evaluation system are elaborated. The evaluation index system of BIS is structured, and the Analytical Hierarchy Process (AHP) technique is used to confirm the index weights, and the fuzzy TOPSIS is selected to determine the synthetically scores of different companies BIS. Then how to use BP neural network to evaluate BIS is discussed. The simulation results finally verify the scientific and effectiveness of this method. The study will provide a more scientific and reliable basis for systems engineering decision analysis. 2011 Published by Elsevier Ltd. Selection and peer-review under responsibility of of Desheng Dash Dash WuWu. Open access under CC BY-NC-ND license. Keywords: business intelligence system (BIS); engineering index system; Analytical Hierarchy Process; fuzzy TOPSIS; BPNN 1. Introduction Modernized enterprise business operations can produce large amounts of data, and traditional analysis tools and methods have been unable to meet timely and accurately requirements of business analysis, therefore the business intelligence system (BIS) is emerged as the times require. With the development of information technology, the BIS have been exploited widely in enterprises, and most of them are using it to do with business quickly and efficiently. Modern BIS reflects a rational management decision-making ability. It can integrate from multiple date types of different sources and discover new knowledge from data to improve accuracy of forecasting and decision-making process. In recent years, domestic and foreign scholars actively engaged in researching on BIS, and some results have been achieved in theoretical research and practical application. Gartner Group was first proposed BIS, which was defined as a class of technology and application to help corporate decisionmaking that made up of data warehouse, query and reporting, online analysis, data mining, data backup and recovery and so on [1]. The BIS evaluation models were established from different levels, and AHP and fuzzy comprehensive evaluation methods were used to assess [2-4]. The support vector machine was chosen to evaluate the BIS [5]. A fuzzy multi-criteria approach, including fuzzy AHP and fuzzy TOPSIS methods, were proposed to assess the BIS [6-8]. Different evaluation BIS performance models were built from the business perspective [9-11]. Although the above methods have basically achieved the desired *Su-Li Yan. Tel.: +86-13641236650; fax:+86-010-51963569. E-mail address: wangying767@126.com. 2211-3819 2011 Published by Elsevier Ltd. Selection and peer-review under responsibility of Desheng Dash Wu. Open access under CC BY-NC-ND license. doi:10.1016/j.sepro.2011.11.076
276 Su-Li Yan et al. / Systems Engineering Procedia 4 (2012) 275 281 goals, there are some defects such as large subjective and arbitrary and low accuracy to determine indicators and no large-scale evaluation. Therefore, the systematic study and analysis on comprehensive evaluation of BIS will promote system beneficially. To be solved the problems of evaluation of BIS, a comprehensive evaluation index system of BIS is established according to the construction principles, and an overall evaluation method based on BP neural network is proposed. In this paper, the objectives and principles of overall evaluation system of BIS are described. On the basis of the evaluation index system, the Analytical Hierarchy Process (AHP) technique is presented to obtain the index weight, and the fuzzy TOPSIS is chosen to determine the operation condition of different companies BIS, then how to use BP neural network to evaluate BIS is discussed. The simulation results finally verify the feasibility and effectiveness of this method. 2. Building the BIS comprehensive evaluation index system 2.1. The objectives and principles of BIS comprehensive evaluation index system The objectives of BIS comprehensive evaluation index system are to reflect accurately benefits for enterprises due to the application of system and to help companies adjust policies and strategic planning and support management decision-making and engage in commercial activities. There are 4 principles should be followed to establish a scientific and reasonable index system. (1) System principle. The index system should meet the overall evaluation function is greater than the simple sum of sub-indicators. (2) Accuracy principle. The selected indicators can reflect the implementation of BIS in firms. (3) Independence principle. The indicators in the same level should not have containing relationships and too much information inclusive. (4) Comparability principle. The chosen indicators, whether qualitative or quantitative, can be used for horizontal and vertical comparison. 2.2. Comprehensive evaluation index system Based on the above objectives and principles, the BIS comprehensive evaluation index system is established, which include system construction operation and maintenance, system user satisfaction, system internal and external influences. The details are given by figure 1. BIS(Goal) System construction operation and maintenance (A) System user satisfaction(b) Internal influences(c) External influences(d) A1 Information infrastructure A2 Management support A3 Maintainability A4 Safety and reliability A5 Full participation A6 Resource utilization B1 Real-time B2 Practicality B3 Friendly B4 Economy C1 Decision-making C2 Customer identification C3 Cost value C4 Innovative business model D1 Sharing D2 Leading D3 Cost-effective D4 Information Quality Fig. 1. BIS Comprehensive Evaluation index system
Su-Li Yan et al. / Systems Engineering Procedia 4 (2012) 275 281 277 (1) Construction, operation and maintenance indicators There are 6 second indicators in view of the BIS construction, operation and maintenance. 1) Information infrastructure. It refers to the software and hardware environment and network infrastructure. 2) Management support. It mainly refers to the cooperation and support of construction and implementation of BIS for different departments. 3) Maintainability. This degree will directly affect the efficient operation in enterprises. 4) Safety and reliability. It refers to the system is not affected by natural and man-made threats and damage during its operation. 5) Full participation. It represents the degree of information democracy and sharing in firms. 6) Resource utilization. It refers to utilization of hardware and software equipments, data information and human resources. (2) Customer satisfaction index There are 4 second indicators in consideration of the internal staff satisfaction. 1) Real-time. It refers to the ability to provide timely information to users for data mining. 2) Practicality. It represents the actual use of the system in enterprises. 3) Friendly. It mainly refers to the good degree of man-machine interface. 4) Economy. It refers to the user using the system need to pay the total costs are reasonable. (3) Internal influences indicators There are 4 second indicators with the view of the impact on internal in enterprises. 1) Decision-making. It refers to use the system to analyze and process data and mining valuable information to improve decision-making level. 2) Customer identification. It represents the system to meet the needs of different users to improve the degree of customer relationships. 3) Cost value. It refers to investment and expansion of system to promote cost value. 4) Innovative business model. It refers to use information means to deal with lots of business and innovative model to enhance competitiveness. (4) External influences indicators There are 4 second indicators considering the impact on external in enterprises. 1) Sharing. It mainly refers to the sharing of information among systems, including the integration of various industry resources and prevention duplication of similar systems. 2) Leading. It represents the guiding of system which has been built to which is not built. 3) Cost-effective. It mainly refers to economic, social and sustainable development benefits. 4) Information Quality. It represents accuracy, validity and reliability of information data. 3. The overall evaluation method of BP neural network based on fuzzy TOPSIS 3.1. The fuzzy TOPSIS method The fuzzy TOPSIS method is based on the sort of ideal point, which is derived from the determination problem of the multivariate statistical analysis. The fuzzy TOPSIS is employed to determine the different companies' scores of BIS. Fuzzy TOPSIS steps can be given as follows: Step 1: Normalized decision matrix model According to alternative values under various properties, we can obtain the decision matrix D.In terms of attributes types in decision-making problems, normalized decision matrix R is calculated as:
278 Su-Li Yan et al. / Systems Engineering Procedia 4 (2012) 275 281 R = R= ( R ) m n = R = i i = 1 m = 1 1 D m D D 1 D (1) (2) Where the property is benefit, we choose equation 1 whereas the equation 2 can be chosen. Step 2: identify positive-ideal ( P + ) and negative ideal ( N ) solutions. The fuzzy positive-ideal solution and negative one are shown in the following equations: + P = { P 1 j n} = {maxr 1 j n} j 1 i m N = { N 1 j n} = {minr 1 j n} j 1 i m Where P + is associated with benefit criteria and N is associated with cost criteria. Step 3: Calculate the distance of each alternative from P + to N using following equations: (3) (4) n + 1 + DDP (, ) = 1 ( Du ( i) P ( ui)) n i= 1 2 (5) n 1 DDN (, ) = 1 ( Du ( i) N ( ui)) n i= 1 Step 4: Calculate similarities to ideal solution. 2 (6) DDN (, ) Si = + DDP (, ) + DDN (, ) where the index value is the score of alternative and lies between zero and one (7) 3.2. Back-propagation (BP) neural network model A typical BP neural network is a multi-hierarchic feedback structure, which is to adjust the network weights through back-propagation algorithm, including input layer, hidden layer and output layer. The network learning process involves forward propagation and back propagation. Information from input layer is treated through hidden layer, and transmitted to output layer in the forward propagation process. If the output value can not be the desired, error signal should return along the original connection path in back propagation process. The error of output layer node should transmit reversely to input layer to adjust the connection weights and thresholds to adapt the requirements of mapping. The model structure of BP neural network is given in figure 2. input output Fig. 2. BP neural network model structure
Su-Li Yan et al. / Systems Engineering Procedia 4 (2012) 275 281 279 BP neural network has the good mapping ability to all functions. A certain number of known samples are required to train the network before using it to evaluate. In this paper, the scores of different firms' BIS, determined by fuzzy TOPSIS method, are as the desired output of training samples of BP neural network. Through network learning process, connection weights and thresholds are constantly adjusted so that the output of network close to the desired output to meet the mapping. 4. Application We choose BIS of 15 companies in telecommunication industry for study, and use BP neural network method for simulation based on fuzzy TOPSIS. The concrete steps are proposed as follows: Firstly, we use AHP to get each index weight and normalized. For instance, we choose the pair wise comparisons with respect to goal to calculate local weights. The details are given in Table 1 and Table 2. Table 1. The pair wise comparisons with respect to goal O A B C D Weight Related values Consistency test A 1 5 1/3 3 0.2282 λmax 4.236003 CR=0.087409 B 1/5 1 1/5 1/2 0.0796 CI 0.078668 <0.10 C 3 5 1 3 0.5543 RI 0.9 It has passed the D 1/3 2 1/3 1 0.1379 consistency test. Table 2. The final weights of 18 indicators Index A1 A2 A3 A4 A5 A6 B1 B2 B3 Weight 0.010 0.055 0.011 0.011 0.034 0.018 0.017 0.009 0.041 Index B4 C1 C2 C3 C4 D1 D2 D3 D4 Weight 0.013 0.335 0.124 0.035 0.060 0.037 0.018 0.061 0.219 Secondly, we select an expert to grade 18 indicators on a basis of 4 evaluation levels, which are [0.91-1.00] for better, [0.81-0.90] for good, [0.71-0.80] for general, and [0.61-0.70] for poor. Thirdly, we choose fuzzy TOPSIS method to determine synthetically scores of 15 companies BIS. There is no need for normalization because 18 indicators are benefit and belong to [0,1]. The fuzzy positive-ideal solution and negative-ideal one can be found among 15 companies by Eq. (3) and (4), then the distance between each company and the fuzzy positive-ideal solution and negative-ideal one can be calculated by Eq. (5) and (6) respectively. Thus, final step solves the similarities to an ideal solution by Eq. (7). The final evaluation results are given in Table 3. Table 3. The final scores of 15 firms BIS F F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 F13 F14 F15 S 1 3 9 6 7 4 2 0.48 7 9 0.51 4 4 8 0.48 8 5 0 Fourth, we establish a BP neural network to comprehensively evaluate the BIS. The input layer nodes of neural network are 18 indicators, and the output layer nodes are synthetically scores of different
280 Su-Li Yan et al. / Systems Engineering Procedia 4 (2012) 275 281 companies' BIS. According to the empirical formula and several simulation comparisons and taking the evaluation error and network time into consideration, we ultimately determine the hidden layer nodes are 12. The hidden layer nodes and the output layer nodes are selected logsig function as activation function and transfer one, and trainlm function is used as training one. The learning rate of correction weight is 1%. The training error is 0.001. The maximum training number is 50. Modelling and network training are achieved in the Mat lab software platform. To train and test the network, we select index values of top 12 firms as input data of training samples, and synthetically scores as the desired output. The data set from 13 to 15 firms is as test set. The error graph can output as shown in first picture of figure 3 when performance function and error precision meet the requirements. 10 0 Performance is 2.31246e-005, Goal is 0.001 4 10-1 2 0.5 Training-Blue Goal-Black 10-2 10-3 10-4 8 6 4 2 10-5 0 0.5 1 1.5 2 2.5 3 3 Epochs 0.488 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 Fig.3. (a) BP neural network training error graph; (b) BP neural network testing results From the above results, we can see the training speed is quickly, and after two iterations the required accuracy is achieved. We can input the trained network three firms data to be tested for simulation. To observe conveniently, we use Mat lab software to output simulation chart, in which the outputs are compared with the desired ones. The contrast results are shown in the second picture of figure 3. From the figure 4, we can find the simulated output values are much closed to the desired ones. As shown in table 4, the score ranking of three companies BIS hold the line before and after simulation, and the relative errors are small. In short, the approach from network to discrete points works well. Table 4. The comprehensive evaluation results of BP neural network Firm The score of fuzzy TOPSIS ranking BP neural network ranking Relative error F13 0.4881 3 41 3 1.2292% F14 49 2 71 2 0.4439% F15 02 1 83 1-0.3861% In practice, we can use BP neural network to evaluate as long as the index values of BIS are given. 5. Conclusion Comprehensive evaluation of BIS is a complex task in engineering. In order to guide the construction and implementation of the system engineering better, this paper has built a hierarchical BIS engineering
Su-Li Yan et al. / Systems Engineering Procedia 4 (2012) 275 281 281 comprehensive evaluation index system on a basis of fully understand research situation. Because various indicators have different ultimately influences on BIS, AHP method is selected to determine index weights which is decomposition of a complex system to make people easy to understand and accept. To avoid subjective impact and uncertainty of human during evaluation process, the overall evaluation method of BP neural network based on fuzzy TOPSIS is discussed. This method not only uses accurate mathematical means to deal with fuzzy objects to realize a scientific and quantitative evaluation for fuzzy information, but simulates the expert system for quantitative evaluation of BIS to ensure objectivity and versatility of results. The final simulation experiment confirmed BP neural network method has a strong applicability in overall evaluation of BIS to provide guidance for use of BIS successfully in enterprises. Although the paper has made above achievements, we are still to study it further because of the limitations of BP neural network itself, including the BP algorithm is easy to fall into local optimum, and training a network has a large possibility to failure and so on. A direction research in the future is how to better improve the generalization performance of BP neural network. Reference [1] Zhang Qiao. Development and Trend Analysis of Business Intelligence. China Securities and Futures, 2009[2], p.14-7. [2] Deng Lihua, Liang Hongjuan. Research on evaluation system of business intelligence system. Telecommunications, 2009[9], p.38-42. [3] Sun Mingwei, Chen Guangxing. The construction of comprehensive evaluation model of business intelligence system. Journal of Tonghua Teachers College, 2009, 30[10], p.46-9. [4] Kamran Rezaie, Ayyub Ansarinejad, Abdorrahman Haeri.Evaluating the Business Intelligence Systems Performance Criteria Using Group Fuzzy AHP Approach. In: 2011 UKSim 13th International Conference on Modelling and Simulation, 2011, p.360-4. [5] Xia Guoen, Shao Pei. Synthetic Evaluation method based on support vector machine for business intelligence system. Application Research of Computers, 2009, 26[5], p.1789-1795. [6] Sinan Apak, Ozalp Vayvay. Evaluating an Intelligent Business Systems with a Fuzzy Multi-criteria Approach. In: 2009 Ninth International Conference on Intelligent Systems Design and Application, 2009, p.391-6. [7] A.H.Lee, W.Chen, C.Chang. A fuzzy AHP and BSC approach for evaluating performance of IT department in the manufacturing industry in Taiwan. In: Expert Systems with Applications, 2008,34[1], p.96-107. [8] C.C.Sun. A performance evaluation model by integrating fuzzy AHP and fuzzy TOPSIS methods. In: Expert Systems with Applications, 2010. [9] Trif,S. Using genetic algorithms in secured business intelligence mobile applications. Informatica Economica, 2011,15[1], p.69-79. [10] Y.H.Lin, K.M.Tsai, W.J.Shiang, T. C. Kuo, and C. H. Tsai. Research on using ANP to establish a performance assessment model for business intelligence systems. In: Expert Systems with Applications, 2009, 36[2], p.4135 4146. [11] A.Popovič, T.Turk, J.Jaklič. Conceptual Model of Business Value of Business Intelligence Systems. Management, 2010, 15[1], p.5 30. [12] Hostmann,B., N.Rayner, G.Herschel. Gartner's Business Intelligence, Analytics and Performance Management Framework. 2009, Gartner: Stamford. [13] Lahrmann,G.,Marx,F.,Winter,R.,Wortmann,F..Business Intelligence Maturity:Development and Evaluation of a Theoretical Model. Proceedings of the 44th Hawaii International Conference on System Sciences(HICSS 2011), 2011, p.10. [14] Dan Ma, Sheng-Wu Zhou, Shuo-Peng Wang.Business Evaluation Based on Entropy Method and BP Neural Network. In: Intelligent Systems and Applications, 2009. ISA 2009, International Workshop on, 2009, p.1-4. [15] Negash, S., P.Gray, Business Intelligence, in Handbook on Decision Support Systems 2, F. Burstein and C.W. Holsapple, Editors. 2008, Springer: Berlin, Heidelberg. p. 175-193.