Development of Virtual Lab System through Application of Fuzzy Analytic Hierarchy Process

Transcription

1 Development of Virtual Lab System through Application of Fuzzy Analytic Hierarchy Process Chun Yong Chong, Sai Peck Lee, Teck Chaw Ling Faculty of Computer Science and Information Technology, University of Malaya Malaysia Abstract Virtual lab has gained popularity among cloud computing practitioners due to its promise of on-demand provisioning of computer resources. However, lack of standardized guidelines hinders the adoption among educational institutions. Achievements of non-functional requirements possess greater challenge due to their qualitative nature. These requirements can be achieved by applying guidelines during software development. However, priority assessment of non-functional requirements is needed before the selection of suitable guidelines. This paper tries to apply fuzzy analytic hierarchy process in order to aggregate group decisions, as well as dealing with ambiguities of decision makers judgement. The application of fuzzy analytic approach in this paper had successful create a solid foundation for software developers to identify the essential non-functional requirement in virtual lab environment. Importantly, the result can provide a promising reference model for better understanding of cloud computing in educational sector. Keyword virtual lab, non-functional requirement, priority assessment I. INTRODUCTION The emergence of cloud computing introduces new options for small medium enterprises, as well as IT industry giants to generate decent revenue and exploration of new marketing strategies. Cloud computing has evolved the way how software are purchased and provided in a very cost effective way. Apart from pushing corporate business to the next level of growth, cloud computing also has strong potential in the education sector especially in terms of setting up a virtual lab environment. There are several initiatives from higher education institution that had successfully integrated virtual lab into their universities [1]. These projects are aiming at providing on-demand cloud computing services through efficient use of available hardware resources. Users are able to access on-demand computational, storage and software resources The success implementation of virtual lab draws attention to higher education sector as in how to make use of cloud computing to improve the way of educating future generations [2]. Virtual lab is capable of providing numerous benefits to higher education institutions. First of all, universities are able to cut down expenses in purchasing hardware and software through virtualization technology. All they need are simply a thin client that is capable of establish a remote connection with a designated server hosting the virtual lab. Besides that, universities are able to avoid over-utilization or under-utilization of hardware resources by modifying virtual machine hardware specification on the fly. This indirectly contributes to green computing due to the fact that under-utilization of computing resources is to be kept minimum. Wastage of power consumption caused by idle computing resources can be avoided. Despite the fact that cloud computing had proven to be beneficial in the education sector, to come out with a good system design and implementation is one of the biggest challenges faced by most of the earlier adopters. Shifting into cloud computing does not make success inevitable, proper planning does. In order to successfully transition to this new environment, system based on cloud environment must be developed with a certain level of software quality besides fulfilling their functional requirements [3]. Software quality implies how the system should behave besides having all the essential functionalities. Fulfilling both functional and non-functional requirements is a demanding task especially when stepping into a new field. II. RELATED WORK There are typically two challenges in requirement engineering. Firstly, software requirements are usually imprecise and define qualitatively. Customers often specify their requirements verbally making the requirements capturing process less effective. Besides that, quality attribute requirements often conflict with each other. For example, designing a system that is extremely secure will inevitably cause negative impact on the performance quality. This kind of 207

2 dilemma has to be identified in the early stage of system design so that stakeholders are aware of the trade-off. Priority analysis is one of the conflict resolution techniques to resolve trade-off between conflicting quality requirements. The relative priority and importance of quality requirements plays a significant role. However, it is often difficult for stakeholders to directly provide the priorities for all quality requirements due to the multifaceted relationship between each other. The nature of prioritizing quality attribute requirements is a multi-criteria decision-making (MCDM) problem. MCDM is a research of methods and procedures by which concerns about evaluating multiple conflicting criteria and derive a way to come to a compromise. This set of criteria often differs in the degree of importance. Analytic Hierarchy Process (AHP) has been a tool that is widely used and adopted by decision makers and researchers to aid in priority analysis [4]. AHP works by taking judgmental input from a group of decision makers and construct a hierarchy of criteria according to their priority. The simplest form of prioritizing criteria in AHP is based on nine-point scale, which expresses preferences between options ranging from equal importance to extreme importance. Regardless of the popularity, AHP is less effective when handling imprecision and uncertainties associated with human judgment. Natural language tends to be ambiguous and vague. It is extremely hard to implement AHP when there are no method to synthesise human s judgement into exact numerical values. Thus, in order to deal with the uncertainties and vagueness of human s judgement, scholar and researchers came out with a modified version of AHP known as fuzzy AHP [5] which incorporates fuzzy set theory introduced by Zadeh [6]. Fuzzy set theory is capable of accepting crisp input, or in this case, uncertain judgement from decision makers. After accepting crisp inputs, fuzzy set theory then determines the degree to which these inputs belong to the appropriate fuzzy sets. This process will then be followed by defuzzification process, which produces a quantifiable result normally in the form of numerical value. By integrating fuzzy set theory, AHP is capable of handling the fuzziness of the data involved in decision making efficiently. Based on the discussion above, it is clear that requirement specification and architecture design phase is crucial to produce high quality software. Software quality attributes are to be specified in the early stage of development. Software engineers can then tailor the software based on the requirements captured beforehand. Nevertheless, during the implementation and deployment phase, developers have to make sure that the stated quality attributes must be present in the system. III. PRIORITY ASSESSMENT OF VIRTUAL LAB IN EDUCATION SECTOR This paper will start off by using fuzzy AHP method discussed above. Fuzzy AHP method will be conducted in order to perform priority assessment of software quality attributes for the development of virtual lab. In both traditional and fuzzy AHP, the process starts by modeling a hierarchy of decisions based on the problem domain. The top of the hierarchy consist of the goal for conducting the test, followed by a group of possible choices to achieve that particular goal. The choices can be further divided into sub-criteria if required. The problem domain for this paper is focusing on virtual lab environment in education sector. The selection of evaluation criteria is based on ISO/IEC 9126 Software Quality which includes six main criteria: Functionality, Reliability, Usability, Efficiency, Maintainability, and Portability [7]. However, since numerous research indicate that the main concern in cloud computing is security, the chosen ISO/IEC 9126 standard is extended to fit into this context. Security is brought out as one of the main criteria for evaluation, thus making the total number of evaluation to seven. A pair-wise comparison among all the possible choices will be conducted to justify the importance between them. Each choice is associated with a weightage in order to reflect the priority of each choice toward the ultimate goal. Decision makers will perform a pair-wise comparison and give weightage using a nine-point scale ranging from 1-9, where a greater value represents higher importance. In order to reach a consensus among the decision makers, triangular fuzzy number (TFN) is used. TFN is capable of aggregating the subjective opinions of all the decision makers through fuzzy set theory. The triangular fuzzy number Txy is constructed using the following formula (1)-(3): where x and y represent a pair of criteria being judged by decision makers. Jxya represents an opinion of decision maker a toward the relative importance for criteria Cx- Cy. As we can see from formula (2), value Mxy is produce by calculating the geometric mean of decision makers scores for a particular comparison. The geometric mean is capable of accurately aggregating and representing the consensus of decision makers[4]. After getting the TFN value for every pair of comparison, a fuzzy pair-wise comparison matrix is established in the form of n x n matrix. Table 1 illustrates an example of the matrix. Table 1: Fuzzy pair-wise comparison matrix C a C b.. C n C a 1 p ab.. p an C b 1/p ab 1.. p bn C n 1/p an 1/p bn.. 1 p ab represents the triangular fuzzy number for the comparison between criteria C a and C b. Comparison between criteria C b to C a is the reverse of C a to C b, thus making the TFN value for C b to C a to be represented as 1/p ab. denotes the TFN value derived from formula (1)-(3) 208

3 Table 2: Fuzzy pair-wise comparison matrix based on accumulated judgement Functionality Reliability Usability Efficiency Maintainability Portability Security Functionality 1 1/4, 1.637, 5 1/6, 1.37, 5 1/7, 1.387, 9 1/5, 1.58, 9 1/7, 1.321, 7 1/9, 0.865, 4 Reliability 1 1/3, 1.488, 5 1/3, 1.675, 7 1/7, 1.304, 5 1/4, 1.16, 7 1/5, 0.861, 9 Usability 1 1/5, 1.062, 5 1/7, 1.508, 9 1/3, 1.13, 7 1/5, 0.861, 9 Efficiency 1 1/3, 2.433, 7 1/4, 1.203, 5 1/7, 1.134, 9 Maintainability 1 1/7, , 5 1/7, 0.655, 4 Portability 1 1/9, , 9 Security 1 The stakeholders who participate in this evaluation include university students and developers who have experiences in cloud computing development. Twenty participants were chosen to ensure the consistency of AHP testing. As discussed by Aull-Hyde and co-author [8], under the condition when comparison matrix size (number of criteria to be evaluated) is 6x6, the group size threshold to achieve acceptable level of inconsistency is twenty participants. This means that as long as the numbers of participants are more than twenty people, one can ensure that the aggregated results will be consistent and revision of judgement is not needed. Each participant was instructed to perform pair-wise comparison and give their judgement based on the relative score of 1-9. The triangular fuzzy numbers were established using formula (1)-(3) and tabulated into a comparison matrix shown in Table 2. The lower side of the matrix was intentionally left blank because it carries insignificant value before defuzzification of TFN value. Following the construction of comparison matrix, defuzzification will take place to produce a quantifiable value based on the calculated TFN value. The defuzzification method adopted in this paper was derived from Lious and Wang shown in formula (4) which is based on the alpha cut manner [9]. and ( ) [ ] (4) On the other hand, represent the right-end boundary value of alpha cut for. and in this context carry the meaning of preferences and risk tolerance of decision makers. These two values range between 0 and 1, in such a way that a lesser value indicates greater uncertainty in decision making. Since preferences and risk tolerance is not the focus of this paper, value of 0.5 for and will be used to represent a balance environment. This indicates that decision makers are neither extremely optimistic nor pessimistic about their judgments. The results from Table 2 were then synthesized using formula (4) with and at 0.5. An example of calculation is shown below for the pair Functionality-Reliability: ( ) ( ) Table 3 represents the result of fuzzy pair-wise comparison after defuzzification process. such that, which represents the left-end boundary value of alpha cut for. 209

4 Table 3: Aggregated fuzzy pair-wise comparison matrix Functionality Reliability Usability Efficiency Maintainability Portability Security Functionality Reliability Usability Efficiency Maintainability Portability Security The next step is to determine eigenvalue and eigenvector of the fuzzy pair-wise comparison matrix. The purpose of calculating eigenvector is to determine the aggregated weightage of particular criteria. Assume that denotes the eigenvector while denotes the eigenvalue of fuzzy pair-wise comparison matrix, [ ( ) ] (5) Formula (5) is based on the linear transformation of vectors, where I represent the unitary matrix. By applying formula (1)-(5), the weightage of particular criteria with respect to all other possible criteria can be acquired. The eigenvectors of associated non-functional requirements in virtual lab were then calculated using formula (5). [ ( ) ] Multiplying eigenvalue with unitary matrix I produce an identity matrix that nullifies each other. Thus, applying formula (5) results in The aggregated result in terms of weightage is tabulated in Table 4. The results obtained are ordered as follows: Functionality (0.257), Reliability (0.192), Usability (0.148), Efficiency (0.121), Security (0.107), Maintainability (0.103), and Portability (0.072). Table 4: Weightage and priority of selected criteria Priority Quality Attribute Weightage 1 Functionality Reliability Usability Efficiency Security Maintainability Portability The result gathered from fuzzy AHP will act as a baseline to model the system architecture for virtual lab system. Attribute-driven design for instance, requires non-functional requirements to be stated explicitly in order to design system architecture. This paper will focus on how to achieve higher software quality through better fulfillment in non-functional attributes. IV. DISCUSSION The result from priority assessment implies that stakeholder emphasis more on the fulfilment of functionality. They are concern on whether or not the virtual lab is capable of delivering appropriate functions to serve the users. Reliability is also a major concern based on the results. The idea behind virtual lab is that users are able to remote access computing resource regardless of location, as long as they have an Internet connection and a web browser. Thus, the system must be extremely reliable and maintain high availability in order to fulfil the user s request. Security on the other hand, falls in 210

5 priority five. This phenomenon is due to the nature of virtual lab, which is normally deployed in private cloud environment and does not involve money transaction. The priority assessment is based on fuzzy AHP approach, which works effectively in aggregating group decisions. Prioritizing requirements usually fail because the involvement of multiple qualitative criteria and scoring on subjective problems. Fuzzy AHP is a suitable evaluation framework capable of handling MCDM problem and uncertain inputs. Applying fuzzy AHP approach in the virtual lab environment not only enables stakeholders to prioritize requirements, but also helps in dealing with disagreement among stakeholders. The aggregated results obtained from priority assessment of software quality is based on the assumption that value and = 0.5 using alpha-cut method. Value and are very much dependent on environmental uncertainties. These values will directly affect the weight of individual criteria and priority ranking. If the participants involved in priority assessment have strong background knowledge on virtual lab, value of and can be increase to indicate a confident judgment. In-depth research and analysis is needed in order to determine the value of and accurately. The accurateness of results using fuzzy AHP can be further improved by investigating the impact of and value toward the weightage of associated non-functional requirements. Conventional methods to aggregate diverse stakeholders opinions such as interviews, questionnaire, and intuitive approach are less effective and bias. This is because the accuracy of interview and intuitive approach are very much depending on the experience of the facilitator. Furthermore, it is very hard for inexperience facilitator to create an effective questionnaire that is capable of cater each and every requirement. On the other hand, the fuzzy AHP approach is capable of minimize bias and maximize accuracy through the usage of fuzzy triangular number. The results from fuzzy AHP can assists in selection of appropriate guidelines to achieve the desire quality attributes. Software engineers can select sets of guidelines that are homogeneous in such a way that does not have conflicting and overlapping relationships with one another based on the prioritization result. By applying these suitable guidelines, software developers will less likely need to reason about all the complexity behind the associated quality attributes. capturing stakeholders requirements and achievement of multiple software quality attributes in a virtual lab environment. This paper has made several contributions toward the establishment of virtual lab in education sector. Software developers able to pinpoint the essential quality attribute to ensure the success implementation of virtual lab solution. This will enable software developers to concentrate on the most important quality requirements and achieve high satisfaction among stakeholders. However, it is to be noted that the implementation of proposed approach is based on a virtual lab case study. The results will differ depending on the experiences and knowledge of participants. Further work can be considered by involving subject matter experts from different higher education institutions across the world. The aggregated opinions from experts around the world can be used to model the system architecture for virtual lab through attribute-driven design. Eventually, the system architecture can act as a standardized framework to introduce virtual lab solution in the educational sector. REFERENCES [1] H. E. Schaffer, S. F. Averitt, M. I. Hoit, A. Peeler, E. D. Sills, and M. A. Vouk, "NCSU's Virtual Computing Lab: A Cloud Computing Solution," Computer, vol. 42, pp , [2] T. Ercan, "Effective use of cloud computing in educational institutions," Procedia - Social and Behavioral Sciences, vol. 2, pp , [3] S. H. Kan, Metrics and models in software quality engineering, 2nd ed. Boston: Addison-Wesley, [4] T. L. Saaty, The analytic hierarchy process : planning, priority setting, resource allocation. New York ; London: McGraw-Hill International Book Co., [5] P. J. M. van Laarhoven and W. Pedrycz, "A fuzzy extension of Saaty's priority theory," Fuzzy Sets and Systems, vol. 11, pp , [6] L. A. Zadeh, "Fuzzy sets," Information and Control, vol. 8, pp , [7] ISO/IEC-9126, "Software engineering - Product Quality," ed, [8] R. Aull-Hyde, S. Erdogan, and J. M. Duke, "An experiment on the consistency of aggregated comparison matrices in AHP," European Journal of Operational Research, vol. 171, pp , [9] T.-S. Liou and M.-J. J. Wang, "Ranking fuzzy numbers with integral value," Fuzzy Sets Syst., vol. 50, pp , V. CONCLUSION AND FUTURE WORK Prioritization of quality requirements plays an important role to help software developers focus on fulfilling higher priority quality within the limited budget and time. However stakeholders often specify requirements quantitatively using some natural language. Judgments of stakeholders are usually subjective and ambiguous. Even if quality requirements can be prioritized, most of the software engineers only look into one quality attribute at one time, neglecting the contradicting effects among each quality attribute. The fact is that most of the software systems are required to have multiple software quality attributes at the same time. This paper addresses the problem of 211