Analysis of IS/IT Service Quality in the Higher Education with SERVQUAL: A Case Study of STMIK MDP Palembang Lisa Amelia 1, Achmad Nizar Hidayanto 2, Ika Chandra Hapsari 2 1 STMIK Multi Data Palembang Rajawali 14 Palembang, Indonesia, 0711-376400, 2 Faculty of Computer Science, Universitas Indonesia Kampus UI Depok, Indonesia, 021-7863419 lizza_cute@yahoo.co.id 1 nizar@cs.ui.ac.id, ika.chandra@ui.ac.id 2 Abstract - This study aims to analyze the quality of IS/IT related services, particularly at a university engaged in IS/IT field. To measure the quality of IS/IT services, we use a modification of the Service Quality (Servqual) Model. This method quantitatively measures the quality of services in the form of questionnaires and contains five dimensions of service quality which are tangibles, responsiveness, reliability, assurance, and empathy. The questionnaire results among 200 respondents showed that the quality of IS/IT service in STMIK MDP is a good enough. However there are still gaps between expectation and service performance of the IS/IT, with the highest and the lowest gaps are on the reliability and the assurance dimension respectively. Finally, it is also can be concluded that users tend to set higher expectation for IS/IT services in the universities engaged in IS/IT field, therefore such universities need to give more attention in their IS/IT facilities. Keywords: IS/IT Service Quality, Servqual, tangibles, responsiveness, reliability, assurance, empathy, performance, expectation INTRODUCTION Information systems/information technology (IS/IT) is very important in running the operations of an organization. IS/IT has role to meet the information needs of the business accurately, fast, and relevantly. In education, the IS/IT is used to gain competitive advantage in the competition among education institutions. IS/IT will work well if the factors inside it runs well too. These factors include well-prepared infrastructure, competent human resources/experts in their field and the appropriate software/applications that meet the needs of the organization. The contribution of the IS/IT that has been running well on an organization must be supported by the quality of services provided to its customers, as in order to compete in the market, the organization cannot only depend on the product quality, but also to the quality of services provided. Quality of service is important to encourage customers to repurchase products or to use services offered. Furthermore, service quality becomes something very important especially in dealing with customers who require a high level of excellence. Educational institutions such as STMIK MDP is demanded to become more capable of providing services to their customers. Customer satisfaction is an important goal in implementing IS/IT, thus it requires continuous monitoring on IS/IT performance. There are several measuring framework that can be used to assess the quality of service. One measuring framework that can be used is Servqual. Servqual is a multi-item scale that can be used to measure customer perceptions of service quality and its influence on
customer satisfaction. Five dimensions measured in servqual are tangibles, reliability, responsiveness, assurance and empathy (Zeithaml, 2004). This study aims to determine whether the use of the IS/IT can provide good quality of service at STMIK MDP. In addition, this study aims to measure the gap between expectation and performance of the IS/IT services in point of view the users. It is expected that this research can be used as the basis for decision making of STMIK MDP in improving its IS/IT services. RELATED WORKS Several studies related to the topic of this paper have been done before. In the area of education, Chua (2004) and Oliveira and Ferreira (2009) adopted servqual method to measure the quality of educational service. Chua (2004) used servqual method to measure the quality of educational services in the context of marketing, whereas Oliveira and Ferreira (2009) adopted a scale for measuring college service quality. Other research conducted by Zhai Huaiyuan and Tan Jie (2008), applied a modification of servqual thus allowing an institution to identify their competitive advantages. The research by Jiang (2006), Rianto (2007) and Fatoni (2009) showed that the servqual method can be used to measure service quality in information systems. Mehran Nejati, Mostafa Nejati, and Azadeh Shafaei (2007) used Servqual method to measure employee satisfaction and determined the gap that exists between employee expectations and company s performance. Stodnick and Rogers (2008) conducted a study to determine whether the scale of servqual can show predictive validity of the test and its relationship with students satisfaction. This study also investigated whether the scele of servqual free of bias level expectations. Singh and Khanduja (2010) used servqual to identify gaps between expectations and perceptions for selecting higher education of customers. In this study, we focus on servqual modification for measuring IS/IT services performance. This research is based on case study at STMIK MDP Palembang, a college that has provided IS/IT services to its customers. SERVQUAL Service Quality (Servqual) Model is a popular approach to measure service quality in marketing research. According to Parasuraman, et. al. (1985), there are five dimensions of servqual method used to measure the quality of services. The five dimensions are: Tangibles: the physical aspect that must exist in an IS/IT service process, eg. tools, equipments and facilities, as tangible evidence to assess the quality of service (Huaiyuan and Jie, 2008). Reliability: the ability of the IS/IT service to perform reliably and accurately (Huaiyuan and Jie, 2008). Responsiveness: the ability of the IS/IT service to respond to the wishes or needs of users, and provide a fast service (Rogers, 2008). Assurance: the ability of the IS/IT service to create a sense of trust and security to the user. Assurance includes the knowledge, competence, courtesy, respectful, and honesty of employees (Nakhai, 2008). Empathy: the ability of the IS/IT to take care and give attention to its users (Fatoni, 2009). The quality of service thus is measured by looking at the gap between quality of services that customers expected and the quality of services that customers perceived. In order measure the gap, we
need to develop a Servqual questionnaire. The Servqual questionnaire is divided into two parts. First part contains items that show the customers expectation on quality of services from each dimension, whereas the second part consists of the customers perception on the quality of services. Table 1 shows the original version of the Servqual questionnaire. Empathy Security Responsibility Reliability Tangibility Table 1. Original version of the Servqual scale Performance Expectation They should have modern equipment. The physical installations should be visually attractive. The employees should be well-dressed and clean. The appearance of company installations should be conserved according to the service offered. When these companies promise to do something in a certain time, they must do it. When clients have any problem with these companies, the latter must be solidary and make them feel secure. These companies should be of confidence. They should provide the service in the time promised. They should keep their records correctly. It should not be expected that they inform clients exactly when the services are to be executed. It is not reasonable to expect immediate availability of company employees. Company employees do not need to be always available to help clients. It is normal for them to be too busy to readily respond to requests. Clients should be able to believe in the company s employees. Clients should be able to feel safe in negotiating with company employees. The employees should be polite. The employees should obtain adequate support from the company to perform their tasks correctly It should not be expected for the companies to pay individual attention to the clients. It should not be expected for the employees to give personalized attention to the clients. It is absurd to expect the employees to know client needs. It is absurd to expect these companies to have the clients best interests as their objective. It should not be expected for the business hours to be convenient for all clients. Source: Olievera (2009) XYZ has modern equipment. XYZ s physical installations are visually attractive. XYZ s employees are well dressed and clean. The appearance of XYZ s physical installations is conserved according to the service offered. When XYZ promises to do something in a certain time, it really does it. When you have a problem with XYZ, it is solidary and makes you feel secure. XYZ can be trusted. XYZ provides the service in the time promised. XYZ keeps its records correctly. XYZ does not inform exactly when services will be executed. You do not receive immediate services from XYZ employees. XYZ employees are not always available to help clients. XYZ employees are always too busy to respond to client requests. You can believe XYZ employees. You feel secure negotiating with XYZ employees. XYZ employees are polite. XYZ employees do not obtain adequate support from the company to perform their tasks correctly. XYZ does not pay individual attention to you. XYZ employees do not give personal attention. XYZ employees do not know their needs. XYZ does not have your best interests as its objective. XYZ does not have convenient business hours for all clients
RESEARCH METHODOLOGY As we said in the introduction section, our research aims to measure the quality of IS/IT services in STMIK MDP. In order to that, we adapt the original version of Servqual, particularly for all items in each dimension to meet the nature of IS/IT services. Figure 1 illustrates the design of our research to measure the quality of IS/IT services in STMIK MDP. We use all Servqual dimensions to capture users expectation and what they feel on IS/IT services. By comparing both aspects, we can examine the IS/IT service quality gap that later can be used as information to improve the quality of IS/IT services in STMIK MDP. Dimensions on Servqual 1. Tangibles 2. Reliability 3. Responsiveness 4. Assurance 5. Empathy Expected Service Perceived Service Service Quality Gap Figure 1. Method and research variable The population of this study is all the users of IS/IT services in STMIK MDP, both students and lecturers. The total number of population is about 2500 people. From this population, samples are taken from 200 people, consisting of 170 students and 30 lecturers. The questionnaire consists of 56 questions which are divided into two parts. The first 28 questions are referring to respondents' expectation on IS/IT services (we refer to as variable X) and the last 28 questions are about the performance of IS/IT services which has been perceived by the respondents (we refer to as variable Y). These 56 questions given to respondents are developed by adapting Servqual model. Table 2 shows the modification that we have done to the Servqual model to measure the IS/IT services in STMIK MDP. Table 2. Modification of Servqual
Expectation Performance Tangible Reliability IS/IT facilities should provide a good quality of information. The software should up to date with the current technology. The IS/IT facilities should support the learning process. Application systems should be easy to operate. The number of PCs should be sufficient with the users need. The computer laboratory should be comfortable. Internet access should be fast. Connection hotspots should be easily accessible. Information provided through the website should be kept up to date. IS/IT services should be no disruption Expectation MDP already has IS/IT facilities that provide a good quality of information. MDP already has software with current technology. MDP already has IS/IT facilities that support learning processes. MDP already has application systems which are easy to operate. MDP provides sufficient number of PCs to its users. MDP already has comfortable computer laboratory. MDP already provides fast internet access. MDP already provides easily accessible hotspots. MDP already provides website with up to date information. MDP already provides IS/IT services with no disruption. Performance Responsiveness Assurance Online discussion forum should be provided in learning process. IS/IT applications should allow users to find information about lecture activities IS/IT applications should be able to help users to determine the feasibility to follow the exam. IS/IT applications should assist in the learning processes Instant messaging application should be able to assist in improving the quality of service. Staffs should handle complaints quickly. Staffs should provide IS/IT services kindly. IS/IT applications should require password IS/IT applications should be used by each user secretly. IS/IT applications should provide information accurately. IS/IT applications should be backed up data in case of loss or destruction of data MDP already provides online discussion for learning process. MDP already provides IS/IT applications that allows users to find information about lecture activities. MDP already provides IS/IT applications which are able to help users to determine the feasibility to follow the exam. MDP already provides applications which assist in the learning processes. MDP already provides instant messaging application which assists in improving the quality of service. MDP s staffs handle complaints quickly. MDP s staffs provide IS/IT services kindly. MDP already provides IS/IT applications that require password. MDP already facilitates IS/IT applications to be used by each user secretly. MDP already provides IS/IT applications that can produce information accurately. MDP already backs up data in case of loss or destruction of data.
Empathy There should be IS/IT application that provides reminder about exams. Input/output data of the applications should not complicate the user. IS/IT applications should give fast response time. IS/IT applications should be easily understood by the user. IS/IT applications should be provided in accordance with user needs. Staffs should understand the difficulties of the users in using the IS/IT services. Staffs should help users in using the IS/IT services. MDP already provides IS/IT application that provides reminder about exams. MDP already provides input/output data of the applications which are not complicate the users. MDP already provides IS/IT applications that give fast response time. MDP already provides IS/IT applications which are easily understood by the users. MDP already provides IS/IT applications which are suitable with user needs. MDP s staffs already understand the difficulties of the users in using the IS/IT services. MDP s staffs already help users in using the IS/IT services. In order to capture measure users expectation and perception on IS/IT services, we used questionnaire that contains the 56 questions according to the above items. The questionnaire was developed by using five points of Likert scale as can be seen in the Table 3. Expectation Strongly disagree : 1 Disagree : 2 Neutral : 3 Agree : 4 Strongly agree : 5 DISCUSSIONS AND ANALYSES Table 3. Likert Scale Used Performance Very dissatisfied : 1 Dissatisfied : 2 Neutral : 3 Satissfied : 4 Very Satissfied : 5 IS/IT Services on STMIK MDP STMIK MDP provides some IS/IT services to its users. For students, the services are mostly aimed to help students managing their study such as: online registration, online student performance (grade of each course and GPA), class schedule, online request letters and download the course material. For teacher, STMIK MDP already provides some applications to help them in managing courses they teach such as: arrange teaching schedule, recapitalize student attendance, give materials and tasks, initiate forum discussion, and input students mark. For administrative staffs, STMIK MDP provides IS/IT applications to fill the list of students, process the mark of a student, check students registration, input lecturers schedule, check student tuition payments and other operational activities. STMIK MDP also provides computer laboratory and hotspots to access internet. Each classroom is already equipped with computer, projector and smart board. Validity and Reliably Test Before we analyze our collected data, we tested our instrument to make sure that we already use a valid and reliable instrument. To calculate the validity of our instrument, we can search r value for each question. R value for 200 sample (df = 198) is 0.1388. If the value of r count (Corrected Item - Total Correlation (CI CT)) positive and larger than the value of r table, then the question item is considered valid. For reliability testing, we used Cronbach alpha value to measure each variable. According to
Sekaran (2006), if the alpha value is less than 0.6 then the reliability of instrument is poor, around the range of 0.7 the reliability of instrument still can be accepted, and more than 0.8 the reliability of instruments is quite good. Result of our calculation on the validity and reliability test shows that our instrument is valid and reliable. R values for all 56 questions are positive and greater than the value of r table, thus all items are valid. The Cronbach alpha values for all variables range from 0.6577 to 0.9259, thus it can be said that that our instrument is reliable. By examining these evidences, it can be concluded that our questionnaire is feasible to be used as an instrument for this study. Servqual Dimensions Analysis To analyze the gap between expectation and perception of users on IS/IT services in STMIK MDP, we used Paired Sample t-test. Each respondent is asked two questions, the first one is his/her expectation on IS/IT services, and the second one is his/her perception on IS/IT services performance which are already provided by STMIK MDP. Thus, the IT service quality can be described by comparing the responses on both questions. Significant value used in this test is 0.05. The formula of Paired Sample t-test is: D t = SD ( ) (1) N Where t refers to t value, D refers to average difference between measurements 1 and 2, SD refers to standard deviation measures of the difference between 1 and 2, and N refers to number of samples. Our calculation on Paired Sample t-test for all items can be seen in Table 4. Table 4. Result of research No Expectation Average Performance Average (P-E) 1. 4,77 3,505-1,265 2. 4,465 3,525-0,94 3. 3,8 3,425-0,375 4. 4,505 3,435-1,07 5. 4,66 3,295-1,365 6. 4,665 3,78-0,885 Rata-rata 4,478 3,494 Average Tangible -0,984 7. 4,725 2,73-1,995 8. 4,67 2,815-1,855 9. 4,635 3,495-1,14 10. 4,375 2,745-1,63 Rata-rata 4,601 2,946 Average Reliability -1,655 11. 4,28 3,07-1,21 12. 4,53 3,555-0,975 13. 4,335 3,555-0,78 14. 4,49 3,625-0,865 15. 4,205 3,105-1,1 16. 4,58 2,895-1,685 Reliability Responsiveness Tangible
17. 4,66 2,925-1,735 Average 4,44 3,247 Average Responsiveness -1,193 18. 4,28 3,825-0,455 19. 4,275 3,72-0,555 20. 4,555 3,515-1,04 21. 4,59 3,35-1,24 Average 4,425 3,603 Average Assurance -0,822 22. 4,385 3,605-0,78 23. 4,335 3,345-0,99 24. 4,34 3,085-1,255 25. 4,56 3,5-1,06 26. 4,545 3,49-1,055 27. 4,445 3,105-1,34 28. 4,375 3,185-1,19 Average 4,427 3,331 Average Emphaty -1,096 Assurance Emphaty According to statistical calculation results on Table 4, we can identify gap between expectation and perception on IS/IT services in STMIK MDP for each Servqual dimension. The followings are our analysis on the gap identified on each Servqual dimension. Tangible Based on the results of tabulation, it is known that the majority of respondents answered strongly agree for expectation, with average value 4.478. The average value obtained of performance perceived is 3.494, which means that all respondents "satisfied" with the services given. The difference between expectation and performance is -0.984. It means that the performance given by STMIK MDP does not meet the expectations of the users. From the Paired T-test on tangible dimension, result 0,001 for significant numbers. It means there is still a gap between expectations and performance in tangible dimension. To improve IS/IT services, STMIK MDP should improve IS/IT facilities that support learning processes, particularly in terms of number of PCs provided and quality of applications which are easy to and providing good quality of information. Reliability Based on the results of tabulation, it is known that the majority of respondents answered strongly agree for expectation, with average value 4.601. The average value obtained of performance perceived is 2.946, which means that all respondents "neutral" with the services given. The difference between expectation and performance is -1.655. It demonstrates that the performance was incompatible with the expectations of users. From the Paired T-test on reliability dimension, result 0,002 for significant numbers. It means there is a gap between expectations and performance in reliability dimension. In order to meet the quality expected form its users, STMIK MDP should improve the reliability dimension in all aspects which include: increasing the speed of internet access, providing easy hotspot connections, providing up to date information via website, and preventing disturbance when users use the IS/IT services. Responsiveness
Based on the results of tabulation, it is known that the majority of respondents answered strongly agree for expectation, with average value 4.44. The average value obtained of performance perceived is 3.247, which means that all respondents "neutral" with the services given. The difference between expectation and performance is -1.193. It means that the performance given by STMIK MDP does not meet the expectations of the users. From the Paired T-test on responsiveness dimension, result 0,033 for significant numbers. It means there is a gap between expectations and performance in responsiveness dimension. Among other indicators, STMIK MDP needs paying attention on the way they handle complaints form its user and the friendliness of its staffs when serving its users. Assurance Based on the results of tabulation, it is known that the majority of respondents answered strongly agree for expectation, with average value 4.425. The average value obtained of performance perceived is 3.603, which means that all respondents "satisfied" with the services given. The difference between expectation and performance is -0.822. It means that the performance given by STMIK MDP still does not meet the expectations of the users. From the Paired T-test on assurance dimension, result 0,009 for significant numbers. It means there is a gap between expectations and performance in assurance dimension. However, among other dimensions, the gap for assurance dimension is the lowest. Thus in order to improve the quality of IS/IT service in this dimension, STMIK MDP should provide more accurate information and give guarantee to its users that their data will be save in case the system crashed. Empathy Based on the results of tabulation, it is known that the majority of respondents answered strongly agree for expectation, with average value 4.427. The average value obtained of performance perceived is 3.331, which means that all respondents "neutral" with the services given. The difference between expectation and performance is -1.096. It means that the performance given by STMIK MDP also does not meet the expectations of the users. From the Paired T-test on empathy dimension, result 0,001 for significant numbers. It means there is a gap between expectations and performance in empathy dimension. Recommendation given to STMIK MDP to improve the empathy dimension are provide applications with simple input /output, improve the response time of applications, adapt the applications so it can be understood easily, and provide staffs who can solve user difficulties. CONCLUSION This research aims to measure the quality of IS/IT services in higher education as portrait the quality of IS/IT services provided by universities in Indonesia. We choose STMIK MDP as our case study, a college engaged in IS/IT field. It is expected that as a popular college engaged in IS/IT field in Palembang, STMIK MDP is more aware on IS/IT services offered to its users. We use modified Servqual method to measure the quality of IS/IT services in STMIK MDP that measures the quality of IS/IT services in five dimensions which are tangible, reliability, responsiveness, assurance and empathy. From our statistical analysis, it is found that all Servqual dimensions produce a gap between expectation and performance with the lowest and highest gap are on assurance and reliability dimension respectively. Therefore, STMIK MDP needs to make improvements to its service quality to increase user satisfaction. This result
also figures out the quality of IS/IT service of higher education in Indonesia, that mostly still needs improvements. REFFERENCE 1. Chua, Clare. (2004). Perception of Quality in Higher Education. Proceedings of the Australian University Quality Forum 2004. 2. Fathoni. (2009). Analisis Kualitas Layanan Sistem Informasi Menggunakan Metode Servqual. Konferensi Nasional Sistem dan Informatika, Bali. 3. Huaiyuan, Zhai., dan Jie, Tan. (2008). The Study of Instrument of Servqual Apllyingin Chinese Higher Education. 4. Jiang, James. J. et.al. (2006). Measuring Information System Service Quality: Servqual From The Other Side. 5. Nejati, Mehran. et.al. (2007). Using Servqual to Measure Employee Satisfaction An Iranian Case Study. Proceedings of the 13 th Asia Pacific Management Conference, Melbourne, Australia, 371-375. 6. Oliveira, Otavio Jose De., dan Ferrira, Euriane Christina. (2009). Adaptation and application of the Servqual scale in higher education. POMS 20 th Annual Conference Orlando, Florida USA. 7. Parasuraman, et.al., (1985). Problem and Strategis in Services in Strategis in Services Marketing. Journal of Marketing, April. pp.35-48. 8. Rahadi, Dedi Rianto. (2007). Peranan Teknologi Informasi Dalam Peningkatan Pelayanan Di Sektor Publik. Seminar Nasional Teknologi, Yogyakarta. 9. Sekaran, U. (2006). Research Methods For Business. Edisi Terjemahan, Edisi Keempat. Salemba Empat. 10. Singh, Rajdeep., dan Khanduja, Dinesh. (2010). Servqual and Model of Service Quality Gaps : Frameworks for Determining and Prioritizing Critical Factors from Faculty Perspective in Higher Education. International Journal of Engineering Science and Technology Vol. 2 (7). 11. Stodnick, Michael., and Rogers, Pamela. (2008). Using Servqual to Measure thr Quality of the Classroom Experience. Decision Sciences Journal of Innovative Education Vol. 6 No.1. 12. Stodnick, Michael., dan Rogers, Pamela. (2008). Using Servqual to Measure thr Quality of the Classroom Experience. Decision Sciences Journal of Innovative Education Vol. 6 No.1. 13. Sumarno. (2003). Faktor-faktor yang dipertimbangkan tamu menginap di hotel (studi kasus tamu Hotel NCS Surabaya). Jurnal Penelitian Ilmu Ekonomi UPN Veteran Jawa Timur 3(1): 20-27. 14. Zeithaml, Valarei.A., dan Mary jo Bitner. (2004). Service Marketing: Integrating Customer Focus Across The Fim. 3rd Ed., Mc GrawHill. New York. AUTHOR BIOGRAPHIES Ms. Lisa Amelia obtained her master degree in Information Technology from Universitas Indonesia. Currently she is working at STMIK MDP Palembang. Her research interests are related to service management, information systems and information technology. Dr. Achmad Nizar Hidayanto is Head of Information Systems/Information Technology Stream, Faculty of Computer Science, Universitas Indonesia. He received his PhD in Computer Science from Universitas Indonesia. His research interests are related to information systems/information technology, e-learning, information systems security, change management, distributed systems and information retrieval.
Ms. Ika Chandra Hapsari is an undergraduate student in Faculty of Computer Science Universitas Indonesia majoring at Information Systems. She has made some publications during her study. Her research interests are related to information systems, change management and data mining.