Measuring service quality in higher education: HEdPERF versus SERVPERF Firdaus Abdullah MARA University of Technology, Jalan Meranek, Malaysia

Transcription

1 The current issue and full text archive of this journal is available at : HEdPERF versus SERVPERF Firdaus Abdullah MARA University of Technology, Jalan Meranek, Malaysia Abstract Purpose This paper aims to test and compare the relative efficacy of three measuring instruments of service quality (namely Higher Education PERFormance (HEdPERF), SERVPERF and the moderating scale of HEdPERF-SERVPERF) within a setting. The objective was to determine which instrument had the superior measuring capability in terms of unidimensionality, reliability, validity and explained variance. Design/methodology/approach After a pilot test, data were collected from students in two public universities, one private university and three private colleges in Malaysia between January and March 2004, by the contact person route. From a total of 560 questionnaires, 381 were usable: a response rate of 68.0 per cent. This sample of nearly 400,000 students in Malaysian tertiary institutions was in line with the generalized scientific guideline for sample size decisions. Data were subjected to regression analysis. Findings A modified five-factor structure of HEdPERF is put forward as the most appropriate scale for the sector. Research limitations/implications Since this study only examined the respective utilities of each instrument within a single industry, any suggestion that the HEdPERF is generally superior would still be premature. Nonetheless, the current findings do provide some important insights into how these instruments of service quality compare with one another. Practical implications The single dominant factor on this study is access, which has clear implications for institutions marketing strategies. Originality/value This is believed to be the first study of its kind carried out among consumers of the service. Keywords Service quality assurance, Higher education, instruments Paper type Research paper 31 Received May 2004 Revised October 2005 Accepted October 2005 Introduction Service industries are playing an increasingly important role in the economy of many nations. In today s world of global competition, rendering quality service is a key for success, and many experts concur that the most powerful competitive trend currently shaping marketing and business strategy is service quality. Since the 1980s service quality has been linked with increased profitability, and it is seen as providing an important competitive advantage by generating repeat sales, positive word-of-mouth feedback, customer loyalty and competitive product differentiation. As Zeithaml and Bitner (1996, p. 76) point out:...the issue of highest priority today involves understanding the impact of service quality on profit and other financial outcomes of the organisation. Marketing Intelligence & Planning Vol. 24 No. 1, 2006 pp q Emerald Group Publishing Limited DOI /

2 MIP 24,1 32 Service quality has since emerged as a pervasive strategic force and a key strategic issue on management s agenda. It is no surprise that practitioners and academics alike are keen on accurately measuring order better to understand its essential antecedents and consequences, and ultimately, establish methods for improving quality to achieve competitive advantage and build customer loyalty. The pressures driving successful organisations toward top quality services make the measurement of service quality and its subsequent management of utmost importance. Interest in the measurement of service quality is thus understandably high. However, the problem inherent in the implementation of such a strategy has been compounded by the elusive nature of service quality construct, rendering it extremely difficult to define and measure. Although researchers have devoted a great deal of attention to service quality, there are still some unresolved issues that need to be addressed, and the most controversial one refers to the measurement instrument. An attempt to define the evaluation standard independent of any particular service context has stimulated the setting up of several methodologies. In the last decade, the emergence of diverse instruments of measurement such as SERVQUAL (Parasuraman et al., 1988), SERVPERF (Cronin and Taylor, 1992) and evaluated performance (EP) (Teas, 1993a, b) has contributed enormously to the development in the study of service quality. SERVQUAL operationalises service quality by comparing the perceptions of the service received with expectations, while SERVPERF maintains only the perceptions of service quality. On the other hand, EP scale measures the gap between perceived performance and the ideal amount of a feature rather than the customer s expectations. Diverse studies using these scales have demonstrated the existence of difficulties resulting from the conceptual or theoretical component as much as from the empirical component. Nevertheless, many authors concur that customers assessments of continuously provided services may depend solely on performance, thereby suggesting that performance-based measure explains more of the variance in an overall measure of service quality (Oliver, 1989; Bolton and Drew, 1991a, b; Cronin and Taylor, 1992; Boulding et al., 1993; Quester et al., 1995). These findings are consistent with other research that have compared these methods in the scope of service activities, thus confirming that SERVPERF (performance-only) results in more reliable estimations, greater convergent and discriminant validity, greater explained variance, and consequently less bias than the SERVQUAL and EP scales (Cronin and Taylor, 1992; Parasuraman et al., 1994; Quester et al., 1995; Llusar and Zornoza, 2000). Whilst its impact in the service quality domain is undeniable, SERVPERF being a generic measure of service quality may not be a totally adequate instrument by which to assess the perceived quality in. Nowadays, is being driven towards commercial competition imposed by economic forces resulting from the development of global education markets and the reduction of government funds that forces tertiary institutions to seek other financial sources. Tertiary institutions had to be concerned with not only what the society values in the skills and abilities of their graduates (Ginsberg, 1991; Lawson, 1992), but also how their students feel about their educational experience (Bemowski, 1991). These new perspectives call attention to the management processes within the institutions as an alternative to the traditional areas of academic standards, accreditation and performance indicators of teaching and research.

3 Tertiary educators are being called to account for the quality of education that they provide. While more accountability in tertiary education is probably desirable, the mechanisms for its achievement are being hotly debated. Hattie (1990) and Soutar and McNeil (1996) oppose the current system of centralised control, in which the government sets up a number of performance indicators that are linked to funding decisions. There are a number of problems in developing performance indicators in tertiary education. One such problem is that performance indicators tend to become measures of activity rather than true measures of the quality of students educational service (Soutar and McNeil, 1996). These performance indicators may have something to do with the provision of tertiary education, but they certainly fail to measure the quality of education provided in any comprehensive way. A survey conducted by Owlia and Aspinwall (1997) examined the views of different professionals and practitioners on the quality in and concluded that customer-orientation in is a generally accepted principle. They construed that from the different customers of, students were given the highest rank. Student experience in a tertiary education institution should be a key issue of which performance indicators need to address. Thus it becomes important to identify determinants or critical factors of service quality from the standpoint of students being the primary customers. In view of that, Firdaus (2005) proposed HEdPERF (Higher Education PERFormance-only), a new and more comprehensive performance-based measuring scale that attempts to capture the authentic determinants of service quality within higher education sector. The 41-item instrument has been empirically tested for unidimensionality, reliability and validity using both exploratory and confirmatory factor analysis. Therefore, the primary question is directed at the measurement of service quality construct within a single, empirical study utilising customers of a single industry, namely. Specifically, the ability of the more concise HEdPERF scale is compared with that of two alternatives namely SERVPERF instrument and the merged HEdPERF-SERVPERF as moderating scale. The goal is to assess the relative strengths and weaknesses of each instrument in order to determine which instrument had the superior measurement capability in terms of unidimensionality, reliability, validity and explained variance of service quality. Eventually, the results of this comparative study were used to refine the HEdPERF scale, transforming it into an ideal measuring instrument of service quality for sector. Research foundations Many researchers (Parasuraman et al., 1985; Carman, 1990; Bolton and Drew, 1991a, b) concur that service quality is an elusive concept, and there is considerable debate about how best to conceptualise this phenomenon. Lewis and Booms (1983, p. 100) were perhaps the first to define service quality as a...measure of how well the service level delivered matches the customer s expectations. Thereafter, there seems to be a broad consensus that service quality is an attitude of overall judgement about service superiority, although the exact nature of this attitude is still hazy. Some suggest that it stems from a comparison of performance perceptions with expectations (Parasuraman et al., 1988), while others argue that it is derived from a comparison of performance with ideal standards (Teas, 1993a, b) or from perceptions of performance alone (Cronin and Taylor, 1992). 33

4 MIP 24,1 34 In terms of measurement methodologies, a review of literature provides plenty of service quality evaluation scales. Some stem from the realisation of conceptual models produced to understand the evaluation process (Parasuraman et al., 1985), and others come from empirical analysis and experimentation on different service sectors (Cronin and Taylor, 1992; Franceschini and Rossetto, 1997b; Parasuraman et al., 1988). The most widely used methods applied to measure perceived quality can be characterised as primarily quantitative multi-attribute measurements. Within the attribute-based methods, a great number of variants exist and among these variants, the SERVQUAL and SERVPERF instruments have attracted the greatest attention. Generally, most researchers acknowledge that customers have expectations and these serve as standards or reference points to evaluate the performance of an organisation. However, the unresolved issues of expectations as a determinant of perceived service quality have resulted in two conflicting measurement paradigms: the disconfirmation paradigm (SERVQUAL) which compares the perceptions of the service received with expectations, and the perception paradigm (SERVPERF) which maintains only the perceptions of service quality. These instruments share the same concept of perceived quality. The main difference between these scales lies in the formulation adopted for their calculation, and more concretely, the utilisation of expectations and the type of expectations that should be used. Most research studies do not support the five-factor structure of SERVQUAL posited by Parasuraman et al. (1988), and administering expectation items is also considered unnecessary (Carman, 1990; Parasuraman et al., 1991a, b; Babakus and Boller, 1992). Cronin and Taylor (1992) were particularly vociferous in their critiques, thus developing their own performance-based measure, dubbed SERVPERF. In fact, the SERVPERF scale is the unweighted perceptions components of SERVQUAL, which consists of 22 perception items thus excluding any consideration of expectations. In their empirical work in four industries, Cronin and Taylor (1992) found that unweighted SERVPERF measure (performance-only) performs better that any other measure of service quality, and that it has greater predictive power (ability to provide an accurate service quality score) than SERVQUAL. They argue that current performance best reflects a customer s perception of service quality, and that expectations are not part of this concept. Likewise, Boulding et al. (1993) reject the value of an expectations-based SERVQUAL, and concur that service quality is only influenced by perceptions. Quester et al. (1995) perform similar analysis to Cronin and Taylor in the Australian advertising industry, and their empirical tests show that SERVPERF performs best, while SERVQUAL performs worst, although the differences are small. Teas (1993a) on the other hand, discusses the conceptual and operational difficulties of using the expectations minus performance approach, with a particular emphasis on expectations. His empirical test subsequently produces two alternatives of perceived service quality measures namely EP and normed quality (NQ). He concludes that the EP instrument, which measures the gap between perceived performance and the ideal amount of a feature rather than the customer s expectations, outperforms both SERVQUAL and NQ. A review of service quality literature brings forward diverse arguments in relation to the advantages and disadvantages in the use of these instruments. In general, the arguments make reference to aspects related to the characteristics of these

5 scales notably their reliability and validity. Recently, Llusar and Zornoza (2000) concur that SERVPERF results in more reliable estimations, greater convergent and discriminant validity, greater explained variance, and consequently less bias than the EP scale. These results are consistent with earlier research that had compared these methods in the scope of service activities (Cronin and Taylor, 1992; Parasuraman et al., 1994). In fact, the marketing literature appears to offer considerable support for the superiority of simple performance-based measures of service quality (Mazis et al., 1975; Churchill and Surprenant, 1982; Carman, 1990; Bolton and Drew, 1991a, b; Boulding et al., 1993; Teas, 1993a; Quester et al., 1995). Research methodology Research objectives On the basis of the conceptual and operational concerns associated with the generic measures of service quality, the present research attempts to compare and contrast empirically the HEdPERF scale against two alternatives namely the SERVPERF and the merged HEdPERF-SERVPERF scales. The primary goal is to assess the relative strengths and weaknesses of each instrument in order to determine which instrument had the superior measurement capability in terms of unidimensionality, reliability, validity and explained variance of service quality. The findings were eventually used in transforming HEdPERF into an ideal measuring instrument of service quality for sector. The various steps involved in this comparative study are shown by means of flow chart in Figure 1. Research design Data were collected by means of a structured questionnaire comprising of four sections namely A, B, C and D. Section A contained nine questions pertaining to student respondent profile. While sections B and C required respondents to evaluate the service components of their tertiary institution, in which only perceptions data were collected and analysed. Specifically, section B consisted of 22 perception-items extracted from the original SERVPERF scale (Cronin and Taylor, 1992), and modified to fit into higher education context. Section C on the other hand is composed of 41 items extracted from the original HEdPERF (Firdaus, 2005), a scale uniquely developed to embrace different aspects of tertiary institution s service offering. As the items were generated and validated within context, no modification was required. All the items in sections B and C were presented as statements on the questionnaire, with the same rating scale used throughout, and measured on a 7-point, Likert-type scale that varied from 1 ¼ strongly disagree to 7 ¼ strongly agree. In addition to the main scale addressing individual items, respondents were asked in section D to provide an overall rating of the service quality, satisfaction level and future visits. There were also three open-ended questions allowing respondents to give their personal views on how any aspect of the service could be improved. The draft questionnaire was eventually subjected to pilot testing with a total of 30 students, and they were asked to comment on any perceived ambiguities, omissions or errors concerning the draft questionnaire. The feedback received was rather ambiguous thus only minor changes were made accordingly, for instance, technical jargons were rephrased to ensure clarity and simplicity. The revised questionnaire was 35

6 MIP 24,1 36 Figure 1. Comparing HEdPERF, SERVPERF and HEdPERF-SERVPERF

7 subsequently submitted to three experts (an academician, a researcher and a practitioner) for feedback before being administered for a full-scale survey. These experts viewed that the draft questionnaire was rather lengthy, which in fact coincided with the preliminary feedback from students. Nevertheless, in terms of number of items in the questionnaire, current study somewhat conforms with similar research works (Cronin and Taylor, 1992; Teas, 1993a, b; Lassar et al., 2000; Mehta et al., 2000; Robledo, 2001) that attempted to compare various measuring instruments of service quality. In the subsequent full-scale survey, data were collected from students of six higher learning institutions (two public universities, one private university and three private colleges) in Malaysia for the period between January and March Data had been collected using the personal-contact approach as suggested by Sureshchandar et al. (2002) whereby contact persons (registrar or assistant registrar) have been approached personally, and the survey explained in detail. The final questionnaire together with a cover letter was then handed personally or mailed to the contact persons, who in turn distributed it randomly to students within their respective institutions. A total of 560 questionnaires were distributed to six tertiary institutions, of these 390 were returned and nine discarded due to incomplete responses, thus leading to a response rate of 68.0 per cent. The number of usable sample size of 381 for a population size of nearly 400,000 students in Malaysian tertiary institutions was in line with the generalized scientific guideline for sample size decisions as proposed by Krejcie and Morgan (1970). In order to determine which instrument had the superior measurement capability, a new scale was developed by merging the two measuring instruments of HEdPERF and SERVPERF. The scope of the empirical investigation notably the methodology utilised for the development of the merged scale was defined. Next, the results obtained from the three instruments were computed, comparing them based on the widely-used criteria of unidimensionality, reliability, validity and their ability to predict service quality. The results of this comparative study were subsequently used to refine the HEdPERF scale, transforming it into an ideal measuring instrument of service quality for sector. Results and discussion Developing the merged HEdPERF-SERVPERF scale The literature appears to offer considerable support for the superiority of SERVPERF in comparison with other generic instruments. HEdPERF on the other hand, has been empirically tested as the more comprehensive and industry-specific scale, which was uniquely designed for. Thus, it would seem rational to combine the two finest scales in developing possibly the pre-eminent one, and subsequently to determine which of these three instruments had the superior measurement capability in terms of unidimensionality, reliability, validity and explained variance of service quality. In developing the new scale, factor analysis was used to determine a new dimensional structure of service quality by merging HEdPERF and SERVPERF items. Specifically, this technique allowed reduction of a large number of overlapping variables to a much smaller set of factors. One critical assumption underlying the appropriateness of factor analysis is to assess the overall significance of the correlation matrix with Bartlett test of sphericity, which provides the statistical probability that the correlation matrix has significant correlations among at least some of the variables. The results were significant, 37

8 MIP 24,1 38 x 2 ð50; N ¼ 381Þ ¼13; 073; ( p ¼ 0.01), a clear indication of suitability for factor analysis. Next, Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was computed to quantify the degree of intercorrelations among the variables, and the results indicate an index of 0.89, a meritorious sign of adequacy for factor analysis (Kaiser, 1970). As for the adequacy of the sample size, there is a 7-1 ratio of observations to variables in this study, which falls within acceptable limits. HEdPERF s proposed measure of service quality was a 41-item scale, consisting of 13 items adapted from SERVPERF, and 28 items generated from literature review and various qualitative research inputs namely focus groups, pilot test and expert validation (Firdaus, 2005). The 41-item instrument has been empirically tested for unidimensionality, reliability and validity using both exploratory and confirmatory factor analysis. Therefore, all the 50 items (28 HEdPERF s and 22 SERVPERF s items) were included in the factor analysis utilising the maximum likelihood method, which was followed by a varimax rotation. The decision to include a variable in a factor was based on factor loadings greater than ^0.3 (Hair et al., 1995). The choice regarding factor loadings greater than ^0.3 was not based on any mathematical proposition but relates more to practical significance. According to Hair et al. (1995, p. 385), factor loadings of 0.3 and above are considered significant at p ¼ 0.05 with a sample size of 350 respondents (N ¼ 381 in this study). Scree test was used to identify the optimum number of factors that can be extracted before the amount of unique variance begins to dominate the common variance structure (Cattell, 1966). The scree plot provided four factors, and these factors were subsequently rotated using a varimax procedure. The variable s communality, which represents the amount of variance accounted for by the factor solution for each variable was also assessed to ensure acceptable levels of explanation. The results show that communalities in 13 variables were below 0.50,...too low for having sufficient explanation (Hair et al., 1995, p. 387). Consequently, a new factor solution was derived with the non-loading variables eliminated and the results yielded four factors, which accounted for 41.0 per cent of the variation in the data (compared to 37.2 per cent of the variance explained in the first factor solution). Table I shows the results of the factor analysis in terms of factor name, the variables loading on each factor and the variance explained by each factor. The four factors identified in Table I can be described as follows: (1) Non-academic aspects. This factor contains variables that are essential to enable students fulfil their study obligations, and it relates to duties and responsibilities carried out by non-academic staff. (2) Academic aspects. This factor represents the responsibilities of academics, and it highlights key attributes such as having positive attitude, good communication skill, allowing sufficient consultation, and being able to provide regular feedback to students. (3) Reliability. This factor consists of items that put emphasis on the ability to provide the pledged service on time, accurately and dependably. (4) Empathy. This factor relates to the provision of individualized and personalized attention to students with clear understanding of their specific and growing needs while keeping their best interest at heart. It is important to note that the four factors identified did not conform exactly with neither the six-factor structure of HEdPERF nor the five-factor structure of

9 Variables Factor 1: non-academic aspects Factor 2: academic aspects Factor 3: reliability Factor 4: empathy Promises kept 0.65 Sympathetic and reassuring in solving problems Dependability 0.52 On-time service provision 0.74 Responding to request promptly 0.51 Trust 0.44 Feeling secured with the transaction Politeness Individualised attention 0.68 Giving personalized attention 0.74 Knowing student needs 0.66 Keeping student interests at heart 0.55 Knowledge in course content Showing positive attitude 0.66 Good communication 0.75 Feedback on progress 0.68 Sufficient and convenient consultation time Excellent quality programmes 0.62 Variety of programmes/specialisations 0.43 Flexible syllabus and structure Reputable academic programmes Educated and experience academicians 0.50 Efficient/prompt dealing with complaints 0.51 Good communication Positive work attitude Knowledge of systems/procedures Providing service within reasonable time 0.51 Equal treatment and respect 0.75 Fair amount of freedom 0.56 Confidentiality of information 0.65 Easily contacted by telephone 0.57 Counselling services Student s union 0.33 Feedback to improve service performance Standardised and simple delivery procedures Eigenvalues Percentage of variance Cummulative percentage of variance Table I. Results of factor analysis (factor loadings) SERVPERF. In fact, the new dimensions extracted were the result of the amalgamation between HEdPERF and SERVPERF scales, in which two factors (non-academic aspects and academic aspects) were found in HEdPERF and the other two (reliability and empathy) were identified in SERVPERF. Comparative test of unidimensionality A highly mandatory condition for construct validity and reliability checking is the unidimensionality of the measure, which is referred to the existence of a single construct/trait underlying a set of measures (Hattie, 1985; Anderson and Gerbing, 1991).

10 MIP 24,1 40 In order to perform a comparative check of unidimensionality, a measurement model is specified for each scale and confirmatory factor analysis is run by means of structural equation modelling within LISREL framework (Joreskog and Sorbom, 1978). Specifically, LISREL 8.3 (Scientific Software International, Chicago, IL) for windows was used to compare the three measuring instruments where individual items in the model are examined to see how closely they represent the same concept. Table II presents the measures of model fit for all the three scales. The overall fit of the model to the data was evaluated in various ways. Specifically, an exact fit of a model is indicated when the p for chi-square (x 2 ) is above a certain value (usually set to p. 0.05) as well as indicated by other goodness-of-fit measures. While chi-square is sensitive to sample size and tends to be significant in large samples, a relative likelihood ratio between a chi-square and its degrees of freedom was used. According to Eisen et al. (1999), a relative likelihood ratio of five or less was considered an acceptable fit, a prerequisite attained by all the three scales. A number of goodness-of-fit measures were proposed to eliminate or reduce the dependence on sample size. These indices have values ranging between 0 and 1, with higher values indicating a better fit. Table II shows the indices for the three scales and all the values are close to 1, indicating that there is an evidence of unidimensionality for the scales (Bryne, 1994). The next measure to consider is the root mean square error of approximation (RMSEA), which is the measure of the discrepancy per degree of freedom. The RMSEA is generally regarded as one of the most informative fit indices (Brown and Cudeck, 1993; Diamantopoulos and Siguaw, 2000). As illustrated in Table II, the RMSEA value for the HEdPERF was 0.07 (dimension Understanding removed), an evidence of fair fit to the data. While both the SERVPERF and the moderating HEdPERF-SERVPERF scales showed a poor fit of Therefore, it was concluded that the modified HEdPERF model fits fairly and represents a reasonably close approximation in the population. Nevertheless, Bryne (1998, p. 119) cautions that:... fit indices provide no guarantee whatsoever that a model is useful... Fit indices yield information bearing only on the model s lack of fit...they can in no way reflect the extent to which the model is plausible; this judgement rests squarely on the shoulders of the researcher. Measures of fit HEdPERF a SERVPERF HEdPERF-SERVPERF Table II. Unidimensionality check Chi-square (x 2 )atp ¼ Degree of freedom (df) Relative likelihood ratio (x 2 /df) Goodness-of-fit index (GFI) Adjusted goodness-of-fit index (AGFI) Comparative fit index (CFI) Non-normed fit index (NNFI) Incremental fit index (IFI) Root mean squared error of approximation (RMSEA) Note: a The overall fit assessment was computed with the exclusion of dimension understanding due to its poor fit, RMSEA ¼ 0.08

11 Comparative test of reliability Unidimensionality alone is not sufficient to ensure the usefulness of a scale. The reliability of the composite score should be assessed after unidimensionality has been acceptably established. In this study coefficient a or Cronbach s a was computed for the service quality dimensions of all the three instruments. As a guideline, an a-value of 0.70 and above is considered to be the criteria for demonstrating internal consistency of new scales and established scales, respectively (Nunnally, 1988). In general, there was relatively good internal consistency in all the dimensions representing the three scales. The Cronbach s a for HEdPERF dimensions ranged from 0.81 to 0.92, with the exception of the dimension understanding (a ¼ 0.63). Owing to its low reliability score, the dimension understanding was removed as part of the scale modification process. The results, as shown in Table III, indicated that the reliability scores of the modified HEdPERF were comparatively superior than SERVPERF s range of , and slightly better than HEdPERF-SERVPERF s range of In fact, previous replication studies (Carman, 1990; Babakus and Boller, 1992; Finn and Lamb, 1991) that compared various measuring instruments within a single setting were unable to demonstrate such superiority. Comparative test of validity While internal consistency estimates of reliability show higher values for the modified HEdPERF scale, the next comparative test involves assessing the validity of the three instruments. For the purpose of this study, two validity tests were conducted namely criterion validity and construct validity. The criterion variable used to compare the three scales was the respondents global assessment of service quality (Bollen, 1989, p. 186). Whereas the variable used in the construct validity tests was the global preference, which was measured by the summation of the overall satisfaction and future visit intentions (Bollen, 1989, p. 188). The degree of criterion and construct validity was subsequently computed using the pairwise correlations between the global assessment of service quality, the global preference and each of HEdPERF, SERVPERF and SERVPERF-HEdPERF measures. The validity coefficients for the three scales are all significant at p ¼ 0.01 level. The criterion and construct validity coefficients were 0.58 and 0.57, respectively, for the modified HEdPERF scale, 0.27 and 0.34, respectively, for SERVPERF scale, and 0.53 and 0.57, respectively, for the moderating SERVPERF-HEdPERF scale. The results 41 HEdPERF dimensions Cronbach alpha (a) SERVPERF dimensions Cronbach alpha (a) HEdPERF-SERVPERF dimensions Cronbach alpha (a) Non-academic aspects 0.92 Responsiveness 0.68 Non-academic aspects 0.91 Academic aspects 0.89 Assurance 0.75 Academic aspects 0.87 Reputation 0.85 Empathy 0.76 Reliability 0.88 Access 0.88 Tangible 0.73 Empathy 0.77 Programme issues 0.81 Reliability 0.74 Understanding a 0.63 Note: a Dimension discarded Table III. Reliability coefficients

12 MIP 24,1 42 indicated that the validity coefficients for the modified HEdPERF scale were significantly greater than both the SERVPERF and SERVPERF-HEdPERF scales. In other words, these findings demonstrated yet again the superiority of the modified HEdPERF in terms of criterion and construct validity. Comparative regression analysis The effect size. The regression model considered the global assessment of service quality as a dependent variable and the service quality scores for the individual dimensions of HEdPERF, SERVPERF and SERVPERF-HEdPERF as the independent variables. A multiple regression analysis was subsequently conducted to evaluate how well these scales predicted service quality level (Table IV). The linear combination of the modified five-dimension HEdPERF (dimension understanding dropped) was significantly related to the service quality level, R 2 ¼ 0:35; adjusted R 2 ¼ 0:34; Fð5; 354Þ ¼ 38:4; p ¼ 0:01: The sample multiple correlation coefficient was 0.59, indicating that approximately 34.8 per cent of the variance of the service quality level in the sample can be accounted for by the linear combination of the five dimensions of HEdPERF scale. Likewise, the five dimensions of SERVPERF was also significantly related to the service quality level, R 2 ¼ 0:24; adjusted R 2 ¼ 0:23; Fð5; 354Þ ¼22:1; p ¼ 0:01: However, the sample multiple correlation coefficient of 0.49 was much lower than HEdPERF s, thus indicating that only 24.0 per cent of the variance of the service quality level can be accounted for by the linear combination of the five dimensions of SERVPERF scale. As for the moderating SERVPERF-HEdPERF scale, the linear combination of the four dimensions was also significantly related to the service quality level, R 2 ¼ 0:31; adjusted R 2 ¼ 0:30; Fð4; 355Þ ¼28:92; p ¼ The sample multiple correlation coefficient was 0.55, slightly lower than HEdPERF s, thus indicating that approximately 30.5 per cent of the variance of service quality level can scales Standardised coefficients (b) Significant ( p) Table IV. Effect size and relative importance of the individual dimensions HEdPERF a (R 2 ¼ 0.34) Non-academic aspects Academic aspects Reputation Access Programme issues SERVPERF (R 2 ¼ 0.23) Responsiveness Assurance Empathy Tangible Reliability HEdPERF-SERVPERF (R 2 ¼ 0.30) Non-academic aspects Academic aspects Reliability Empathy Note: a The modified version with dimension understanding removed

13 be explained by the four dimensions. The findings from the regression analysis once again demonstrated that the modified HEdPERF scale was better in explaining the variance of service quality level. The relative influence. Table IV shows the results of the relative influence of the individual dimensions of the three scales. The dependent variable was the global assessment of service quality level. The resultant output of HEdPERF had an adjusted R 2 of 0.34 ( p ¼ 0.01) and yielded only one significant dimension namely Access, which concurred with the findings by Firdaus (2005). It alone accounted for 13.7 per cent ð0:37 2 ¼ 0:14Þ of the variance of service quality level, while the other dimensions contribute an additional 21.1 per cent ( per cent). This implied that the dimensions Non-academic aspects, Academic aspects, Reputation, and Program issues did not contribute significantly towards explaining the variance in the overall rating. SERVPERF scale on the other hand had an adjusted R 2 of 0.23 ( p ¼ 0.01) and yielded two significant dimensions namely Tangible and Assurance, which explained almost all the variance in the service quality level. The other dimensions namely Responsiveness, Reliability and Empathy did not contribute significantly towards explaining the variance in the overall rating. As for the SERVPERF-HEdPERF scale, the adjusted R 2 was 0.30, and yielding two significant dimensions namely non-academic aspects and academic aspects. The two dimensions accounted for 23.0 per cent ((0.26 þ 0.22) 2 ¼ 0.23) of the variance of service quality level, while the other two dimensions contribute an additional 7.5 per cent ( per cent). This implied that the dimensions reliability and empathy did not contribute significantly towards explaining the variance in the overall rating. The modified HEdPERF scale The empirical analysis indicated that the modified five-factor structure with 38 items resulted in more reliable estimations, greater criterion and construct validity, greater explained variance, and consequently a better fit (Table V). Besides the better quantitative results, the modified HEdPERF scale also had the advantage of being more specific in areas that are important in evaluating service quality within higher education sector. Hence, can be considered as a five-factor structure with conceptually clear and distinct dimensions namely non-academic aspects, academic aspects, reputation, access and programme issues. The current study also showed that SERVPERF performed miserably. Although SERVPERF was developed and subsequently proven as the superior generic scale to measure wide-ranging service industry, it did not provide a better perspective for the setting. 43 Criteria Modified HEdPERF SERVPERF HEdPERF-SERVPERF Cronbach alpha (a) range Criterion validity Construct validity Adjusted R RMSEA Table V. Comparison of the three scales

14 MIP 24,1 44 Conclusions and implications The objectives of this research were twofold. The primary goal was to test and compare the relative efficacy of the three conceptualisations of order to determine which instrument had the superior measurement capability in terms of unidimensionality, reliability, validity and explained variance. The other objective is concerned with enhancing the HEdPERF scale, thus transforming it into an ideal measuring instrument of service quality for sector. The tests were conducted utilizing students sample from Malaysian tertiary institutions, and the findings indicated that the three measuring scales did not perform equivalently in this particular setting. In fact, the results led us to conclude that the measurement of service quality by means of the HEdPERF method resulted in more reliable estimations, greater criterion and construct validity, greater explained variance, and consequently better fit than the other two instruments namely SERVPERF and HEdPERF-SERVPERF. In short, the findings demonstrated an apparent superiority of the modified five-factor structure of HEdPERF scale. In what may be the first of its kind within setting, the regression analyses compared the three scales so as to determine how well they predicted service quality level. Although the modified five-factor structure of HEdPERF clearly outperformed the SERVPERF and the moderating HEdPERF-SERVPERF dimensions in terms of explaining the variances in service quality level, the implications of these findings are less clear. Since this study only examined the respective utilities of each instrument within a single industry, any suggestion that the HEdPERF is generally superior would still be premature. Nonetheless, the current findings do provide some important insights into how these instruments of service quality compare with one another. The current results also suggest that the dimension access is the most important determinant of, thus reinforcing the recommendation made by Firdaus (2005). In other words, students perceived access to be more important than other dimensions in determining the quality of service they received. As the only HEdPERF dimension to achieve significance, access is concerned with such elements as approachability, ease of contact and availability of both the academics and non-academics staff (Firdaus, 2005). Tertiary institutions should, therefore, concentrate their efforts on the dimension perceived to be important rather than focusing their energies on a number of different attributes, which they feel are important determinants of service quality. While the idea of providing adequate service on all dimensions may seem attractive to most service marketers and managers, failure to prioritise these attributes may result in inefficient allocation of resources. In conclusion, the current study provides empirical support in favour of the idea that the modified five-factor structure of HEdPERF with 38 items may be the superior instrument in measuring service quality within setting. Limitations and suggestions for future research The current study allows us to understand how three measuring instruments of service quality compare to one another. To date, these scales have not been compared and contrasted empirically thus making this research a unique contribution to the services marketing literature. However, this work may pose more questions than it provides

15 answers. The present findings suggest that the modified HEdPERF scale is better suited in service settings. Caution is necessary in generalizing the findings although considerable evidence of relative efficacy was found in the modified HEdPERF scale. Given that the current study is limited to one service industry, this assertion would need to be validated by further research. Future studies should apply the measurement instrument in other countries, in other industries, and with different types of tertiary institutions in order to test whether the results obtained are general and consistent across different samples. References Anderson, J.C. and Gerbing, D.W. (1991), Predicting the performance of measures in a confirmatory factor analysis with a pretest assessment of their substantive validities, Journal of Applied Psychology, Vol. 76 No. 5, pp Babakus, E. and Boller, G.W. (1992), An empirical assessment of the SERVQUAL scale, Journal of Business Research, Vol. 24 No. 3, pp Bemowski, K. (1991), Restoring the pillars of, Quality Progress, October, pp Bollen, K.A. (1989), Structural Equations with Latent Variables, Wiley, New York, NY. Bolton, R.N. and Drew, J.H. (1991a), A longitudinal analysis of the impact of service changes on customer attitudes, Journal of Marketing, Vol. 55, pp Bolton, R.N. and Drew, J.H. (1991b), A multi stage model of customer s assessments of service quality and value, Journal of Consumer Research, Vol. 17, pp Boulding, W., Kalra, A., Staelin, R. and Zeithaml, V.A. (1993), A dynamic process model of service quality: from expectations to behavioural intentions, Journal of Marketing Research, Vol. 30, pp Brown, M.W. and Cudeck, R. (1993), Alternative ways of assessing model fit, in Bollen, K.A. and Long, J.S. (Eds), Testing Structural Equation Models, Sage, Newbury Park, CA. Bryne, B.M. (1994), Structural Equation Modelling with EQS and EQS/Windows-Basic Concepts, Applications and Programming, Sage, Thousand Oaks, CA. Bryne, B.M. (1998), Structural Equation Modeling with LISREL, PRELIS and SIMPLIS: Basic Concepts, Applications and Programming, Lawrence Erlbaum Associates, Mahwah, NJ. Carman, J.M. (1990), Consumer perceptions of service quality: an assessment of the SERVQUAL dimensions, Journal of Retailing, Vol. 66, pp Cattell, R.N. (1966), The scree test for the number of factors, Multivariate Behavioural Research, Vol. 1, pp Churchill, G.A. and Surprenant, C. (1982), An investigation into the determinants of customer satisfaction, Journal of Marketing Research, Vol. 19, pp Cronin, J.J. and Taylor, S.A. (1992), service quality: reexamination and extension, Journal of Marketing, Vol. 56, pp Diamantopoulos, A. and Siguaw, J.A. (2000), Introducing LISREL, Sage, London. Eisen, S.V., Wilcox, M. and Leff, H.S. (1999), Assessing behavioural health outcomes in outpatient programs: reliability and validity of the BASIS-32, Journal of Behavioural Health Sciences & Research, Vol. 26 No. 4, pp Finn, D.W. and Lamb, C.W. (1991), An evaluation of the SERVQUAL scale in a retailing setting, in Holman, R. and Solomon, M.R. (Eds), Advances in Consumer Research, Association for Consumer Research, Provo, UT, pp

16 MIP 24,1 46 Firdaus, A. (2005), The development of HEdPERF: a new measuring instrument of service quality for, International Journal of Consumer Studies, online publication, 20 October. Franceschini, F. and Rossetto, S. (1997b), On-line service quality control: the Qualitometro method, De Qualitac, Vol. 6 No. 1, pp Ginsberg, M.B. (1991), Understanding Educational Reforms in Global Context: Economy, Ideology and the State, Garland, New York, NY. Hair, J.F. Jr, Anderson, R.E., Tatham, R.L. and Black, W.C. (1995), Multivariate Data Analysis with Readings, Prentice-Hall, Englewood Cliffs, NJ. Hattie, J. (1985), Methodology review: assessing unidimensionality of tests and items, Applied Psychological Measurement, Vol. 9, pp Hattie, J. (1990), Performance indicators in education, Australian Journal of Education, No. 3, pp Joreskog, K.G. and Sorbom, D. (1978), Analysis of Linear Structural Relationships by Method of Maximum Likelihood, National Educational Resources, Chicago, IL. Kaiser, H.F. (1970), A second-generation little jiffy, Psychometrika, Vol. 35, pp Krejcie, R. and Morgan, D. (1970), Determining sample size for research activities, Educational and Psychological Measurement, Vol. 30, pp Lassar, W.M., Manolis, C. and Winsor, R.D. (2000), Service quality perspective and satisfaction in private banking, Journal of Services Marketing, Vol. 14 No. 3, pp Lawson, S.B. (1992), Why restructure? An international survey of the roots of reform, Journal of Education Policy, Vol. 7, pp Lewis, R.C. and Booms, B.H. (1983), The marketing aspects of service quality, in Berry, L., Shostack, G. and Upah, G. (Eds), Emerging Perspectives on Services Marketing, American Marketing, Chicago, IL, pp Llusar, J.C.B. and Zornoza, C.C. (2000), Validity and reliability in perceived quality measurement models: an empirical investigation in Spanish ceramic companies, International Journal of Quality & Reliability Management, Vol. 17 No. 8, pp Mazis, M.B., Ahtola, O.T. and Klippel, R.E. (1975), A comparison of four multi-attribute models in the prediction of consumer attitudes, Journal of Consumer Research, Vol. 2, pp Mehta, S.C., Lalwani, A.K. and Han, S.L. (2000), Service quality in retailing: relative efficiency of alternative measurement scales for different product-service environments, International Journal of Retail & Distribution Management, Vol. 28 No. 2, pp Nunnally, J.C. (1988), Psychometric Theory, McGraw-Hill, Englewood-Cliffs, NJ. Oliver, R.L. (1989), Processing of the satisfaction response in consumption: a suggested framework and research propositions, Journal of Consumer Satisfaction, Dissatisfaction, and Complaining Behaviour, No. 2, pp Owlia, M.S. and Aspinwall, E.M. (1997), TQM in a review, International Journal of Quality & Reliability Management, Vol. 14 No. 5, pp Parasuraman, A., Zeithaml, V.A. and Berry, L.L. (1985), A conceptual model of service quality and its implications for future research, Journal of Marketing, No. 49, pp Parasuraman, A., Zeithaml, V.A. and Berry, L.L. (1988), SERVQUAL: a multiple-item scale for measuring consumer perceptions of service quality, Journal of Retailing, Vol. 64 No. 1, pp Parasuraman, A., Berry, L.L. and Zeithaml, V.A. (1991a), Refinement and reassessment of the SERVQUAL scale, Journal of Retailing, Vol. 67 No. 4, pp

17 Parasuraman, A., Berry, L.L. and Zeithaml, V.A. (1991b), More on improving service quality measurement, Journal of Retailing, Vol. 69 No. 1, pp Parasuraman, A., Zeithaml, V.A. and Berry, L.L. (1994), Reassessment of expectations as a comparison standard in measuring service quality: implications for future research, Journal of Marketing, Vol. 58, pp Quester, P., Wilkinson, J.W. and Romaniuk, S. (1995), A test of four service quality measurement scales: the case of the Australian advertising industry, Working Paper 39, Centre de Recherche et d Etudes Appliquees, Group esc Nantes Atlantique, Graduate School of Management, Nantes. Robledo, M.A. (2001), and managing service quality: integrating customer expectations, Managing Service Quality, Vol. 11 No. 1, pp Soutar, G. and McNeil, M. (1996), a tertiary institution, Journal of Educational Administration, Vol. 34 No. 1, pp Sureshchandar, G.S., Rajendran, C. and Anantharaman, R.N. (2002), Determinants of customer-perceived service quality: a confirmatory factor analysis approach, Journal of Services Marketing, Vol. 16 No. 1, pp Teas, R.K. (1993a), Expectations, performance evaluation, and consumers perceptions of quality, Journal of Marketing, Vol. 57 No. 4, pp Teas, R.K. (1993b), Consumer expectations and the measurement of perceived service quality, Journal of Professional Services Marketing, Vol. 8 No. 2, pp Zeithaml, V.A. and Bitner, M.J. (1996), Services Marketing, McGraw-Hill, Singapore. 47 Further reading Crawford, F. (1991), Total Quality Management, Committee of Vice-Chancellors and Principals Occasional Paper, London, December. To purchase reprints of this article please reprints@emeraldinsight.com Or visit our web site for further details: