Applying An Excellence Model To Schools

GLOBAL QUALITY Applying An Excellence Model To Schools by Pedro M. Saraiva, Maria J. Rosa and João L. d Orey Students, parents and society are demanding much more of schools as education becomes more and more important for national economic competitiveness, growth and even survival. The education sector, therefore, needs to better understand quality concepts and apply quality s principles and tools. 1, 2 Interesting developments and projects related to quality management in education have been In 50 Words Or Less Education is becoming increasingly important for nations wanting to compete in the global marketplace. The education sector needs to understand and use quality methods. A large-scale study in Portugal shows excellence models can be successfully used to help schools continuously improve. conducted across the world, mainly in the United States and Europe. For example, ASQ has played a role in the adoption of quality principles and tools in education through its Education Division and the Koalaty Kid Alliance. These and many other initiatives have already been written about extensively. 3-9 The almost universal trend of concern about quality in education took root in Portugal in the early 1990s and inspired us to conduct research at all levels from kindergarten to higher 10, 11 education. Excellence Models Among several approaches that can be used to guide the implementation of quality management principles in schools, one that has been followed with success involves the adoption of excellence models to support self-assessment practices and continuous improvement. 12-15 Related to this has been the creation of specific national and international quality award categories associated with education such as the Baldrige National Quality Program s development of education criteria in the United States. In Portugal, a large-scale project at 47 schools was based on the application of the European Foundation for Quality Management (EFQM) Excellence Model. 16-18 This well-known model had 46 I NOVEMBER 2003 I www.asq.org

already been widely used for achieving excellence by businesses, other leading organizations, government agencies and national quality award programs throughout Europe, including Portugal. As far as we know, this project seems to be the largest simultaneous application of self-assessment in education ever based on excellence models. Project Scope AEEP is a Portuguese national association of several hundred Portuguese private schools, ranging from kindergarten to secondary education. 19 The association decided there was a need to offer its associates the possibility of participating in a self-assessment process, with the ultimate goal being continuous performance improvement. To achieve its purpose, AEEP established partnerships with technical experts and a consulting company (QUAL), which would be responsible for designing, preparing, helping in the implementation of and supporting the self-assessment project. Each school conducted its own self-assessment process, in accordance with documents and training materials prepared for the project based on an adaptation of the EFQM model of excellence for educational institutions and counting on the support of consultants with specific training as EFQM model assessors. EFQM Excellence Model The EFQM Excellence Model is not prescriptive but instead recognizes there are different ways to achieve organizational excellence. It provides a sound framework for the diagnosis and evaluation of the excellence levels attained, leading to continual improvement based on nine criteria that cover about 30 subcriteria (see Figure 1, p. 48). The nine criteria belong to one of two possible categories: enablers and results. Enablers are directly concerned with what is done and the way it is done, while results have to do with what a given organization derives from the way enablers are managed and what it achieves as performance. For each subcriterion, self-assessment teams identify strong points and areas for improvement, and they give a final quantitative score to each of the nine model criteria. From these partial scores, an overall aggregate excellence level (ranging from zero to a theoretical maximum of 1,000 points) can be quantified at each school. When adapted to an educational context, the essence of this model can be synthesized as follows: Excellent performance in a school, students, teachers, nonteaching staff and society is achieved through leadership that catalyzes the institutional policy and strategy and the management of people, QUALITY PROGRESS I NOVEMBER 2003 I 47

GLOBAL QUALITY FIGURE 1 partnerships, resources, and educational and support processes. Project Methodology The first phase of our quality improvement research project consisted of the design and development of a set of supporting documents. A sixpiece packet was created and first tested on a few teachers. This packet included: Self-assessment guidelines. A self-assessment document. A self-assessment case study of one school. A case study scoring document. An improvement facilitator handbook. Guidelines for preparing self-assessment action plan reports. Self-assessment guidelines made up the core document, which consisted of an adaptation for the EFQM excellence model to educational contexts. It deviated from the EFQM model as little as possible so users could refer to and compare results with those obtained using the EFQM model in other fields of activity. The self-assessment document presented the methodology and tools to be used in each school for conducting its self-assessment. Two of the EFQM evaluation approaches (questionnaire and pro forma) were described in the document because schools used a combination of both in this project. At this initial stage of methodology application, we believe it is important not to overemphasize the relevance of quantitative evaluations, but rather to The EFQM Excellence Model Leadership Enablers People Policy and strategy Partnerships and resources Processes Innovation and learning People results Customer results Society results Results Key performance results stress the importance of self-assessment as an improvement driving force through its identification of improvement opportunities and actions. Nevertheless, surveys allowed us to aggregate and compare the quantitative profiles of excellence associated with the several participating schools at a more global level. The case study, describing practices and results, was a training tool for the schools self-assessment teams. It allowed a more practical understanding of how the model can be applied to a particular school. The case study scoring document provided a possible solution for the hypothetical school, based on the identification of strong points, areas for improvement, actions to implement and responses to the questionnaire provided by a team of experts on the application of the EFQM model. The improvement facilitator handbook served as a supporting document for the improvement facilitator the person responsible for leading the self-assessment project in each school. It explained and summarized what must happen, when and with whom. It also listed the things that should not be forgotten at each stage. Guidelines for preparing self-assessment and improvement action plan reports included some suggestions and hints related to project management and its good practices. The overall project consisted of a number of different tasks, according to the approximate schedule shown in Table 1. Depending on the particular characteristics and size of each school, the total number of self-assessment teams formed was between one and nine (one for each criterion). Teams had a mixture of school directors (principals), teachers, administrative staff, parents and students. Some Key Results Table 2 profiles the 47 different schools that participated in the project by education level and number of students. Overall, the 48 I NOVEMBER 2003 I www.asq.org

TABLE 1 Project Tasks and Overall Scheduling Task one June to September 2000 Project announcement and promotion Task two October and November 2000 Improvement facilitators training Task three October to December 2000 Self-assessment teams definition and training Task four January to May 2001 Self-assessment meetings Task five March to July 2001 Preparation of self-assessment and action plan reports Task six September 2001 Overall project final report Task seven January to December 2001 Implementation of action plans Task eight November and December 2001 Evaluation of practical results and changes in schools Task nine January 2002 Final project closing seminar schools employed 2,116 teachers and had nearly 30,000 students. The results obtained from the questionnaires, which correspond to 44 schools (Figure 2, p. 50), illustrate the excellence profiles associated with the nine criteria considered in the model. Confidence intervals are presented for the average percentage score assigned to each of the nine model criteria using a 95% confidence level, according to the selfassessment survey guidelines suggested for the EFQM model. These values show a significant difference between the scores that correspond to results criteria in general (overall average of 20.3%) and those associated with enablers criteria (overall average of 39.8%). This difference was not generally caused by a perception of poor school performance, but rather that the majority of the schools were still at an early stage of measuring results, with few indicators being monitored, compared or evaluated systematically. Combining these two partial components results in an interesting aggregate average score of 302 points out of 1,000, with most schools lying in the 250 to 350 points range. Such values provide evidence some schools have already reached remarkable levels of excellence. Indeed, the EFQM evaluation scale employed is quite demanding, with 500 considered a very good score that would be expected for national quality award candidates in Europe and 700 corresponding to European Quality Award winners. TABLE 2 Evaluation of the actual implementation of improvement actions resulting from self-assessment also provides positive information, since most schools actually implemented more than 60% of the planned actions within six months of completion of their self-assessment. Lessons Learned Much was learned during this research project: Most of the schools participating in this project lacked broad, objective and relevant indicators systematically collected and treated as performance improvement driving forces. Although some of the schools were doing very interesting and successful things, benchmarking practices and comparisons of results and approaches were very scarce because most of the schools tended to operate independently. The research team was quite surprised with some excellent and innovative practices it uncovered sometimes where least expected. Even when we compared these schools with School Profiles Education levels Number of Total number Average number of schools of students of students per school Kindergarten 27 2,634 98 Grades one to four 29 5,188 179 Grades five and six 25 4,721 189 Grades seven to nine 27 7,631 283 Grades 10 to 12 21 6,466 308 Others 8 1,754 219 QUALITY PROGRESS I NOVEMBER 2003 I 49

GLOBAL QUALITY Percentage score 60 50 40 30 20 10 0 Leadership noneducational organizations, there was evidence of many excellent schools operating in Portugal. The study s methodologies and paradigms were well received and accepted by most of Self-assessment is in the end thinking... to change, transform, renovate... grow. FIGURE 2 the participating schools, although some minor refinements and improvements were suggested. The results obtained so far indicate the EFQM excellence model can be easily applied and adapted by schools as a powerful performance improvement driving force. To have a large number of schools simultaneously follow the same methodology steps and then share their experiences proved quite pow- 50,1 46,4 42,6 42,8 38,3 33,7 Policy and strategy 43,3 39,1 34,9 43,1 39,8 36,6 40,0 36,2 32,3 People Partnerships and resources Processes Confidence intervals for a 0.05 significance level. 21,7 17,9 14,2 Customer results 21,7 17,5 13,4 People results 29,5 24,0 18,4 Society results 28,4 23,4 18,4 Key performance results erful, even if information exchanges were occasionally somewhat constrained because of confidentiality issues. As one school wrote in its final report, fully capturing the project s spirit, Self-assessment is in the end thinking... to change, transform, renovate,... grow. REFERENCES 1. Priorities and Strategies for Education A World Bank Review, World Bank, 1995. 2. Spencer B. Graves, Common Principles of Quality Management and Development Economics, Quality Management Journal, Vol. 2, No. 2, pp. 65-79. 3. Y.C. Cheng and W.M. Tam, Multi-Models of Quality in Education, Quality Assurance in Education, Vol. 5, No. 1, 1997. 4. Robert Lundquist, Quality Systems and ISO 9000 in Higher Education, Assessment and Evaluation in Higher Education, Vol. 22, No. 2, pp. 159-172. 5. Mel Farrar, Structuring Success: a Case Study in the Use of the EFQM Excellence Model in School Improvement, Total Quality Management, Vol. 11, No. 4/6, pp. 691-696. 6. R.S. McAdam and W.R. Welsh, A Critical Review of the Business Excellence Quality Model Applied to Further Education Colleges, Quality Assurance in Education, Vol. 8, No. 3, pp. 120-130. 7. Lee Jenkins, Improving Student Learning: Applying Deming s Quality Principles in the Classroom, second edition, ASQ Quality Press, 2003. 8. ASQ Education Division, Successful Applications of Quality Systems in K-12 Schools, ASQ Quality Press. 2003. 42,8 39,8 36,8 Enablers Percentage score according to the EFQM model criteria, expressed as confidence intervals for a 0.05 significance level. The average point estimate, lower and upper limit of the confidence interval are the values presented for each criterion. 50 I NOVEMBER 2003 I www.asq.org Excellence Levels for Participating Schools 24,0 20,3 16,6 Results 33,3 30,2 27,1 Global 9. ASQ Koalaty Kid, School Self- Assessment Guide for Performance Excellence, ASQ Quality Press, 2003. 10. Patricia Sá and Pedro Saraiva, The Development of an Ideal Kindergarten Through Concept Engineering/Quality Function Deployment, Total Quality Management, Vol. 12, No. 3, pp. 365-372. 11. Maria Rosa, Pedro Saraiva and Henrique Diz, Excellence in Portuguese Higher Education Institutions, Total Quality Management & Business Excellence, Vol. 14, No. 2, pp. 189-197. 12. Self-Assessment Guidelines for the Public Sector: Education, Euro-

pean Foundation for Quality Management, 1996. 13. Modelo Iberoamericano de Excelencia en la Gestión Interpretacion Para la Educatión, Fundación Iberoamericana Para la Gestión de la Calidade, 2000. 14. Modelo Europeo de Gestión de Excelencia Adaptación a los Centros Educativos, Secretaría General Técnica, Ministerio de Educación Cultura y Deporte (Spain), 2001. 15. Mark L. Blazey, Karen S. Davison and John P. Evans, Insights to Performance Excellence in Education 2003: An Inside Look at the 2003 Baldrige Award Criteria for Education, ASQ Quality Press, 2003. 16. The EFQM Excellence Model 1999 Public and Voluntary Sector, European Foundation for Quality Management, 1999. 17. Assessing for Excellence A Practical Guide for Self-Assessment, European Foundation for Quality Management, 1999. 18. Determining Excellence A Questionnaire Approach, European Foundation for Quality Management, 1999. 19. AEEP homepage, www.aeep.pt. PEDRO M. SARAIVA is an associate professor in the chemical engineering department and pro-rector of the University of Coimbra, Portugal. He is also founder of Qual Ltd., winner of the 1998 Feigenbaum Award and ASQ country councilor for Portugal. MARIA JOÃO PIRES DA ROSA is a doctoral student at the University of Aveiro in Portugal. JOÃO LAGOA D OREY is executive coordinator for the Portuguese Charter of Engineers and a collaborator of Qual Ltd. Please comment If you would like to comment on this article, please post your remarks on the Quality Progress Discussion Board at www.asq.org, or e-mail them to editor@asq.org. EXAMINERS WANTED The Malcolm Baldrige National Quality Award, the nation s premier recognition for performance excellence, is seeking experts in business, education, and health care to volunteer on the 2004 Board of Examiners. As a Malcolm Baldrige Examiner you will: Learn how leading organizations achieve performance excellence Network with some of the nation s foremost quality professionals Use your expertise to improve America s competitive position See what it s all about! For more information call us at (301) 975-2036, E-mail us at nqp@nist.gov. Online application available November 2003. B a l d r i g e A w a r d Visit us online for an application! www.baldrige.nist.gov/examiners/qp.htm QUALITY PROGRESS I NOVEMBER 2003 I 51