The author, Ron Baker, is one of the directors of the commission that sends evaluators to this college.

Transcription

1 Introductory Note: The accreditation process has changed dramatically in the last ten years. This article explains why and outlines the new expectations that are used for evaluation. The author, Ron Baker, is one of the directors of the commission that sends evaluators to this college. In the article, Baker refers to external constituencies applying pressure to accreditation commissions to raise their standards of accountability. These constituencies would include the general public (taxpayers), political parties, state legislatures and the Congress. It also includes college and university systems around the US. One of the unexplained ironies here is that institutions of higher education themselves have lobbied for tougher standards of accreditation because since the 1980s they have faced competition from new, private institutions that offer accredited courses in non-conventional ways. The new types of colleges no campus, classes on weekends, online courses, etc. can receive federal funds for financial aid and give degrees that have the same standing as though given by traditional institutions. Big university systems like the University of California have gone to Congress and argued that these new colleges may not be offering courses equivalent to the public institutions or traditional private colleges. They argue that these degrees may not have rigorous standards backing them up. Some of these colleges have very high aggregate grade point averages, leading critics to say their system is based on the slogan, You pay, you get the A. Also, the students at these institutions are eligible for financial aid and thus compete for the funds with conventional institutions. Therefore, regional accreditation commissions have adopted more rigorous standards of accreditation and accountability. -- Bill Swanson EVALUATING QUALITY AND EFFECTIVENESS: REGIONAL ACCREDITATION PRINCIPLES AND PRACTICES Ronald L. Baker Associate Executive Director, Commission on Colleges and Universities, Northwest Association of Schools and of Colleges and Universities, th Avenue NE, Suite 100, Redmond, Washington (1) This article outlines the philosophy and principles of regional accreditation as it responds to increasing calls for quality assurance while maintaining traditional values of self-regulation and quality improvement in higher education. (2) Quality is one of higher education's most deeply held values, yet the characteristics of quality and measures of effectiveness for higher

2 education are neither easily identified, clearly understood, nor universally accepted. Therefore, it is difficult to delineate a common set of appropriate and universally applicable criteria that enable authentic evaluation of all institutions of higher education. As a result, a judgment of quality in higher education was traditionally determined more by implicit perceptions of institutional reputation and characteristics than by explicit evidence of outcomes and achievements. In recent years, shifts in societal attitudes and expectations are forcing higher education to move toward specific meaningful assessment and reporting of the quality of its infrastructures, processes, and outcomes. (3) Throughout its history the higher education community has been the primary adjudicator of quality in American colleges and universities. More often than not, judgments relating to quality were formed by intuitive perceptions of institutional reputation and resources rather than from explicit evidence of achievement of intentions and implied outcomes.( n1) More recently, however, higher education's role as sole arbitrator of institutional effectiveness has eroded, due, in part, to a decrease in public confidence regarding the ability of colleges and universities to authenticate the achievement of results. One indicator of the decline of confidence is skepticism regarding the meaning, relevance, and significance of traditional grades and degrees as effective measures of achievement of intended institutional and program outcomes. Contributing to the decline of public confidence is a shift of values from higher education as a social model that benefits individuals directly and society indirectly to an economic resource model that requires a direct and immediate return on investments.( n2) As a result, colleges and universities have experienced unprecedented demands for accountability based upon explicit evidence of achievement of intended results.( n3) (4) Institutional constituencies increasingly expect colleges and universities to produce data-driven evidence of the quality and achievement of intended outcomes to document institutional effectiveness. Unfortunately, higher education has been slow to adjust to these developments, believing that society's interests are best served by advancing the academy's interests and therefore, the public should simply trust in higher education's ability to regulate itself.( n4) Over the past decade, however, quality assurance systems such as accreditation have expanded their evaluation criteria to include an emphasis on the achievement of institutional outcomes as well as a judgment of institutional intentions and capacity. In doing so, accreditation is preserving long-held educational values of quality improvement and self-regulation while simultaneously addressing society's needs for accountability and quality assurance.

3 ACCREDITATION (5) In the United States, accreditation is the oldest and best known seal of collegiate quality.( n5) It has dual purposes of fostering quality improvement and providing quality assurance and is considered to be the cornerstone of self-regulation.( n6) Accreditation fulfills its dual purposes through a collegial process of institutional self-assessment and critical peer evaluation based upon criteria established by voluntary non-governmental associations. Although an accreditation evaluation is not an institutional audit, it does ensure that the institution meets minimum standards of quality.( n7) (6) There are two primary types of accreditation: regional (institutional) accreditation and specialized (program) accreditation. Regional accreditation commissions consider the institution as a whole. They evaluate the entire institution using qualitative standards that emphasize achievement of institutional mission and goals and do not specifically monitor or accredit individual programs or subject content areas. Specialized accreditation associations on the other hand evaluate individual programs for compliance with quantitative program-specific standards that are independent of institutional mission and goals. (7) Regional accreditation is simultaneously the academy's primary mechanism for quality assurance and one of its major avenues for self-improvement.( n8) It is a mission-driven process that reflects the diversity of institutions within American higher education. Consequently, quality cannot always be defined in precisely the same terms for all institutions, so an institution must be evaluated in terms of the purposes it seeks to accomplish. Evaluation processes require the institution, and units within the institution, to examine continuously their own goals, operations, and achievements. In demonstrating the achievement of institutional purpose, accredited institutions are expected to build a convincing case that results are consistent with intentions. More than an emphasis on meeting minimum indicators of quality, regional accreditation focuses on institutional capacity and the will, culture, and ability of an institution to improve continuously.( n9) (8) While performing a number of important functions, regional accreditation encourages purposeful change and improvement toward maximum institutional and educational effectiveness in fulfilling the mission and goals. Recognition as a regionally accredited institution warrants the confidence of the educational community and the public with regard to the institution's performance, quality, and integrity by ensuring that an accredited institution:

4 ( 1) has clearly defined and appropriate educational objectives, ( 2) has conditions under which institutional objectives can reasonably be achieved, ( 3) appears to be substantially accomplishing its objectives, and ( 4) is reasonably organized, staffed, and supported to continue to do so.( n10) (9) Regional accreditation is grounded in traditional academic values of self-regulation, academic integrity, and collective responsibility.( n11) It reflects the belief that a determination of institutional quality exceeds the sum of measures of quality of individual units within the institution.( n12) Consequently, regional accreditation commissions render a comprehensive judgment regarding the overall integrity and viability of an institution as a whole, rather than judgments of individual institutional units or programs. CRITERIA FOR EVALUATION (10) That institutional mission statements nearly always include a reference to quality or excellence is not surprising. It is also not surprising to find that quality and excellence are rarely defined and criteria for evaluating quality are virtually nonexistent. Because institutional effectiveness is contextual and educational outcomes are multiple and diverse, it is difficult to define a single set of criteria to measure quality and effectiveness in absolute terms for all institutions.( n13) While educational results in general, and student learning outcomes in particular, would appear to be a common basis for measurement, they vary to reflect institutional characteristics, cultures, and missions. Regional accreditation is charged with the task of identifying indicators of quality that apply equally to a variety of higher education institutions with otherwise vastly different missions and goals. Consequently, regional accreditation does not compare one institution against another. (11) In regional accreditation, evaluation criteria are agreed upon by member institutions as characteristics of quality. They are catalytic rather than prescriptive and serve as a framework that applies equally to all institutions. They ensure that all accredited institutions meet minimal expectations. Further, they provide a foundation for feedback and assistance to institutions as they seek to fulfill their own missions and goals. Judgments of institutional quality and effectiveness are determined in relationship to the appropriateness of its mission and goals; adequacy and effectiveness of its resources; and manner and degree to which it achieves its mission and goals.( n14) Institutional evaluation, therefore, proceeds from the institution's own definition of its mission and goals. An evaluation of institutional quality and effectiveness is based upon established criteria used to determine the

5 extent to which institutional mission and goals are achieved. (12) Historically the actual content of accreditation standards may have had less to do with the effectiveness of the evaluation criteria than the consistent actions of institutions and evaluators.( n15) Evaluation criteria were typically based upon implications drawn from an examination of infrastructure, resources, and processes. More recently, regional accreditation commissions require institutions to document the achievement of intended outcomes as a means of determining effectiveness in achieving institutional mission and goals. The resulting evaluation criteria, therefore, are appropriately broad and deliberately vague, but do contain expectations that institutions demonstrate the adequacy, appropriateness, and availability of resources and processes used to achieve intended outcomes. The criteria do not, however, explicitly define those terms since they are more appropriately dictated by institutional mission and goals. (13) For the Commission on Colleges and Universities, the higher education regional accreditation commission of the Northwest Association of Schools and of Colleges and Universities, evaluation criteria take the form of eligibility requirements, standards, and related policies. Eligibility requirements are institutional characteristics and conditions required for initial and ongoing accreditation. They reflect critically important elements of the Commission's standards and policies and form the foundation for accreditation. Commission standards and related policies, the primary criteria by which quality is evaluated, are interrelated to emphasize the interconnected nature of the institution. They assist the institution, peer evaluators, and the Commission in determining the quality and effectiveness of the institution in achieving its mission and goals. They are qualitative statements that, with evidence, the institution may be judged by peers to be effective in achieving its mission. ROLE OF ASSESSMENT (14) Regional accrediting commissions expect each accredited institution to define its mission, set goals that lead to the fulfillment of the mission, identify indicators of goal achievement, develop and implement methods of assessing its effectiveness, evaluate the results of the assessments, and demonstrate that assessments and evaluations are used in an ongoing cycle of planning and evaluation.( n16) Collectively, these expectations help assure the public and the educational community of institutional integrity with regard to institutional performance and institutional purpose.

6 (15) Assessment of educational quality has always been and remains at the heart of the accreditation process, but quality must not be determined solely on the basis of economy, productivity and efficiency.( n17) Within regional accreditation, quality is measured primarily in terms of institutional integrity, the characteristics of resources and processes, and manner and degree to which an institution fulfills its mission and goals. In that context, assessment--the process of gathering data or measures and assembling them in some understandable form--is an important means to document evidence of outcomes and achievements.( n18) Further, regional accreditation expects institutions to form judgments based upon an evaluation of the assessment data and to use the results of those evaluations to inform and improve planning. (16) Assessing institutional results is a means to provide a foundation of evidence to demonstrate that the institution is willing and equipped to examine itself on an ongoing basis.( n19) In earlier times, this assessment tended to focus upon processes, structures, and resources such as the academic degrees held by faculty members, the number of books in the library, and the size of institutional budgets. Evaluation proceeded on the assumption that the quality of implied institutional outcomes was dependent on the quality of the institutional "inputs." In recent years, however, external constituencies have become increasingly skeptical of the validity of judgments of quality based more upon a review of institutional intentions and resources than evidence of institutional outcomes and results.( n20) (17) While clearly not abandoning steady attention to inputs, regional accrediting commissions have responded to calls for results by adopting criteria that enable an evaluation of institutional outcomes as well as institutional inputs as part of an overall evaluation of the institution.( n21) The effect of this movement is that accredited institutions are now required to provide substantive data-driven evidence of quality and effectiveness in achieving missions and goals. Given the diversity of institutional missions and goals, however, regional accreditation commissions do not dictate a particular form of assessment as the best or most appropriate means of determining quality and effectiveness.( n22) (18) Regional accreditation evaluation criteria do not prescribe specific methods nor do they dictate the characteristics and form of assessment and evaluation. Evaluation criteria do, however, stress outcomes assessment as an essential part of ongoing institutional self-study, assessment, and evaluation. In accord with its mission and structure, each accredited institution is expected to formulate a plan which provides for a comprehensive assessment of outcomes and, further, to

7 incorporate the results of assessments to improve planning that leads to successful achievement of mission and goals.( n23) To illustrate the importance of assessment to regional accreditation, the Commission on Colleges and Universities of the Northwest Association of Schools and of Colleges and Universities includes an explicit reference to assessment, evaluation, measurement, or judgment of quality and effectiveness in each of its standards for accreditation. (19) As a result of the uniqueness of individual institutional characteristics and missions, some systems of assessment are more formalized than others; some are more quantitative while others are more qualitative; some are well-developed and others are evolving; some are adopted from outside the institution while others are developed within the institution. Assessment and evaluation can be conducted in a variety of ways and based upon a variety of data sources. Regardless of the specific strategy selected by the institution, regional accreditation commissions encourage assessment of multiple indices of quality and effectiveness to document the institution's progress in fulfilling its mission and goals. (20) Assessment and evaluation are intended as means to demonstrate institutional effectiveness, foster institutional improvement, and demonstrate accountability. Unfortunately, without a clear sense of the purpose for assessment, knowledge of what is to be assessed, and understanding of how the results of assessment will be used, assessment efforts all too often become an end in themselves. (n24) If the results of assessment are not used to inform planning and decisions, colleges and universities may find themselves in a position of being data-rich and information poor. PRESERVATION OF PRINCIPLES (21) The relatively recent emphasis on assessment by regional accreditation reflects an original and historic interest by the accrediting associations on the quality of educational results.( n25) In part, however, it constitutes a reaction to pressure from external constituencies to be more directly accountable for the results of their efforts. At the turn of the 21st century, therefore, regional accreditation commissions are expanding their efforts to review and implement evaluation criteria and procedures that assure the public and peer institutions that an accredited institution not only has appropriate resources and processes in place to achieve its mission and goals, but that it assesses and documents its effectiveness in achieving intended outcomes. Consequently, institutions are providing more direct

8 data-based evidence of achievement of outcomes. Those data, in turn, lead to more informed judgments of quality and effectiveness and foster public confidence that the academy is able to regulate itself effectively. Further, assessment data enable the public to make first hand judgments regarding institutional accountability and stewardship of resources and benefit institutions by providing important information regarding progress in fulfilling missions and goals. (22) Regional accrediting commissions do not determine institutional missions and goals nor do they specify institutional, program, or student learning outcomes. Those decisions should be made by the institution and its stakeholders.( n26) However, once the institution articulates its mission and goals, regional accreditation commissions do require accredited institutions to engage in ongoing planning to achieve their missions and goals and to evaluate how well, and in what ways, they are achieving their missions and goals. In doing so, the institution is expected to assess itself in a regular, systematic, and substantive manner and, thus, provide evidence of the achievement of intended outcomes. Further, institutions are expected to document that assessment data are gathered, evaluated, and used to inform broad-based, continuous evaluation and planning that results in improvements to programs and services.( n27) Ultimately, internal and external institutional assessment and evaluation are to provide evidence that documents the achievement of indicators of quality and performance, thus ensuring that the distinctive mission of the institution and the quality of its programs and services are both affirmed and advanced.( n28) (23) Recent changes in accreditation standards and policies are intended to preserve the values and principles of quality and effectiveness while allowing the criteria for institutional evaluations to adapt as appropriate and warranted. As a result, regional accreditation commissions have incorporated criteria based upon goal achievement and the level of agreement with regard to institutional results and institutional intentions as the primary indices of higher educational quality and effectiveness.( n29) Recent actions taken by the Commission on Colleges and Universities of the Northwest Association of Schools and of Colleges and Universities illustrate the tenet of preserving principles of quality while allowing evaluation criteria to vary. (24) One of the Commission's long-standing Eligibility Requirements for continuing accreditation was intended to ensure that the accredited institution "has at least a core library and learning resources (emphasis added) appropriate to its mission, and it provides access to specialized library and learning resources needed for independent work in the fields and at the levels represented by its offerings. If it depends in part on other institutions for specialized library and

9 learning resources, it can demonstrate that they are adequate, easily accessible, and used."( n30) The principle underlying this requirement is that library and information resources are essential to the learning process and, thus, are necessary to the success of any educational enterprise. This eligibility requirement embodied the historical belief that a physical library with adequate and appropriate resources was the best way to ensure availability and access to library and information resources needed to support student learning and scholarly activity. (25) Recent advances in technology and delivery systems enable alternative means for institutions to achieve this intended outcome. Effective September 1, 2001, the Commission modified its eligibility criteria to ensure that "the institution provides library resources, technology and services (emphasis added) for students and faculty appropriate for its mission and for all of its educational programs wherever located and however delivered."( n31) The Commission's modification preserves the principle regarding the importance of library and information resources as a critical contributor to the quality and effectiveness of educational outcomes while recognizing that institutions may now choose from multiple options that manifest fidelity with that principle. (26) In light of the revision to the library and information resources eligibility requirement, the Commission has undertaken a process to review and revise Standard Five, Library and Information Resources. The purpose of the revision is to ensure internal consistency between the spirit and construct of the eligibility requirement and the criteria for evaluation contained within the Commission's standard on library and information resources. The results of these efforts reinforce the Commission's values and principles of quality and accreditation while putting them into a modern context that reflects a proactive response to a mandate to evaluate institutions in a relevant and meaningful manner. (27) As it continuously reviews its standards and practices for judging institutional quality and fostering quality improvement, regional accreditation is adapting its evaluation criteria to justify the confidence of the public and the educational community that self-regulation is meaningful and effective in judging quality and effectiveness. One way it encourages that confidence is by requiring member institutions to conduct continuous, effective, and relevant assessment and evaluation of its infrastructure, processes, and outcomes to document clear and compelling evidence of achievement of results that correlate closely with institutional mission and goals. Acknowledgment: The views expressed in this article are those of the author and should not be attributed to the Commission on Colleges and Universities or the Northwest Association of Schools and of Colleges and Universities.

10 NOTES AND REFERENCES (n1.)roger Peters, "Some Snarks Are Boojums: Accountability and the End(s) of Higher Education," Change 26(6) (1994): (n2.)peter T. Ewell, "A Matter of Integrity: Accountability and the Future of Self-Regulation," Change 26(6) (1994): (n3.)ralph A. Wolff & Olita D. Harris, "Using Assessment to Develop a Culture of Evidence," in Changing College Classrooms: New Teaching and Learning Strategies for an Increasingly Complex World, edited by D. F. Halpern (San Francisco: Jossey-Bass 1994), pp Available: ERIC Document Reproduction Service No. ED (n4.)ewell, "A Matter of Integrity: Accountability and the Future of Self-Regulation," p. 27; James L. Hudgins, "Institutional Effectiveness: A Strategy for Renewal," Community College Journal 63(5) (1993): (n5.)e. Grady Bogue, "Quality Assurance in Higher Education: The Evolution of Systems and Design Ideals," New Directions for Institutional Research 25(3) (1998): (n6.)ralph A. Wolff, "Restoring the Credibility of Accreditation," Trusteeship 1(6) (1993): (n7.)bogue, "Quality Assurance in Higher Education: The Evolution of Systems and Design Ideals," p. 10; Wolff, "Restoring the Credibility of Accreditation," p. 23. (n8.)peter T. Ewell, "Outcomes, Assessment, Institutional Effectiveness, and Accreditation: A Conceptual Exploration," in Accreditation, Assessment, and Institutional Effectiveness: Resource Papers for the COPA Task Force on Institutional Effectiveness (1992): Available: ERIC Document Reproduction Service No. ED (n9.)ibid., p. 12. (n10.)commission on Colleges and Universities, Accreditation Handbook (Bellevue, WA: Author, 1999). (n11.)ewell, "A Matter of Integrity: Accountability and the Future of Self-Regulation," p. 28. (n12.)thomas P. Hogan, "Methods for Outcomes, Assessment Related to Institutional Accreditation," in Accreditation, Assessment, and Institutional Effectiveness: Resource Papers for the COPA Task Force on Institutional Effectiveness, pp (n13.)kay McCullough Moore, "Assessment of Institutional Effectiveness," New Directions for Community Colleges 14(4) (1986): (n14.)hudgins, "Institutional Effectiveness: A Strategy for Renewal," p. 43. (n15.)ewell, "A Matter of Integrity: Accountability and the Future of Self-Regulation," p. 29. (n16.)commission on Colleges and Universities, Accreditation Handbook, p. 36. (n17.)south Carolina Higher Education Assessment, Network Recommendations for Defining and Assessing Institutional Effectiveness. SC: Author (1994). Available: ERIC Document Reproduction Service No. ED (n18.)gene Packwood, "Issues in Assessing Institutional Effectiveness," in Assessing Institutional Effectiveness in Community Colleges, edited by Don Doucette & Billie Hughes (1990), pp Available: ERIC Document Reproduction Service No. Ed (n19.)ewell, "Outcomes, Assessment, Institutional Effectiveness, and Accreditation," p. 17. (n20.)hogan, "Methods for Outcomes, Assessment Related to Institutional Accreditation," p. 38. (n21.)ibid.

11 (n22.)ralph A. Wolff, "Assessment in the Western Accrediting Commission for Senior Colleges and Universities," North Central Association Quarterly 65(2) (1990): (n23.)commission on Colleges and Universities, Accreditation Handbook, p. 37. (n24.)wolff & Harris, "Using Assessment to Develop a Culture of Evidence," p (n25.)ewell, "Outcomes, Assessment, Institutional Effectiveness, and Accreditation," p. 1. (n26.)david G. Steadman, "Accreditation is not School Accountability," NASC Report 13(2) (2001): 1, 4-5. (n27.)commission on Colleges and Universities, Accreditation Handbook, p. 36. (n28.)bogue, "Quality Assurance in Higher Education," p. 15. (n29.)peter T. Ewell, "Effectiveness and Student Success in Community Colleges: Practices, Realities, and Imperatives." in Effectiveness and Student Success: Transforming Community Colleges for the 1990's. Proceedings from the Conference, edited by D. Walleri (1989): 1-9. Available: ERIC Document Reproduction Service No. ED (n30.)commission on Colleges and Universities, Accreditation Handbook, p. 7. (n31.)see Eligibility Requirements of the Commission on Colleges and Universities (revised December 2000). Available: procedures/procedures1.html.