Innovative Teaching Models at a Research University: Establishing an Evaluation Framework and Delineating Lessons Learned

Transcription

1 Evaluation Framework 1 Innovative Teaching Models at a Research University: Establishing an Evaluation Framework and Delineating Lessons Learned Melinda R. Hess Jeffrey D. Kromrey Sanaa Bennouna Elizabeth Gulitz Ann Barron Thomas Lang Shauna Schullo Ha Phan Heather Scott Kristine Y. Hogarty Melissa Venable Amy Hilbelink Gianna Rendina-Gobioff Lou Carey University of South Florida Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada, April 11-15, 2005.

2 Evaluation Framework 2 Abstract This study describes the development and implementation of a comprehensive evaluation framework applied to a five-year technology enhancement project at a major metropolitan research university. Project goals included development of innovative teaching models, expansion of the university s infrastructure to support the new models, and establishment of a coherent program of research. Model effectiveness was examined with respect to cost, effectiveness, access, and implications for underserved populations. The paper describes the evaluation framework for the project; the development and validation of instrumentation for data collection from multiple sources, and the utility of the resulting data for both informing the project and planning enhancements to the evaluation framework. Finally, the practical lessons learned during the conduct of the evaluation are delineated.

3 Evaluation Framework 3 Innovative Teaching Models at a Research University: Establishing an Evaluation Framework and Delineating Lessons Learned Introduction Educational practices and processes are undergoing a rapid and ever-expanding evolution in order to meet the needs of the emerging information society. The use of technology in education and the development of distance learning courses and programs have increased at a rapid pace (Eaton, 2002). Initiatives for retooling the infrastructure of higher education as well as adjusting faculty development opportunities have generated a variety of programs in which technology plays integral roles. The effective development and implementation of such programs requires carefully planned and executed evaluation strategies. The purpose of this paper is to disseminate the framework, methods and instrumentation that were developed for the evaluation of a five-year technology project at a large, metropolitan, Carnegie Doctoral/Research University-Extensive institution. The major goals of the project to be evaluated (Innovations in Technology and Teaching) include development of innovative teaching models through the combination of technology and pedagogy; expansion of the university s infrastructure to support the new models; and establishment of a coherent program of research related to innovations in teaching, technology, and learning. The effectiveness of these models was then examined with respect to cost, effectiveness, access, and potential implications for underserved populations. Objectives Three objectives guided this work. First, a logical, synthesized framework needed to be developed to adequately and thoroughly evaluate the impact and success of this project. The framework includes the questions leading the evaluation, the primary data sources that inform our efforts, the schedule of data collection activities, and the communication network required to support our evaluation efforts. Secondly, appropriate and efficient instruments and protocols were developed and subsequently examined for evidence of reliability and validity. Finally, the results were analyzed; delineating the utility of the information gathered and planned enhancements to the framework. As activities and processes were undertaken to meet these objectives, anecdotal and observational data were gathered to provide a list of lessons learned, including an outline of aspects that were effective and those that were not as successful.

4 Evaluation Framework 4 Project Description This university wide five-year grant was funded for $2.75 million with a 100% cost share from the university, totaling $5.5 million dedicated to develop innovative teaching models integrating synchronous and asynchronous instruction, and to examine the impact on cost, effectiveness and access. The challenge was to combine emerging technologies and innovative pedagogy to provide not just access, but equality of educational opportunity. Improvements in infrastructure were essential in order to develop, implement and test innovative teaching models. It required the cooperation and person power from numerous divisions including faculty and staff from six colleges, Educational Outreach (which facilitates the integration of technology in both face-to-face and distributed instruction), Academic Computing, the Center for 21 st Century Teaching Excellence, a partner education institution, industry partners, and the public broadcasting station, WUSFTV. The composition of the grant s strategic implementation team included an Associate Dean for Academic Affairs for an academic college who assumed the role of Principal Investigator (PI), the Associate Vice President for Academic Affairs for the university, co-pi, and a director who acted as the project manager. This structure was instrumental in helping frame, select, sponsor, and successfully execute projects with an overarching university and academic community scope, and within a limited timeframe. The PI who was immersed in faculty and student issues at the college level framed and validated key projects to implement, the co-pi was instrumental in helping secure university resources and provide sponsorship. The dedicated project manager acted as a single point of contact, structured the overall project, collaborated with the different sub-groups to keep the project on track, and helped resolve roadblocks. The grant had three operational categories, each with several objectives and accompanying activities (Figure 1). Infrastructure Enhancement, the first category, addressed enhancing the university infrastructure to support new delivery models by leveraging existing hardware as well as expanding capacity for distance based course delivery through the purchase of software, new hardware, the redesign of learning environments and faculty development. Development of faculty involved in the distance learning programs supported by this project was accomplished through hands-on workshops based on a training needs assessment, one-on-one teaming with an experienced instructional designer, and continuous collaboration between faculty, project staff, and leveraging of existing university resources. Examples of major infrastructure changes included creating new and

5 Evaluation Framework 5 upgrading existing technology enhanced classrooms, as well as installing a video conference bridge capable of serving 18 video conference classrooms on the main campus and five satellite campuses; purchasing compression hardware and a video indexing solution to facilitate collaboration between the university and the local public broadcasting station; purchasing a synchronous software solution to facilitate online interactions within courses, for academic advising, and for meeting/research purposes outside of the classroom environment; upgrading of satellite uplink to digital format and providing opportunities for faculty development. Online MA Programs Medical Workbench Mobile Environmental Lab Specialized Operational Innovations Disaster Management Hybrid BA Program Media Innovations Summative Formative Facility Enhancements Infrastructure Professional Development Evaluation Equipment Software Processes Synchronous DL Instruction Laptop Initiative Higher Education Policy & Practices Research Intellectual Property Assessing DL Courses Products Figure 1. Model of ITT initiatives relative to evaluation activities. The second category, Specialized Operational Innovations, addressed the need for flexible solutions to support underserved populations by developing courses and degree programs that implement novel strategies. These strategies supported online delivery of courses and included establishing instructional design teams to work with faculty to create and deliver several web courses (including three online graduate degree programs and one undergraduate degree program at a partner institution). Templates, processes, and web based tools were created and implemented to provide students and faculty support during course design and delivery (e.g., online library research

6 Evaluation Framework 6 module, syllabus template, synchronous software use, interactive narrated presentation and video tools). The Specialized Operational Innovations category also included the implementation of new software and technology resources for education. A software solution called the Medical Workbench, was purchased in support of academic programs in the health professions and athletic training. This system uses personal digital assistants (PDAs) to document and catalog acquired clinical skills, provide feedback from faculty, and implement a custom curriculum management and scheduling solution. Another novel strategy was to encourage learning by retrofitting a university shuttle bus into a flexible mobile field classroom that can carry state of the art scientific or electronic music equipment to various off campus sites. The third category, Research, included the design and implementation of selected research topics related to technology and distance learning, which will inform policies and practices of benefit to higher education institutions. Topics included the assessment of faculty awareness of intellectual property, copyright law, and fair use issues for online learning; an analysis of intellectual property policies of major research institutions; optimal structures for synchronous webbased courses; integral use of laptop computers in teacher preparation programs; and the development of a distance learning course evaluation system. Evaluation of the Project and Development of the Framework For each of the planned activities, specific evaluation questions, instrumentation, and data collection strategies were developed. Evaluation activities sought to (a) provide timely, detailed information to project directors about the progress of the major grant activities, while providing opportunities to modify the activities and timelines on an as-needed basis; and (b) provide an independent analysis of the impacts and outcomes that are attributable to project activities. The evaluation of a project with the degree of scope and breadth such as this required formulation of a general plan to guide initial efforts with the realization that flexibility was paramount. The nature of the funding for the project, a Congressional Award, permitted both project management and evaluation team members greater latitude to adjust resource allocation and anticipated activities to meet changing needs and availability of resources. Early on, the project management team recognized that a project of this scope, both in objectives and time frame (5 years), required intensive evaluation efforts and, as a result, devoted approximately 10% of the overall award toward evaluation activities. This permitted the evaluation team to approach the

7 Evaluation Framework 7 evaluation of this project from multiple perspectives, using a variety of evaluation techniques. The availability of multiple data sources, data collection and data analysis strategies was designed to provide project leaders and management with useful information throughout the project. One of the most significant contributors to the development, implementation and conduct of an in depth and informative evaluation is the composition of the evaluation team. The flexibility and resources provided to the evaluation efforts permitted the assembly of a team of experts with a mixture of evaluation and measurement expertise complemented with experts who were content experts in the areas targeted by the grant: technology and pedagogy. The evaluation team began with the six individuals from the Department of Measurement and Research as well as Instructional Technology in the College of Education. This team laid the groundwork for initial expectations of the evaluation and constructed a general evaluation plan to begin evaluation efforts. As the project progressed, the team grew in order to accommodate the increasing evaluation needs and diverse elements that became more apparent as the project matured. During the most intense phases of the project, the evaluation team comprised as many as 13 members. The framework and associated process and products resulting from this project were comprised of the three operational categories of the project with the evaluation piece a fourth, overreaching part of the resulting model (see Figure 1). The general framework called for using both formative and summative methods to evaluate the process undertaken to fulfill the goals and objectives of the award as well as evaluation of the resulting products. At the beginning of the evaluation process, data sources were identified, process and methods for evaluation decided upon, and instrumentation developed. As the project progressed, especially through the first year of evaluation activities, these data sources, process and instrumentation were adjusted to better meet the needs of the project overall, and the evaluation efforts specifically, as events informed the team about the effectiveness, efficiency and operability of the evaluation elements put in to place. Although the nature of the project was such that the team was able to adjust processes and instruments as time went on, the team was very sensitive to ensuring that any notable changes were well thought out and integrated into the process in such a way as to permit continuity and linking of information throughout the duration of the evaluation. After the first semester, few major adjustments were required except, at times, changes in the logistics of the data collection, e.g., delivering web-based survey through a different means to overcome previous technical difficulties identified in the first method of online delivery of the instrument.

8 Evaluation Framework 8 Evaluation Methods and Data Sources Many educational program evaluation models are available in the literature that can be collectively useful for focusing formative evaluations for programs and courses. For example, Krathwohl (1998) presented a useful overview of several approaches, including expert-judgment, goal-based, goal-free, management-oriented, adversarial, consumer-oriented, responsivenaturalistic, and participant-oriented. Such models are especially helpful for identifying which evaluation questions to ask, what data sources to tap for information related to the questions, and how to treat such data once they are obtained. For this endeavor, two evaluation models (Stufflebeam s (2002) CIPP project management model and Dick and Carey s (2001) systemsbased instructional design and development model) were used to plan the framework for the evaluation of the courses, technologies, and programs under consideration. In addition, the Joint Committee Standards for Educational Program Evaluation (1994) were used to further inform and guide evaluation activities and processes. The major questions guiding the evaluation are presented in Table 1. The questions are presented in line with the major strands that comprise the project: Infrastructure Enhancement (e.g., training, equipment acquisition), Specialized Operational Innovations (e.g., online course and program development, mobile laboratory) and Higher Education Policies and Practice Research (e.g., intellectual property rights, effective evaluation of online courses). A variety of data sources were used to address the evaluation questions (Table 2). Both qualitative data (in the form of interviews, focus groups and document analyses) and quantitative data (e.g., rating scales, usage logs and measurable faculty and student outcomes) were gathered. Three major components of evaluation were included: an analysis of the planned project activities, evaluation of the processes and progress of each component of the project, and evaluation of impacts and outcomes. Each of these components is described in detail. Planned Project Activities. This component consisted of an analysis of documents and interviews with major participants in the projects. The evaluation of previous needs assessments and individual project proposals provided a comprehensive overview of the scope of work that was funded under the grant, the proposed timelines for individual projects, and the extent of planned data collection and analysis activities. Fundamental to this aspect of the evaluation is the question of the extent to which the funded projects represent the depth and breadth of innovative models. Data from these analyses were used formatively to inform the potential revision of project activities.

9 Evaluation Framework 9 Evaluation of Processes. Information was collected from the Principal Investigators (PIs) and participants in the individual funded projects. Information provided in usage logs, problem reporting forms, and training records were monitored and reports of progress were produced. In addition, data were gathered each semester in the form of faculty questionnaires and meetings with the PIs to facilitate timely adjustments, if needed. These reports and leadership meetings were supplemented by conducting focus groups and interviews in order to share ideas and identify potential problems that may be common across projects. Finally, data from student enrollment, perceptions of usability and outcomes assessments were used to inform potential modifications of the projects. Evaluation of Impacts and Outcomes. During the second year of the evaluation activities, information on impact and outcomes were collected and analyzed. These data consist of student achievement and perception information, interviews and questionnaires for users of the enhanced infrastructure, and products of the funded research projects. The focus of this aspect of the evaluation is an assessment of the extent to which the purposes of the project have been fulfilled and the goals of the project have been met. In addition, evaluation data were gathered to inform recommendations for improvement in subsequent developmental activities related to innovative teaching and technology. Details on the specific faculty and student surveys, interview protocols, course observation forms and problem logs have been presented elsewhere as have reports on their psychometric properties and the results of their administration in this project (see, for example, Kromrey, Hogarty, Hess, Rendina-Gobioff, Hilbelink, & Lang, T., 2005; Hess, Kromrey, Lang, Hogarty, & Phan, 2004; Hess, Carey, & Kromrey, 2004; Kromrey, Hogarty, Barron, Schullo, Hilbelink, Singh, & Venable, 2004; Schullo, Barron, Kromrey, Hogarty, & Hilbelink, 2004; Schullo, Kromrey, Barron, Hogarty, Hilbelink, & Venable, 2004; Hogarty, Kromrey, Barron, Carey, & Hess, 2004; Smith, Kromrey, Barron, Carey, Hogarty, & Hess, 2004; Hess, Kromrey, Phan, Subramaniam, Carey, Hogarty, Schullo, & Smith, 2004; Kromrey, Phan, Hess, Subramaniam, Carey, Hogarty, Schullo, Smith, 2004). These detail data will not be repeated here, but copies of all instruments and reports are available on the project website at

10 Evaluation Framework 10 Table 1 Evaluation Questions. Infrastructure Enhancement Equipment What are the goals of the equipment acquisitions (i.e., what needs were to be met)? What are the nature, structure, purpose and cost of the intended innovation? To what extent does the enhanced infrastructure meet the identified needs and fulfill the intended purpose? How can the equipment infrastructure and its use be improved? Training Initiatives What are the goals of the faculty and staff training and how were they determined? To what extent has the training been effective? What is the impact of the training on the trainers? What training needs remain and how can training be improved? Course and Program Development To what extent are the funded development projects innovative (to the field as a whole and to the faculty members)? What are faculty conceptions of innovative learning models? To what extent are faculty prepared for the development and delivery of the funded courses and programs? What supports are available to faculty for development and delivery? What obstacles are encountered during development and delivery? How are these obstacles overcome? To what extent are the programs and courses effective? How can course development and delivery be improved? Technology Innovation Research To what extent do funded research projects reflect the depth and breadth of potential innovations in technology, teaching and learning? To what extent are deliverables provided and results disseminated? To what extent do these funded projects lead to additional external support? How can the support of research in technology and teaching innovations be improved?

11 Evaluation Framework 11 Table 2 Data Sources for the Project Evaluation. Grant Activities Enhancement of Infrastructure Equipment Data Sources Document Analysis Interviews Usage Logs User Questionnaire User Interviews Training Initiatives Needs Assessments Training Records Learner Evaluations Outcome Measures Course and Program Development Document Analysis Innovation Scale Faculty Questionnaire Focus Groups Assistance Logs Student Enrollment Student Evaluations Usability Assessment Student Outcome Measures Faculty Time Logs Technology Innovation Research Document Analysis PI Progress Reports PI Final Reports

12 Evaluation Framework 12 Discussion The development and implementation of the methods and instrumentation used within the framework for the evaluation of this project resulted in a series of processes and tools that not only informed this project, but which can be used in subsequent evaluation efforts. For example, the iterative nature of the development and delivery of many of the instruments permitted ongoing psychometric analyses of surveys with various student samples to help inform the generalizability of the instruments developed. As such, the team was able to take instruments developed in support of this initiative (e.g., student surveys for online courses) and use them as a basis to develop an instrument that could be adapted for other online courses and programs. A rather interesting outcome of the evaluation process engaged upon in this project was the ability of program managers to use the ongoing feedback from the evaluation team in conjunction with other anecdotal information from other individuals engaged in the project (e.g., instructional designers, faculty). This ongoing collaboration contributed greatly to the effectiveness of the overall project, including expansion of the utility of resources made available through the grant. Equipment and facilities either acquired through, or enhanced by, the award were able to be utilized in other capacities beyond, and in addition to, their original intent. This rather unexpected phenomena greatly enhanced the impact of the project on the university s ability to implement new and innovative instructional techniques and methods that were not originally planned as part of the grant-funded activities. Collaborations across departments and colleges often enabled the framing of new synergistic initiatives, which resulted in improved student and faculty collaboration, service, support, and enrichment. Examples of some of the unexpected programs and processes that resulted included: (1) teaming of athletic training program from the college of Education and health sciences colleges during the planning and implementation of the medical workbench/pda solution; (2) leveraging of the flexible mobile science field classroom as a field multi-media production lab for music educators; (3) repurposing of learning objects and entire courses from the degree programs into other courses and certificate programs; and (4) submission of new grants which incorporated acquired resources and newly developed products ( e.g. the development and implementation of the first section 508 ADA compliant course at USF). The ability to implement solutions for the entire university often enabled the grant PIs to obtain a negotiating advantage from corporate vendors who provided in-kind collaborations in exchange for the opportunity to further test and

13 Evaluation Framework 13 develop their products within a large academic environment. These examples, along with outcomes from other initiatives conducted under the grant, both intended, and unintended, have facilitated a sustainable infrastructure that will continue to advance the university s commitment to improving student access. In addition to these university-wide benefits, important contributions to the education and development of doctoral students resulted from the evaluation of the project. Students pursuing a Ph.D. in Educational Measurement and Research and those pursuing a Ph.D. in Instructional Technology received extensive hands-on experience in instrument development, sampling, data collection and data analysis. In addition, graduate students participating in this evaluation assisted with writing and delivering professional papers based on the work. Finally, two doctoral students used the research components of this grant as topics for their dissertations. Of course with any endeavor, especially one of this magnitude, anticipated as well as unanticipated challenges were encountered. For example, in the first two years the grant focused on implementing satellite technology to broadcast via two-way audio and one-way video course content synchronously. However, as other technologies were perfected and became more affordable, teaching strategies were redirected to incorporate more affordable or user friendly technologies such as web-based multi-media solutions, PDAs, and the synchronous software. The ability to overcome issues like this was, at least in part, due to the flexibility of the type of award that supported this project as PIs were given the latitude to make decisions to adjust their original proposal to better meet the intent of the grant using advancements made post-award. These adjustments were then incorporated into the evaluation efforts. Other issues encountered by the evaluation team specifically included some of the more typical problems that occur in evaluation (e.g., lack of responses from program participants to surveys) as well some that reflect the realities of relying on technology. For example, the development of surveys for both students and instructors went well, however delivery of these surveys through the Internet proved problematic. After three semesters of trying to rectify issues encountered through the delivery of these instruments through means readily available through the university, the team finally resorted to purchasing a license to deliver these instruments through an outside vendor. The cost-effectiveness of this move proved to be very worthwhile when the personnel time that had been used to try and rectify issues with local servers and programming far exceeded any reasonable amount of time and cost that should have been associated with these activities.

14 Evaluation Framework 14 The contribution of the in-depth and intensive evaluation efforts undertaken in this program were instrumental in helping program management adapt and expand resources and processes developed under this grant to serve a greater span of faculty, students and programs and serves as a positive example of what can happen when evaluation is an integral part of a project and used in a formative sense. Although those unfamiliar with what goes into a rigorous evaluation may, at first, be unsure of the cost benefits of a strong evaluation effort, the outcomes of a project that uses evaluation to inform ongoing efforts, as well as provide overall summative information, will find that it is very likely that they will reap both intended and unintended benefits. Significance for Evaluation in Education This study provides a foundation upon which to build and develop necessary tools and plans that can be readily adapted by a host of individuals, groups, and key stakeholders involved in higher education. In addition to awareness of tools and processes needed to examine the effectiveness of evolving instructional delivery modes, educators at all levels need to be cognizant of potential pitfalls and detrimental practices to avoid, or, if necessary, compensate for them. The ability to identify such pitfalls as well as identify strengths relies, at least in part, on a substantive evaluation effort. This effort must include identification and utilization of appropriate experts, both in applicable content areas as well as those in measurement and evaluation. Furthermore, evaluation members and project managers must engage in ongoing discussions and feedback to permit timely adjustments to the project as needed. In this era of increasing accountability but shrinking budgets, it may be difficult to allocate resources necessary to do evaluation well; however, this project has shown itself to be a very good illustration of the benefits that strong evaluation can provide and enforces the concept that real program improvement requires appropriate and rigorous evaluation.

15 Evaluation Framework 15 References Dick, W., Carey, L., and Carey, J. O. (2001). The systematic design of instruction. Boston, MA: Allyn & Bacon. Eaton, J. S. (2002). Maintaining the delicate balance: Distance-learning, higher education accreditation, and the politics of self-regulation. American Council on Education. Washington, DC. Hess, M. R., Kromrey, J. D., Lang, T., Hogarty, K. Y. & Phan, H. (2004, November). Evaluating Online Instruction at a Major Research University: Assessment of Processes and Products to Facilitate Development. Paper presented at the annual meeting of the Florida Educational Research Association, Tampa. Hess, M., Kromrey, J. D., Phan, H., Subramaniam, C., Carey, L., Hogarty, K., Schullo, S. & Smith, G. (2004, February). Effective evaluation of online courses in higher education: The development and validation of tools and processes to meet changing needs. Annual meeting of the Eastern Educational Research Association. Hess, M., Carey, L. & Kromrey, J. D. (2004, November). Graduate Degree Programs Delivered via Distance: Development of a Comprehensive Evaluation Process and Protocol. American Evaluation Association, Atlanta. Hogarty, K. Y., Kromrey, J. D., Barron, A. E., Carey, L. M. & Hess, M. R. (2004, June). Development, Delivery, and Effectiveness: Evaluation of Innovative Online Instruction at a Research University. National Educational Computing Conference, New Orleans. Joint Committee on Standards for Educational Evaluation (1994). The Program Evaluation Standards. Thousand Oaks, CA: Sage Publications. Krathwohl, D. R. (1998). Methods of educational and social science research. New York: Longman. Kromrey, J. D., Hogarty, K. Y., Hess, M. R., Rendina-Gobioff, G., Hilbelink, A., Lang, T. (2005, January). A Comprehensive System for the Evaluation of Innovative Online Instruction at a Research University: Foundations, Components and Effectiveness. Paper presented at the College Teaching and Learning Conference, Lake Buena Vista, Florida. Kromrey, J. D., Hogarty, K. Y., Barron, A. E., Schullo, S., Hilbelink, A., Singh, O. & Venable, M. (2004, November). Innovative Online Instruction at a Research University: Development

16 Evaluation Framework 16 and Implementation of a Comprehensive Evaluation System. Association for the Advancement of Computing in Education: elearn 2004, Washington, D. C. Kromrey, J. D., Phan, H., Hess, M., Subramaniam, C., Carey, L., Hogarty, K., Schullo, S. & Smith, G. (2004, February). A Framework for Evaluation of a Large Scale Technology Innovation Project: Lessons Learned in a Higher Education Initiative. Annual meeting of the Eastern Educational Research Association. Schullo, S., Barron, A. E., Kromrey, J. D., Hogarty, K. Y. & Hilbelink, A. (2004, November). A Case Study on the Use of Synchronous Web-based Course Systems in Online Classrooms at a Research University. Association for the Advancement of Computing in Education: elearn 2004, Washington, D. C. Schullo, S., Kromrey, J. D., Barron, A. E., Hogarty, K. Y., Hilbelink, A., & Venable, M. (2004, November). Synchronous Web-based Course Systems to Augment Distance Learning: A Comparison of Two Packaged Programs. Association for the Advancement of Computing in Education: elearn 2004, Washington, D. C. Smith, G. F., Kromrey, J. D., Barron, A. E., Carey, L. M., Hogarty, K. Y. & Hess, M. R. (2004, March). Assessing the Pedagogical and Technological Quality of Online Courses. Society for Information Technology and Teacher Education, Atlanta. Stufflebeam, D. L. (2002). CIPP evaluation model checklist: fifth installment of the CIPP model. Kalamazoo, MI: The Evaluation Center.

17 Evaluation Framework 17 Acknowledgements This work was supported, in part, by the University of South Florida and the Fund for the Improvement of Postsecondary Education, under Grant No. P339Z $2,774,950 in federal funds were provided for the project, representing 50% of the total project costs. The remaining 50% of the project costs ($2,774,950) were financed by nonfederal sources. The opinions expressed are those of the authors and do not reflect the views of the United States Department of Education or the University of South Florida.