Alternative and Performance-based Assessment in Online Learning: Analysis, Framework, and Strategies to Systematically Assess Student Learning

Transcription

1 Alternative and Performance-based Assessment in Online Learning: Analysis, Framework, and Strategies to Systematically Assess Student Learning Mahnaz Moallem, Ph.D. The University of North Carolina at Wilmington AECT 2004 Conference Oct , Chicago, IL

2 Purposes of Presentation Analyze Student-centered, Project-based, Performance-based Assessment system (SPPA) to identify its implementation problems in face-to-face courses Identify potential of online learning environment for implementation of SPPA system Propose a design framework for SPPA system in online courses Demonstrate the application of the model using an example online course Share some of the enablers and barriers

3 A Conventional Teacher-Centered & Outcome-based Assessment Model Outcome Instruction Assessment Establish Learning Outcomes Identify & Develop Assessment Implement Instruction Assess Learning Outcomes & Effectiveness of Instruction Re-mediate and/or Revise Instruction

4 Student-centered, Project-based, Performance-based Assessment system Supp orts measur ing learning m ore authent ically Develops a deeper dialog over the course materials Foster both individual and group oriented learning activities Provides a diversity of learning opportunities Enables s tudents to display critic al thinking skills and greater depth of knowledge And most importantly

5 Student-centered, Project-based, Performance-based Assessment system Emphasizes: Integrating assessment & instruction Progress in learning & in becoming more sophisticated learners Pedagogical changes in teaching methods

6 Student-centered, Project-based, Performance-based Assessment system Real World Problem or Project Continuous Assessment Self-Assessment/ Feedback Performance-based, Process-based, Progress-based Assessment Peer-Assessment/ Feedback Expert-Assessment/ Feedback Continuous Assessment Continuous Assessment

7 SPPA System in Face-to-Face Classrooms: Implementation Problems While the concept of dynamic & continuous assessment is appealing it is difficult to manage timely and formative feedback

8 SPPA System in Face-to-Face Classrooms: Implementation Problems Limitation of time, tools & resources for gathering & documenting dayto-day assessment data constrains the effectiveness of SPPA system

9 SPPA System in Face-to-Face Classrooms: Implementation Problems It is difficult to: Be an effective facilitator Ask appropriate guiding questions Document student progress when solving problems/projects

10 SPPA System in Face-to-Face Classrooms: Implementation Problems It is difficult to: Promote individual students growth without providing individual assessment Keep the focus of individual assignments on selfassessment

11 SPPA System in Face-to-Face Classrooms: Implementation Problems Self-assessment tools do not seem to work effectively without the instructor s feedback & guidance. It is difficult to document the effects of self-assessment tools on individual growth & team performance

12 SPPA System in Face-to-Face Classrooms: Implementation Problems Given limitation of time and tools, using multiple sources of assessment is difficult to manage

13 SPPA System in Online learning Environment Online learning environment: Has the highest potential for real world, authentic performance assessment Offers new tools and resources that make implementation of SPPA a reality Provides a much better environment for practicing duties of a mentor, coach and/or facilitator

14 Online Learning Environment: New Tools & Resources Database and interactive system to track, monitor & document students activities automatically Easy access & easy development process for conventional assessment tools

15 Online Learning Environment: New Tools & Resources Opportunity to selfassess own knowledge through automatic and instant feedback Multiple communication tools to facilitate as well as to document dialogue

16 Online Learning Environment: New Tools & Resources Content & communication tools to: Develop projects/tasks that could incorporate multiple resources & perspectives Complete project/tasks at various intervals Provide continuous formal & informal feedback

17 Online Learning Environment: New Tools & Resources Unlimited & self-paced access to course materials & communication tools Interactive & asynchronous communication tools to promote thoughtful & reflective commentaries Learner s & instructor s access to the record of responses, answers, feedback, etc. to analyze and interpret one s own performance & to reflect

18 A Model for Assessment of Online Learning Real World Project or Problem Expert- Assessment Self- Assessment Expert- Assessment Self- Assessment Expert- Assessment Self- Assessment Initial Assessment Progress Assessment Product Assessment Peer- Assessment Peer- Assessment Peer- Assessment

19 Designing Online SPPA System: An Example

20 Product Assessment: Designing Real World Problem/Project The first step was to develop a complex and multifaceted real world problem or project that required students to Demonstrate achievement of all course goals Engage in challenging real world activities Work in teams Work with real clients and/or the professional community

21 Product Assessment: Designing Real World Problem/Project Made sure that the real world problem/project was complex enough to: Break it into several subtasks or mini projects to be completed at various intervals Monitor its subtasks Structure the accomplishment of the major task Develop a timeline for student work Incorporate process-based, continuous assessment & feedback

22 Product Assessment: Designing Self Assessment Tools A series of self-assessment rubrics was developed (self assessment) One or two completed projects were linked to the course project to help set the standards for best performance (selfassessment) The instructor reviewed teams progress and product and provided written feedback (expert assessment) The instructor scheduled a chat session with each team and reviewed the written comments at least once for each part of the project (expert assessment)

23 Product Assessment: Designing Self Assessment Tools Provided an online space for students to work on their course project continuously, and in collaboration with their team members throughout the semester (peer assessment) Used Student Presentations area within WebCT course management system (peer & expert assessment)

24 Assessing Progress The next step in designing SPPA system was to develop related problems or cases for each unit of instruction to: Provide learning experiences for the completion of the course project Address domain specific knowledge and skills to complete the course project Identify & further developed resources and supporting instructional materials Develop and form unit content and instruction

25 Assessing Progress: Expert, Peer & Self- Assessment Tools Self-assessment quizzes to assess background knowledge with automatic expert feedback (self & expert assessment) Individual assignments and forum discussion to assess understanding and synthesis of readings and domain specific knowledge with immediate expert and peer feedback (Peer & Expert assessment) Conversation space for teams working on unit problems (peer & expert assessment) Team evaluation instruments to assess team s performance (peer assessment) Opportunity to review and compare other teams products and feedback (peer assessment)

26 Initial Assessment The third step was to develop opportunities for assessing students initial understanding of the materials in order to assist: Students in making their initial thoughts explicit to themselves Students compare their initial understanding with their later conceptualization of the materials & identify progress The instructor to identify coaching and guiding strategies

27 Initial Assessment: Self, Peer & Expert Assessment Tools Individual assignments that consisted of a list of thinking questions Non-graded, self-assessment quizzes with immediate feedback using WebCT Quiz Maker Team Forum discussion Learning, thinking & multiple intelligences inventories to identify one s own learning and thinking styles

28 Reflections: Enablers Use of SPPA Systems made assessment a natural extension of instruction Non-graded and reusable instructional activities gave both students and the instructor useful feedback on teachinglearning process Constant feedback enabled students to be strategic in their own learning process and enabled the instructor to adapt the instructional process to meet the needs of students

29 Reflections: Enablers Assessment of actual performance enabled students to translate what they had learned in different units of instruction to a real-life application Within SPPA system it was possible to provide a wide range of assessment tools and techniques SPPA offered more opportunities and ways for providing feedback SPPA made the management, collection and transfer of assessment information easier

30 Reflections: Barriers Accessibility and responsiveness of the instructor Instructor s philosophy of learning and assessment and his/her expectations of students Fostering a participatory online learning environment