ASSESSING THE ONLINE LEARNER. Rena Palloff and Keith Pratt

Transcription

1 ASSESSING THE ONLINE LEARNER Rena Palloff and Keith Pratt

2 Welcome Rena Palloff and Keith Pratt

3 Learning Outcomes for This Session After reviewing this pre-recorded session, you should be prepared to: Align course design with assessment strategies. Describe characteristics of online assessment. Select appropriate assessment methods to determine student competency levels. Develop an online assessment activity and a rubric to evaluate student performance. 3

4 Start with Your Learning Outcomes What a student will be able to know or do at the end of a learning unit or course Measurable Observable action or behavior 4

5 Then Move to Learning Objectives Unit or activity based contribute to the achievement of outcomes. Describe cognitive skills achieved (i.e., what the student will LEARN. Once you know what you want your students to be able to know, do, or learn, you can begin to design activities and assessments. 5

6 Cycle of Course Design Purpose of Course (Course Outcomes & Competencies) Student Assessments Learner Characteristics Course Activities Learning Objectives

7 Designing Courses That Align with Assessment Provide clear guidelines and objectives. Tasks and assignments should be relevant not only to the subject matter, but to students lives as well. Students should understand what is expected of them. Assessment should be in alignment with the course as a whole and is not to be seen as a separate and cumbersome task. When the activities and assessment techniques are in alignment, instructors and students alike are more satisfied with the outcome of the learning process. 7

8 Assessment Basics The following are some principles that should guide student assessment in an online course: Design learner-centered assessments that include selfreflection. Design and include grading rubrics for the assessment of contributions to the discussion as well as for assignments, projects, and the collaboration itself. Include collaborative assessments through public posting of papers along with comments from student to student. 8

9 Assessment Basics The following are some principles that should guide student assessment in an online course: Encourage students to develop skills in providing feedback by providing guidelines to good feedback and by modeling what is expected. Use assessment techniques that fit the context and align with learning objectives. Design assessments that are clear, easy to understand, and that are likely to work in the online environment. Ask for and incorporate student input into how assessment should be conducted. 9

10 Assessment Basics Angelo and Cross (1993) note that effective classroom assessment should be: Learner-centered Teacher-directed Mutually beneficial Formative Context-specific Ongoing Firmly rooted in good practice 10

11 Differences Between Online and Classroom Assessment Online Involvement in online discussion can improve reading/writing skills. Engagement in discussion can deepen knowledge acquisition and critical thinking skills. Technology can be used to assist in the assessment process. Classroom Can improve reading/writing skills if assessment involves written assignments beyond tests/quizzes. Active learning techniques can deepen knowledge acquisition. Technology is not often used for assessment.

12 Assessment in Online Courses Six key qualities for assessment of online students (Morgan and O Reilly, 1999): A clear rationale and consistent pedagogical approach Explicit values, aims, criteria, and standards Authentic and holistic tasks 12

13 Assessment in Online Courses Six key qualities for assessment of online students (Morgan and O Reilly, 1999): A facilitative degree of structure Sufficient and timely formative assessment Awareness of the learning context and perceptions 13

14 Ways to Measure Knowledge Acquisition Use good questioning techniques to review text and other reading material Rubrics Self-assessments Peer assessments Authentic assessments 14

15 Tests and Quizzes Online Generally used as the first resource for online instructors because: Course management systems contain test and quiz functions. Instructors may have limited time or resources available to construct other forms of assessment (Rocco, 2007). 15

16 Why Tests and Quizzes May Not Be the Best Choice May not align with course objectives or mode of instruction. The need to write good questions not all instructors do this well! Generally measure knowledge acquisition but not application. May actually encourage cheating. 16

17 Alignment of Objectives with Exam Questions Rocco (2007) provides the following alignment of test question types with desired course outcomes: Objective Level Multiple Choice True/False Matching Completion Short Answer Short Essay Long Essay Knowledge X X X X X Comprehension X X Application X X Analysis X X Synthesis X X Evaluation X X 17

18 Use Quizzes as a Form of Self- Assessment Practice tests and quizzes align well with the use of final exams online. Helps students understand what will be expected of them in terms of test questions and how to use the technology for the test. Practice tests and quizzes improve student performance in the course (Michigan State University, 2005). 18

19 Questioning Techniques Use open-ended questions that encourage students to talk to one another about what they ve read. Use questions that encourage further research outside of assigned reading. Use questions that encourage application of material read, such as case studies, scenarios, etc. As the instructor, do not respond to every post. Strategically jump in to link or extend learner discussion. Do assess learner contributions to discussions! 19

20 Rubrics Should support self-assessment, selfmonitoring, and instructor assessment Should be based on Bloom s taxonomy, and criteria for each level of response assessed should be clear 20

21 Rubrics Define the performance levels for each gradable activity element. Provide students with a concrete way of evaluating their own performance as well as the performance of the members of their team in collaborative activity. 21

22 Types of Rubrics Holistic Rubrics Measure overall course performance Analytic Rubrics Break performance or product into its component parts Primary Trait Rubrics Similar to analytic rubrics with the exception that they measure a common trait across all categories (i.e., participation in the online course) 22

23 Basic Discussion Rubric Summary Level Posts = C Analytic Level Posts = B Synthesis Level Posts (i.e., You made me think! ) = A B l o o m s T a x o n o m y 23

24 Using Rubrics in Assessment The following is an example of how to use a rubric for online discussion: Attached is a rubric for participation in online discussions. The way to use this rubric is to look at the gradable items listed down the lefthand side of the grid. Across the top are the performance levels, which roughly (and I do mean roughly!) equate to grades. To earn an A, for example, you would not need to fall in all of the boxes down the righthand column your performance in discussions might be extremely strong with a combination of items from the TWO columns on the right (proficient and distinguished) and maybe even one item that you feel shows basic performance. The idea is to look at the total picture of your discussion performance. 24

25 Using Rubrics in Assessment, cont. So, at mid-term and again at the end of the term, what I want you to do is to send me an (I'll nudge you at mid-term) telling me where you think you fall on the rubric. For example, you might say that for including and applying course concepts and theories, you feel that you fall into the proficient category while you feel you are distinguished in applying real-world examples and applications of the theories. You'll go through each category and tell me where you think you fall on the grid. I don't generally give mid-term grades, so what I'll be doing is responding to your with my own assessment based on your self-assessment. I hope that's not too confusing! If there's a discrepancy in our perceptions, we'll discuss it. If we agree, then great! At the end of the term, I'm going to ask you to do this again. But, this time, I'm going to ask you to tell me what grade you think you've earned as well. Questions about this? Please ask! 25

26 Determining Performance Levels for Activities What evidence can students provide that would show they have accomplished what you want them to? What are the highest expectations for student performance? What is the worst performance you can imagine beyond not doing the assignment at all? 26

27 Determining Gradable Activities Activities should directly relate to course learning objectives. All course activities should relate to one another. Activities should be based on previous experience with like assignments. Activities should relate to the skill set of students entering the course or should help develop new skills. 27

28 Self-Assessments Depending on the level of the learners, criteria can be instructor-developed, peer-developed, or independently developed. Work best with some guiding questions. Can be used effectively for individual activities as well as overall performance in the course. 28

29 Peer Assessments After collaborative activity, can be used to debrief Can be in the form of a private to the instructor including both self-assessment and assessment of all other members of the group and team Can include suggested grades or simply be in narrative form 29

30 Authentic Assessments Generally take the form of application activities such as simulations, role playing, or use of case studies. Portfolios can illustrate ongoing knowledge acquisition and application. Effectively demonstrate not only acquisition of knowledge but ability to apply that knowledge in professional or other settings. Effectively assessed through the use of rubrics. 30

31 Using the Online Environment for Assessment Advantage Computer generated/scored tests and quizzes Internet-based research projects Peer review and assessment technologies Internet-based case studies Synchronous technologies to facilitate collaboration 31

32 Map Your Outcomes to Assessment Activities! Learning outcomes (include: knowledge, cognitive, key and subject specific skills) select core learning outcomes that appear across units it is not necessary to list all learning outcomes. Learningteaching activity You will need this for the learning-teaching-assessment cycle part just give idea of different teaching and learning used to accommodate a learning outcome. Type of assessment E.g., essay, group work, presentation, work-based, extended project, etc.

33 NEXT Develop an assessment activity and a rubric you can use to evaluate it!

34 Thank you! LIVE SESSION: TUESDAY, JUNE 26 12:00 2:30PM EST