Evaluating Students in the Classroom Faculty Development Conference February 2007 D. P. Shelton, MSN, RN, CNE, EdD (c)

Transcription

1 Evaluating Students in the Classroom 2007 Faculty Development Conference February 2007 D. P. Shelton, MSN, RN, CNE, EdD (c)

2 Objectives Describe assessment strategies for use in the classroom. Explore the benefits and weaknesses of selected assessment strategies. Develop an assessment for a nursing course.

3 What naturally attracts students

4 Aligning grades and goals

5 Evaluation OR Assessment? Integrate assessment with teaching and learning Don t separate testing from grading

6 Teaching is a process of effective decision making Deciding What to teach How to teach it How long to teach it Whether to group students What questions to ask What follow-up to ask What to review When to review Etc. (McMillan, 2001)

7 Assessment Assessment is the process of collecting, synthesizing, and interpreting information to aid in decision making (Airasian, 1997) Four major, sequential components Purpose Method of measurement or description Evaluation Use

8 Purpose Assessment begins with identification of the specific purpose for collecting and interpreting the information. Once the purpose has been identified, appropriate methods for gathering and synthesizing the information can be identified.

9 To identify if students have mastered a concept or skill To motivate students to be more engaged in learning. To communicate expectations to students To give students feedback To encourage student self-evaluation To evaluate the effectiveness of instructional approaches (McMillan, 2001)

10 Method of Measurement or Description Quantitative or qualitative assessment? Selected-response (objective) or constructed response (subjective)? A balanced assessment approach is needed.

11 Evaluation Judgment on the worth or value of the performance. Interpretation of what the results mean and how they can be used.

12 Use How the information gathered is used to make decisions about students, instruction, curriculum, faculty and the program. Inform faculty on how much students know and understand Documenting what students have learned Accreditation Evaluating curriculum

13 Concepts to Consider Validity Reliability Fairness

14 Designing an Assessment Develops naturally Respects students Assesses what students have had the opportunity to learn Provides students with the opportunity to show what they can do Facilitates the instruction process Provides dependable information about students. Smith, Smith, & De Lisi, 2001

15 Higher Level Learning Preparing students to meet the demands of nursing and the changing health care system, nursing faculty are faced with Identifying essential content Mastery of the knowledge Develop cognitive, affective, and psychomotor skills Ability to apply concepts and theories to new situations Problem solving Critical thinking (Oermann & Gaberson, 2006)

16 What are the options? Multiple choice tests True-false tests Short answer tests Essays Case method/study Discussion Debate Papers Projects Demonstrations/ performances Group work Posters Reaction papers Labs Reports Portfolios Journals

17 Case Method, Case Study, Unfolding Cases Analyze a clinical scenario and answer related questions Effective for developing problem solving and critical thinking Small group activity or individual assessment (Oermann & Gaberson, 2006)

18 Concept Maps Mind maps, diagram, schematic Visual format of concepts and their relationships Demonstrates cognitive synthesis skills with minimum writing Gain insight into students thinking Can be difficult to follow Billings & Halstead, 2005

19 Discussion Individually or in small groups Evaluating problem solving, decision making and critical thinking abilities Higher level questioning with clarifying questions Socratic method of questioning (Oermann, 2004)

20 Debate Evaluating student s ability to analyze problems and issues in depth Consider alternative points of view Formulate a position

21 One Minute Papers Obtains regular feedback from learners about their learning Instructors can benefit from ongoing assessment of classroom learning Two main questions What is the most important thing you learned today? What question(s) do you still have in mind? (Angelo & Cross, 1993)

22 Muddiest Point Simple technique Can obtain data related to a particular topic Instructor determines what area of content to be assessed Reserve a few minutes at break or end of class Index cards or slips of paper distributed What was the muddiest point in? Collect responses and analyze during break or before the next class session Instructor responds to student feedback as soon as possible (Melland & Volden, 1998)

23 RSQC2 More cumbersome and results take longer to analyze RECALL beginning of class list what they recall from the previous class SUMMARIZE summarizes as many of the most important points into one written summary statement QUESTIONS students write one or two remaining questions from the previous class CONNECT in one or two written sentences students explain how the main point of the previous class connect with the overall course objectives COMMENT students are invited to write one or two evaluative comments about the class (Melland & Volden, 1998)

24 Papers Can achieve many learning outcomes Carefully select and design the assessment Purposes Critique, integrate and report on literature findings Analyze concepts and theories and apply to situations Improve their problem-solving and critical thinking skills Gain experience in formulating their ideas and communicating them to others Develop writing skills Concern - reliability

25 Types of Writing Assignments In the classroom - Term papers Research papers Research critique Evidence-based practice paper Theory papers Reaction papers Case study analysis Journaling Critiques

26 Evaluating Written Assignments Develop predetermined criteria Address quality of content Organization of ideas Process of arriving at the decisions Writing style Format

27 Other Tips Include clear directions about the purpose and format of the assignment Develop specific criteria for evaluation and review with students before hand Read all papers anonymously Skim to gain an overview of how the students approach the paper Read in random order Read each paper twice before scoring If unsure about grade, have colleague read

28 Journaling Increase student s self-awareness through contemplation and reflection Can be very structured with specific components Paper or online formats Providing feedback is essential Time consuming High degree of trust

29 Seven Components for Responding to Journaling Praising Describing Diagnosing Judging Predicting and reviewing growth Record keeping Recognizing/praising growth Billings & Halstead, 2005

30 Portfolios Organized collection of evidence that shows a student s progress, special achievements, skills and attitudes over time. A nursing course or a program of study Serve a variety of purposes Linn & Miller, 2005

31 Key Steps Specify the purpose Provide guidelines for selecting portfolio entries Define student role in selection and selfevaluation Specify evaluation criteria Use portfolio in instruction and communication

32 Group Work Inevitability of groups/teams Groups influence behavior Groups enhance achievement Groups enhance relationships Groups enhance social skills Groups help make assessment meaningful Groups empower teachers to enlarge the scope of assessments Group enables the assessment of a wider range of outcomes Johnson & Johnson, 2004

33 Reasons for Group Work Peer learning can improve the overall quality of student learning Group work can help develop specific generic skills sought by employers Group work may reduce the workload involved in assessing, grading and providing feedback to students

34 Common Issues and Concerns Lack of perceived relevance, lack of clear objectives Inequity of contribution Overuse

35 Designing Group Activities that Work Providing explicit guidelines Determining group membership Establishing the role(s) and responsibility of group members Scheduling group meetings Defining group processes and procedures

36 Weighing-up the Options for Group Assessment Whether what is to be assessed is the product of the group work, the process of the group work, or both What criteria will be used to assess the aspect(s) of group work of interest Who will apply the assessment criteria and determine marks How will marks be distributed

37 Types of Group Activities Presentations Projects Research critiques Teaching plans Staff development inservice Community assessment Seminar

38 Develop Scoring Rubrics All constructed response assessment require performance criteria Performance criteria guidelines, rules, or principles by which assessment are judged. Describe what to look for in assignments to judge quality. Place the emphasis on criteria for assessing Rubrics Arter & McTighe, 2001

39 Checklist List of components that must be present in the work. Check off components that are present. Simple to use can only use them where a judgment of present or absent is sufficient

40 Performance Lists More sophisticated Consists of things to rate and a rating scale Each element is scored based upon a possible number of points

41 Element Points Possible Points Earned Assessment data 20 Appropriate nursing diagnosis 5 NOC/NOC indicators 10 NIC/NIC activities 10 Evaluation 10 Revisions 15 Format 5 Total 75

42 Rubrics A rubric is a scoring guide that seeks to evaluate a student's performance based on the sum of a full range of criteria rather than a single numerical score. Benefits: Consistency in scoring Improved instruction Types of Rubrics Holistic or analytic trait Task specific or general Number of score points possible

43 Holistic rubrics assess student work as a whole. Holistic rubrics tend to be used when a quick or gross judgment needs to be made. Analytic rubrics identify and assess components of a finished product. Analytic rubrics are more common because teachers typically want to assess each criterion separately, particularly for assignments that involve a larger number of criteria.

44 Analytic Rubric Criteria 4 points 3 points 2 points 1 point Has a plan for Investigation The plan is thorough The plan is lacking a few details The plan is missing major details The plan is incomplete and limited Use of Materials Manages all materials responsibly Uses the materials responsibly most of the time Mishandles some of the materials Does not use materials properly Collects the Data Thorough collection Some of the data Major portions of the data are missing The data collection consists of a few points

45 Generic or general rubric is used across similar performances. Use the same rubric to judge all oral presentations, all critical thinking or all group interactions. Task specific rubrics are rubrics that can only be used for a single task or for a particular method

46 Number of Score Points No hard and fast rules Consider the nature of the assessment and the purpose of the scoring Literature recommends 3 to 6 points Need enough score points to distinguish quality

47 Developing a Rubric 1. Determine precisely the product, process or performance for which the rubric is to be written 2. Determine the intended audience/users 3. Determine the intended purpose, impact, effect, and/or result that is expected to be apparent as a result of the product, performance, or process. (If this were done excellently, what would we observe.) Ater & McTighe, 2001;

48 4. Obtain examples ( exemplar, acceptable, and unacceptable characteristics) 5. Identify the qualitative characteristics that determine the exemplary quality levels for each criteria. 6. Develop the unacceptable quality description for each criteria by modifying/negating each exemplary criteria appropriately. 7. Develop and label appropriately any desired intermediate quality levels for each criteria. 8. Continuously review the rubric

49 Selecting a Strategy Consider philosophy of the faculty regarding accountability and responsibility for learning Clearly delineate the purpose Consider the setting Choose the best strategy for the purpose Determine the procedure for the strategy selected Establish validity and reliability Evaluate the overall effectiveness Match the strategy with the domain of learning Communicate grading expectations Billings & Halstead, 2005

50 References Airasian, P. W. (1997). Classroom assessment. (3 rd ed). New York: McGraw Hill Angelo, T.A. & Cross, P. K. (1993). Classroom assessment techniques: A handbook for college teachers. (2 nd ed). San Francisco: Josey-Bass Publishers. Arter, J. & McTighe, J. (2001). Scoring rubrics in the classroom. Thousand Oaks, CA: Corwin Press Inc. Billings, D.M. & Halstead, J. A. (2005) Teaching in nursing: A guide for faculty. (2 nd ed). St Louis: Elsevier Saunders. Johnson, D. W. & Johnson, R.T. (2004). Assessing students in groups. Thousand Oaks, CA: Corwin Press Inc.

51 Linn, R.L. & Miller, M.D. (2005). Measurement and assessment in teaching. (9 th ed). Upper Saddle River, NJ: Pearson McMillian, J. H. ()2001). Essential assessment concepts for teachers and administrators. Thousand Oaks, CA: Corwin Press Inc Melland, H.I. & Volden, C.M. (1998). Classroom assessment: Linking teaching and learning. Journal of Nursing Education,27(6), Oerman, M.H.& Gaberson, K.B. (2006). Evaluation and testing in nursing education. (2 nd ed). New York, NY: Springer Publishing. Smith, J.K., Smith, L.F., & De Lisi, R. (2001). Natural classroom assessment. Thousand Oaks, CA: Corwin Press Inc