Program Level Assessment Report for 2012-2013 PROGRAM NAME, DEGREE NAME (e.g. Organizational Leadership, B.S.):_Reading Masters, M.Ed. COLLEGE in which PROGRAM is housed:_college of Education, Health, & Human Services REPORT PREPARED by:_hannah Chai and Lee Welz, Program Co-Directors A. ACTIONS TAKEN TO IMPROVE STUDENT LEARNING What actions did you take in 2012-2013, based on previous assessment findings, to improve student learning in your program? (Refer back to plans indicated in Response to Assessment Findings in 2011-2012 Assessment Report.) Strengthened and refined the assignments with regards to the details and requirements for all Key Assessment courses in the Reading Endorsement and Reading Master s Program. New rubrics that are aligned with 2010 IRA Standards were created for all Key Assessment courses. Key Assessment Courses ED7060 Theoretical Foundations of Reading ED7220 Teaching Writing, K-12 ED7450 Content Reading Instruction K-12 ED7560 Practicum 1: Intervention for At-Risk Rd. ED7650 Practicum 2: Professional Learning & Leadership ED7810 Literacy Research Seminar Synthesis Project Child Case Study Key Assessment Assignments Literacy Handbook At Risk Reader Assessment: Instructional Portfolio Coaching and Outreach Project Literacy Research Capstone Project B. STUDENT LEARNING OUTCOMES ASSESSED AND EXAMINED Which Program Level Student Learning Outcomes did you assess and examine during 2012-2013? List the Program Level Student Learning Outcomes using the format of Graduates will be able to. (Please note that due to specialized accreditation requirements, accredited programs may be required to assess and report on all program level student learning outcomes every year; accredited programs should report in a manner that will align with their accreditation. Programs not carrying specialized accreditation may assess all of their learning outcomes every year but may choose to report on 2-3 per year, looking at several years of data.) The Reading Endorsement and Reading Master s Program use and continually assess the 2010 IRA Standards: Standard 1: Candidates will understand the theoretical and evidence-based foundations of reading and writing processes and instruction. Standard 2: Candidates will use instructional approaches, materials, and an integrated, comprehensive, balanced curriculum to support student learning in reading and writing. Standard 3: Candidates will use a variety of assessment tools and practices to plan and evaluate effective reading and writing instruction. Standard 4: Candidates will create and engage their students in literacy practices that develop awareness, understanding, respect, and a valuing of differences in our society. Standard 5: Candidates will create a literate environment that fosters reading and writing by integrating foundational knowledge, instructional practices, approaches and methods, curriculum materials, and the appropriate use of assessments. Standard 6: Candidates will recognize the importance of, demonstrate, and facilitate professional learning and leadership as a career-long effort and responsibility.
C. METHODS FOR COLLECTING DATA Which students were included in the assessment? (For example, all seniors completing Course X in Spring 2013, all graduating seniors, etc.) The data was collected from all Reading Endorsement and Reading Master s Program candidates who have completed the Key Assessments and met the Key Assessment requirements in the following courses: ED 7060, ED7220, ED7450, ED7560, ED7650, and ED7810. D. ASSESSMENT MEASURES What key assessments/assignments/student work did you examine to directly assess the Program Level Student Learning Outcomes listed above? What, if any, indirect assessments (e.g. exit survey, alumni survey, focus groups, etc.) did you use to indirectly assess the Program Level Student Learning Outcomes listed above? Type and Number of Assessment Key Assessment Courses Key Assessment Assignments Assessment 1: Licensure assessment Praxis/Pearson Reading Subtest 1 & 2 Assessment 2: Assessment of content knowledge Assessment 3: Assessment of candidate s ability to plan instruction Assessment 4: Assessment of practicum Assessment 5: Assessment of candidate s effect on student learning Assessment 6: Additional assessment addressed by IRA Standard 6 Professional Learning and Leadership Assessment 7: Additional assessment addressed by IRA Standard 6 Professional Learning and Leadership ED7060 Theoretical Foundations of Reading ED7450 Content Reading Instruction K- 12 ED7560 Practicum 1: Intervention for At-Risk Rd. ED7220 Teaching Writing, K-12 ED7650 Practicum 2: Professional Learning & Leadership ED7810 Literacy Research Seminar Synthesis Project Literacy Handbook At Risk Reader Assessment: Instructional Portfolio Child Case Study Coaching and Outreach Project Literacy Research Capstone Project E. SIGNIFICANT FINDINGS What did you find from your assessments? What did your data reveal about how well students are achieving the Program Level Student Learning Outcomes that you listed above? Standard 1: Candidates will understand the theoretical and evidence-based foundations of reading and writing processes and instruction. Standard 1 was addressed within ED706 Synthesis Project and ED7810 Capstone Project. ED706 Synthesis Project data revealed that 10 students (Fall 2012 and Spring 2013) met the criterion in displaying foundational knowledge of reading and writing supported by theories and research. In ED7810 Capstone Project, candidates displayed their theoretical and foundational knowledge through an action research inquiry project. (See data chart below.) Standard 2: Candidates will use instructional approaches, materials, and an integrated, comprehensive, balanced curriculum to support student learning in reading and writing. ED745 Literacy Handbook Project data showed that 18 candidates were on target and 3 candidates were found acceptable in providing evidence of designing instructional materials that supported and motivated candidates learning. The Literacy Handbook was the compilation of a balanced curriculum soundly situated and supported by research and theory. (See data chart below.) Standard 3: Candidates will use a variety of assessment tools and practices to plan and evaluate effective reading and writing instruction. ED7560 At-Risk Reader Assessment and Instructional Portfolio data provides evidence of (See data chart below.) Standard 4: Candidates will create and engage their students in literacy practices that develop awareness, understanding, respect, and a valuing of differences in our society.
ED712 Literature Instruction for Diverse Learners provides evidence that 7 candidates were deemed Acceptable or on Target in creating a literate environment that fostered and supported the needs of diverse learners. (See data chart below.) Standard 5: Candidates will create a literate environment that fosters reading and writing by integrating foundational knowledge, instructional practices, approaches and methods, curriculum materials, and the appropriate use of assessments. Data from ED745 Literacy Handbook Project supports the integration of theories and research as candidates planned and presented evidence to support student learning in their instructional Literacy handbook. (See data chart below.) Standard 6: Candidates will recognize the importance of, demonstrate, and facilitate professional learning and leadership as a career-long effort and responsibility. ED7650 Coaching and Outreach Project data presents information that 13 candidates were deemed Acceptable or on Target in working with others through an Outreach and Coaching project. (See data chart below.) Data Chart Key Assessment Data Strengths Weaknesses ED706 Theoretical Foundations of Reading ED745 Content Reading Instruction K-12 ED7220: Child Case Study ED712 Literature Instruction for Diverse Learners ED7560 Practicum 1: Intervention for At-Risk Readers and Writers Fall 2012 7 students met the following criterion: Formatting, Impact, Information Base, and Major Theories. Spring 2013 3 students met the following criterion: Formatting, Impact, Information Base, and Major Theories. Target 18 students met the following criterion: Planning, Implementation, and Coaching. Acceptable 3 students met the following criterion: Planning, Implementation, and Coaching. Exceeds or meets standards 12 students met the required criteria of the key assessment. Professional Behavior: Acceptable 7 students met this Collaboration: Target 1 student met this Collaboration: Acceptable 6 students met this Dispositions: Target 6 students met this Dispositions: Acceptable 1 student met this Presentation: Target 6 students met this Presentation: Acceptable 1 student met this Professional Development: Target 6 students met this Professional Development: Acceptable 1 student met this Summer 2012 100% of students exceeded 13 of the 14 required criteria. Summer 2013 94.44% of students exceeded in 9 of the required criteria. 88.89% of students exceeded in 5 of the required criteria. 5.56% met standards in 3 of the required criteria. Fall 2012 1 student did not meet the following criterion: Formatting, Impact, Information Base, and Major Theories. Spring 2013 1 student did not meet the following criterion: Formatting, Impact, Information Base, and Major Theories. Omitted 12 students omitted the Key Assessment assignment, therefore, they did not meet the following criterion: Planning, Implementation, and Coaching. Did not meet standards 1 student did not meet required criteria of the key assessment. Professional Behavior: Omitted 2 students did not meet this criterion due to an Collaboration: Omitted 2 students did Dispositions: Omitted 2 students did Presentation: Omitted 2 students did Professional Development: Omitted 2 students did not meet this criterion due to an Summer 2012 1 student omitted documentation of clear objectives. Summer 2013 5.56% of students omitted documentation of the 9 required criteria. 11.11% of students omitted documentation of the 2 required criteria.
ED7650 Practicum 2: Professional Learning & Leadership ED7810 Literacy Research Seminar Praxis: Teaching Reading Test Results 2010-2011 2011-2012 Outreach Project: Target Average 11 students Outreach Project: Acceptable Average 2 students Coaching Project: Target Average 10 students Coaching Project: Acceptable Average 3 students Research Exemplary 9 students met the project Results and Discussion Exemplary Average 8 students Results and Discussion Acceptable Average 1 student Presentation Exemplary 2 students met the project Presentation Acceptable 7 students met the project Reflection Exemplary 7 students met the project 2010-2011 Candidates performed and average of 6.3% higher than the state average and 5.5% higher than the national average. 2011-2012 Candidates performed and average of 3.7% higher than the state average and 7.1% higher than the national average. Outreach Project: Not Observed 1 student did not meet the observed project Coaching Project: Not Observed 1 student did not meet the project Reflection Unacceptable 1 student did not meet the project Reflection None (Omitted) 1 student did not meet the project requirements due to an 2010-2011 Candidates lowest scores averaged at 77% in the following two test categories: Alphabetic Principles/Phonics and Word Analysis and Comprehension 2011-2012 Candidates lowest score averaged at 77% in the following test category: Alphabetic Principles/Phonics and Word Analysis F. DISCUSSION OF RESULTS How were results shared? With whom were they discussed? The program directors discussed assessment results with the Assistant Dean of Assessment and Accreditation on November 6, 2013. The program directors discussed strengths and weaknesses of the candidates in the Reading Endorsement and Master s program with the members of the Literacy Committee on December 18, 2013. Key Assessment Assignments and Rubrics were discussed with the Literacy Committee members and administrators, as well as additional faculty and instructors of the courses. Masters Syllabi were revised to reflect the changes of the revised Key Assessment assignments and rubrics. (See H for minutes from the Literacy Committee meetings) Strengths and weaknesses were discussed via email, phone conversation, and face-to-face discussions with appropriate adjuncts and instructors in our program. G. ACTIONS PLANNED TO IMPROVE STUDENT LEARNING Based on what you learned from your assessment of the Program Level Student Learning Outcomes, what actions do the faculty in your program plan to take to improve student learning in your program/area? Describe the steps faculty have taken/will take to use information from the assessments for improvement of student performance and the program. List additional faculty meetings or discussions and planned or actual changes to curriculum, teaching methods, approaches, or services that are in response to the assessment findings. Areas for Improving Student Learning Praxis/Pearson Reading test results revealed that candidates were weaker in Alphabetic Principles/Phonics and Word Analysis subcategories Plans for Improving Student Learning For candidates missing Phonics as a part of their Reading Core, Phonics and Word study will facilitate closer understanding of alphabetic principles and word study analysis.
Candidates who have omitted evidence for key assessments Instructors teaching Key Assessment courses with candidates who do not meet the criterion, may offer an incomplete so that candidates have time and opportunity to submit the necessary evidence documenting their learning. Issues with data collection: Omitted key assessment data Consistent key assessment data collection for 2013-2014 H. SUPPORTING DOCUMENTS (recommended) Please attach minutes of program faculty meeting where discussion of results and action planning occurred and any other relevant documents. Reading Committee meeting minutes April 3, 2013. Pertaining excerpts from the meeting minutes below. TK20 Key Assessment plan for 4/17 working meeting. Bring old rubrics from Tammy s packet. Bring in IRA 2010 Standards align to IRA Standards Richelle will bring a model 7810 to share. 7060-Hannah & Phoebe will work on 7220-Richele & Lee will work on 7450-Anna & Sue will work on 7560 and 7650 will still need to be worked on after by Hannah and Lee Moving Forward with Reading Agenda shared with the Chair of Teacher Education Department on July 25, 2013, Reading Committee on July 29, 2013, and the Dean of the College of Education and Human Services on August 13, 2013. Pertaining excerpts from the meeting notes below.