Master of Arts Teaching Mathematics 1

Transcription

1 Master of Arts Teaching Mathematics 1 According to our Assessment work plan written and submitted to PESB in 2008, we are now sending the program report templates to you at the beginning of fall quarter. It is anticipated that the new timeframe will allow departments more time to discuss the data analyses, and more time for department chairs and program coordinators to use those discussions to complete the reports. These reports are due at the end of fall quarter and should be submitted to the OREA (siskob@cwu.edu and copied to depaepej@cwu.edu) before winter break. Although you have done a great job of producing these reports in the past, we hope the new schedule is more conducive for continuous program growth and improvement. In case you are new to our process, the following template contains embedded prompts guided by NCATE and PESB compliance standards. Because advanced programs use distinctly different standards, we ask that you refer to your standards when writing this report. Program coordinators are asked to share these summaries with program faculty to facilitate discussions. The culminating synthesis of those conversations (interpretations and conclusions of the data summaries) should be written in the spaces provided. The primary purpose of the report is to demonstrate how the CTL uses data to update (improve) programs and unit operations, as well as the assessment system.

2 Master of Arts Teaching Mathematics 2 Section 1. Briefly describe the CTL/ or your program s conceptual framework and how it guided the design of your program s assessment process? In other words and similar to the University s Request, note how the outcomes are linked to department, college, and university mission and goals. Be sure to reference how the faculty and professional community in your program were involved in development, implementation, and ongoing evaluation of those measures and processes. The MAT program is designed around the following goals: 1. To enable mathematics teachers to demonstrate broad and deep knowledge in the core mathematical content areas of analysis, geometry, algebra, and statistics. 2. To enable mathematics teachers to foster logical and mathematical thought processes in themselves and in their students. 3. To enable mathematics teachers to foster conjecturing and inventing solution strategies in themselves and in their students. 4. To enable mathematics teachers to foster analytic, synthetic, and evaluative thinking skills in themselves and their students. 5. To enable mathematics teachers to foster problem-solving strategies in themselves and in their students. 6. To enable mathematics teachers to foster connections among mathematical subfields, science, and social science. 7. To enable mathematics teachers to improve their teaching by effectively assessing student work, their teaching, and their programs. 8. To enable mathematics teachers to build and foster learning communities within their professional domains. 9. To enable mathematics teachers to foster equity within their professional domains. 10. To enable mathematics teachers to exercise leadership in mathematics education within their professional domains. These goals were developed in 2007 and vetted among the mathematics education faculty at that time. Since then, they have been augmented for clarity. The MAT Director (Dr. Lundin) developed a survey instrument aligning with the stated goals.

3 Master of Arts Teaching Mathematics 3 Section 2. Please describe your program s assessment protocols. 1) What method(s) do you use to follow the progress of your students? i) MAT summer survey. ii) MAT student assessments by course and instructor iii) MAT Project progress 2) What method(s) do you use to follow-up on the performance of your graduates? (THIS WAS OUR ONLY AREA FOR IMPROVEMENTS CITED IN 2009) i) Math Alumni Survey Results: pmbat6vu_3 3) Is the system course based, end of program based, or other? The methods presented in 1) are course-based and program-based. The method of 2) above is end-of-program. 4) What measures are used and when are they administered? The measures in 1) Summer Program; The measures of 2) annual. 5) How does the program eliminate bias and test for measurement fairness, accuracy, and consistency? In the case of Alumni Survey, anonymity of responses promotes fairness. In the case of the MAT survey, multiplicity of use promotes consistency (reliability). 6) What technologies are employed to compile, aggregate, summarize, and report the data? Web technology (Survey Monkey) was used for the Math Alumni Survey. 7) How is the program assessment process administered (all program faculty or a designee)? i) The MAT Survey is administered by the MAT Director (Dr. Lundin). ii) Alumni Survey is administered through the Testing Center via Survey Monkey. 8) How often do faculty meet to examine data? The Department of Mathematics Chair and the MAT Director solicit responses after results are disseminated annually. 9) How are data shared with candidates to help them reflect and improve? The MAT Director shares results with MAT students during the first week of the MAT summer program. 10) How does the process maintain a record of formal complaints and subsequent resolutions? There have been no formal complaints during the MAT Director s tenure. In the case that they arise, these complaints will reside with the Department of Mathematics Chair, as is department policy. 11) How are these records taken into consideration during discussions of program improvement? Department policy calls for discussion among Chair and other parties for resolution of the complaint. (This response will satisfy CWU Assessment of Student Learning Report # 2 and #3) Type your response here.

4 Master of Arts Teaching Mathematics 4 Section 3. This section is for all advanced candidates. Please summarize how well the candidates of your program are meeting the program s Content Standards. For the advanced teacher preparation programs, summarize candidate achievement of Content Knowledge. For other school professionals programs (administrator, school counseling, and school psychology) summarize candidate achievement of standards which relate to Knowledge and Skills. Be sure to reference State standards as well. Conclude by stating the improvements made or plans to improve the program, which have helped or will help candidates achieve higher levels of Content Knowledge performance. (CWU Assessment of Student Learning Report #3) Type your response here. NOTE: The MAT Director was on sabbatical for the year Consequently, no MAT survey was issued during SUMMER However the director expects to continue the survey process during the summer of Consequently, all conclusions are based on responses of the Alumni Survey administered during the academic year. That items of that survey directly align with the goals of the MAT program. There were 8 respondents: On questions regarding content knowledge, 75%-98% of the candidates responded that they were prepared or very prepared by the MAT program to teach content. Noted Weaknesses: Respondents split opinions on the making connections to science, social science, and technology standard. While 50% felt prepared by the program in this area, the other half felt only Somewhat Prepared.

5 Master of Arts Teaching Mathematics 5 Section 4. This section is only for advanced teacher preparation programs. Please summarize how well the candidates of your program are meeting standards associated with Pedagogical Content Knowledge and Skills. Note: only those standards that integrate content and content specific pedagogy should be cited here. This section relates to the interaction of the subject matter and effective teaching strategies that help students learn the subject matter. It requires a thorough understanding of the content to teach it in multiple ways, drawing on the cultural backgrounds and prior knowledge and experiences of students. For further description read the target level of 1b. page 17 of the NCATE 2008 Standards. For State standards go to: Conclude by stating the improvements made or plans to improve the program, which have helped or will help candidates achieve higher levels of Pedagogical Content Knowledge and Skills performance. (CWU Assessment of Student Learning Report #3) From 75% or more of the respondents agreed that they were Prepared or Very Prepared by the MAT Program in following areas of pedagogy: teaching content, teaching the use of logical and mathematical evidence, teaching conjecturing and problem-solving. Weaknesses: The respondents again split the connections standards: 50% felt Partially Prepared, while the other half felt Prepared to teach making connections.

6 Master of Arts Teaching Mathematics 6 Section 5. This section is only for advanced teacher preparation programs. Please summarize how well the candidates of your program are meeting standards associated with Professional and Pedagogical Knowledge and Skills. This section relates to those program standards associated with historical, economic, sociological, philosophical, and psychological understandings of schooling and education. It also includes knowledge about learning, diversity, technology, professional ethics, legal and policy issues, pedagogy, and the roles and responsibilities of the profession of teaching. For a description read the target level of 1c. on page 18 of the NCATE 2008 Standards.. For State standards go to: Conclude by stating the improvements made or plans to improve the program, which have helped or will help candidates achieve higher levels of Professional and Pedagogical Knowledge and Skills performance. (CWU Assessment of Student Learning Report #3) 1. Of the 8 respondents, 75% stated they were Prepared or Well-Pepared during the MAT program for leadership in mathematics education, while 25% stated they were Somewhat Prepared.

7 Master of Arts Teaching Mathematics 7 Section 6. This section is only for advanced teacher preparation programs. Please summarize how well the candidates of your program are meeting standards associated with Student Learning. These data demonstrate the effects candidates have had on the students they teach. For a full description read the target level of 1d. page 19 of the NCATE 2008 Standards.. For State standards go to: Conclude by stating the improvements made or plans to improve the program, which have helped or will help candidates achieve higher levels of Student Learning performance. (CWU Assessment of Student Learning Report #3) We do not yet gather results on student achievement of the teachers in our MAT program. However, most MAT Projects involve action research. Consequently, they have as their main focus, increasing student achievement.

8 Master of Arts Teaching Mathematics 8 Section 7. This section is only for other school professional programs. Please summarize how well the candidates of your program are meeting standards associated with Student Learning. These data demonstrate how well candidates are prepared to critique and reflect on their work within the context of student learning. How well are candidates prepared to establish environments that support student learning, collect and analyze data related to student learning, and apply strategies for improving student learning within their own jobs and schools. For State standards go to: Conclude by stating the improvements made or plans to improve the program, which have helped or will help candidates achieve higher levels of Student Learning performance. (CWU Assessment of Student Learning Report #3) MAT Projects are action research projects. As such they are designed to promote effective teaching and measure the results of that action. Any of our students who completed the program, by virtue of their MAT Projects, have established an exemplary learning assessment cycle.

9 Master of Arts Teaching Mathematics 9 Section 8. This section is for all advanced programs. Please summarize how well the candidates of your program are meeting the program s Dispositional standards. Candidates are expected to work with students, families, colleagues, and communities in ways that reflect the professional dispositions expected of professional educators. For a full description read the target level of 1g. on page 20 of the NCATE 2008 Standards. Conclude by stating the improvements made or plans to improve the program, which have helped or will help candidates achieve higher levels of achieving dispositional skills of a professional educator. (CWU Assessment of Student Learning Report #3 example 3) Of the 8 respondents, 63% stated they were Prepared or Well-Prepared during the MAT program for promoting learning communities within their professional domains, while 25% stated they were Somewhat Prepared and 12.5% stated that they were Not at All Prepared.

10 Master of Arts Teaching Mathematics 10 Section 9. This section is for all advanced programs AND A WEAKNESS FOUND IN OUR LAST REVIEW. Please describe what measures are used to specifically provide data on candidate s abilities to help students from diverse populations learn? What do the results of these measures reveal? To help you answer this question, please show data relative to how well candidates perform on the following elements: 1) Candidates understanding of diversity, including ELL and students with exceptionalities 2) Candidates abilities to develop and teach lessons that incorporate diversity 3) Candidates demonstration of sensitivity to cultural & gender differences 4) Candidates abilities to incorporate multiple perspectives in instruction 5) Candidates abilities to develop classroom/school climates that value diversity 6) Candidates understanding of teaching and learning styles and adaptation of instruction 7) Candidates disposition valuing in particular fairness and learning by all Please refer to the additional attachment, _through_ _Garduate_Diversity_Survey _Report.pdf for survey of graduates results. FOR THE NEXT REPORT YOU MUST CONDUCT YOUR OWN SURVEYS UNLESS YOU MAKE OTHER ARRANGEMENTS WITH OREA. In summary, have the analyses and summaries of these data provided any evidence that the program needs to make adjustments in the preparation of candidates? Of the 8 respondents, 75% stated they were Prepared or Well-Prepared during the MAT program for promoting equality within their professional domains, while 25% stated they were Somewhat Prepared. Notably, all equity criteria mentioned here are covered in the MAT Program s Math 523 and Math 524 methods courses. Consequently, all graduate students completing the MAT Program have demonstrated awareness of the seven diversity issues in this item. Moreover, any such training is both redundant and pedantic, given the intense training in undergraduate teaching programs and the intense focus on diversity among in-service programs. We do not measure, nor do we have an instrument to measure, any of the 7 subfields suggested in this item. Suggestion: We suggest you work with NCATE officials to review the necessity of this disaggregation under a reasonableness criterion for advanced programs. Indeed, while we heartily support equity preparation, particularly in the pre-professional programs, we just as heartily deny the utility of gathering disaggregated measurements like these for advanced programs. Furthermore, if the need for this data rests with NCATE, that need should be supported by NCATE s own program assessment, affirming the value of gathering that data for advanced programs.

11 Master of Arts Teaching Mathematics 11 Section 10. Please fill in the table below: 2008/ /10 Applicants 7 5 Admits 5 3 Completers 6 1 Non-completers 1 1 What program interpretations and conclusions can be made about the above data? What efforts are planned to increase applicants, retain candidates, and increase completion rates? 1. The MAT Program Director, the Department of Mathematics, and CWU s Office of Research and Graduate Studies is currently discussing the balance between program admission and program quality. The MAT Director, Dr. Lundin, participates in that on-going discussion. The recent economic decline seems to have taken a toll on applicants. 2. Teachers are resource stressed during their academic year. Consequently, they find working on MAT projects to be a challenge. Completing the project is daunting for many, although this barrier is no less threatening across programs or institutions. To better provide impetus, mentoring, and momentum to complete the projects, the MAT Director has made recent changes to the MAT Program. The program s Research Design in Mathematics Education course and its Advanced Statistical Analysis course have been shifted to the purview of the Department of Mathematics, allowing better alignment with MAT Projects. Furthermore, the MAT Director teaches the research course, focusing new students MAT Projects in meaningful, yet doable directions. Finally, the MAT director continues to monitor students progress throughout the program, intervening if no progress is made.

12 Master of Arts Teaching Mathematics 12 Section 11. Please discuss the results of post-graduate employer satisfaction with candidates of your program. How were the data collected? Has the program considered preparation changes as a result of knowing how employers rated the graduates of the program? Were these results discussed with current candidates, all faculty, and PEAB members? Our last program review found Programs at the advanced level do not consistently nor systematically gather data from graduates or employers of graduates. What has the program done to overcome this AREA for Improvement? (CWU Assessment of Student Learning Report # 3) We did not gather this data, nor is it legal or proper to do so without permission from participants. Think about deleting or altering this item. Furthermore, consider the following: How accurate could such information be, given that permission must be obtained from any employee whose performance is under scrutiny?

13 Master of Arts Teaching Mathematics 13 Section 12. Please discuss the results of post-graduate candidate satisfaction with how well the program prepared them for their current positions. How were the data collected? Has the program considered preparation changes as a result of knowing how well candidates rated their program satisfaction? Were these results discussed with current candidates, all faculty, and PEAB members? Please provide a table of placement rates for your candidates. (CWU Assessment of Student Learning Report # 3) Alumni Survey: % of the respondents agreed or strongly agreed that they were very satisfied with [their] education % of the respondents agreed or strongly agreed that the MAT Program provided useful perspectives on critical issues facing the profession.

14 Master of Arts Teaching Mathematics 14 Section 13. Please discuss ways in which the program is planning to improve the program assessment process. What did the program do in response to last year s assessment information? Please reference data when you answer this question. (CWU Assessment of Student Learning Report #4 and #5) With the Alumni Survey, we have made a notable improvement in assessing the MAT Program over previous years. Improvements to the assessment system will stem from improving the Alumni Survey and reinstituting the MAT Student Survey. The MAT Program Director was on sabbatical last year, so there was no assessment cycle completion. Completing the annual MAT Program assessment cycle should resume this year. We suggest that NCATE Assessment Directors do the following to enhance their data collection: First, argue against and remove inherently unsound data solicitations in these queries, such as employer posthire surveys. Second, we suggest you examine the grain size of solicited data, and avoid redundant and useless data collection. Third, we suggest that you systematize some form of disaggregation strategy to unburden teachers and the rest of us from pedantic data collection tasks. (See remarks in Section 7.) Finally, because this collection system changes annually, realize that accurately completing one of these reports takes two to three times longer than the estimated two hours. Begin now to look toward the new embedded clinical requirements that NCATE announced in November 2010, and formulate a valid, reliable, and PRACTICAL data collection plan that aligns with the clinically oriented programs soon to be assessed. NOTE: At the heart of any assessment plan, by NCATE s own words, is student achievement. Professional portfolios should indeed document this above all. In the case of graduate teachers, program assessments should document their achievements as graduate students AND how those achievements impact their own students. We know this. In the advent of the upcoming NCATE movement toward clinical practice, we suggest you meet with us to revise your assessment practices, so that we all know what valid and reliable data will be needed. Finally, many of us have resisted using LiveText in our programs for very good reasons. It is cumbersome, expensive, and provides only face value validity and reliability information. Perhaps most importantly, LiveText has no ultimate utility for those whose progress it is supposed to document. Suggestions: Think about making LiveText revisions that compel teachers to want to use the product as a showcase. If such a utility is not present, the product will never be viable. Second, work with users to show them how to implement LiveText as a showcase. Third, if LiveText can t be made intuitive, attractive, and compelling, instead of merely an invalid and unreliable data repository, quit using it. We are willing to work with you toward a new plan, given the monumental NCATE changes.