Assessment Plan. Wayne State University. Academic Programs. College of Liberal Arts and Sciences

Transcription

1 Assessment Plan Wayne State University Academic Programs College of Liberal Arts and Sciences Classical and Modern Languages, Literatures, and Cultures MA in Language Learning Main Language Learning mission statement Providing Department: MA in Language Learning Mission Statement The Master of Arts in Language Learning offers professional development in the theory, research, and practice of foreign language learning and teaching, advanced study of the foreign language and its cultures, and exposure to a complementary cognate area to enhance teachers professional knowledge and skills. The primary audience of the program is Metro Detroit foreign language teachers, many of whom are already certified foreign language teachers, who need and want continuing professional development. Progress: 1.0: MALL Outcome 1_ Summarize theory and research Providing Department: MA in Language Learning Learning Outcome Description Students will accurately summarize theory and research in second language acquisition and foreign language pedagogy Page 1 of 12

2 1.0: MALL Assessment 1_Research papers, book reviews, and MA essays 1. Data source: Students research papers, book reviews, and Master s essays in LGL courses in which Development/Reinforcement/Practice or Mastery of the outcome is expected (5810/7810, 5820/7820, 5830/7830, 5860/7860, 7999). Scores from one section of the grading rubric will used. (See #4 below.) 2 and 3. Data gathering and timeline: MALL faculty will collect these assignments as part of normal class requirements each semester the courses are offered. (For AY13-14 data were gathered from W13 through W14 to establish baseline performance. Future academic year data will only include scores from one Fall, Winter, and Spring/Summer.) 4. Data scoring: Student work will be scored by MALL faculty using the shared MALL essay grading rubric, which includes a subscore titled "Review of lit(erature)". 5. Scale: Scores on the "Review of lit" section of each grading rubric for these assignments will serve as the data for this assessment. Because the points possible may vary across classes, all scores will be reported as a percentage for the purposes of program assessment. 6. Criteria for acceptable performance: The criterion level of performance for successful achievement of the "Summarize" learning outcome is an 85% average score across all students and assignments on this assessment. 7. Review of results: MALL faculty will conduct an annual review of students performance on this assessment by April 30 each year. Scores on students summaries of theory and research from Winter 2013 through Winter 2014 averaged 96%. This average exceeds the target 85% average for this learning outcome. MALL assessment data file all LOs Baseline No survey data were collected. from the AY13-14 assessment of LO 1.0 revealed that the grading rubrics used to assess student performance provided ambiguous information. They inadvertently combined into a single score students' ability to objectively report the contents of published research with their ability to evaluate the validity of that research. As a result, we have taken two steps: 1. Redefinition of our original learning outcomes: We have redefined LO 1.0 from "Analysis of theory and research" to "Summarize theory and research" to focus on students' ability to objectively report the contents of published research. We have redefined LO 2.0 to focus on the use of theory and research to effectively evaluate pedagogical materials and activities. The revised outcomes are already entered in Compliance Assist. 2. Revision of our grading rubrics: We have drafted a modified grading rubric for the MA essay, and during AY we will revise the comparable sections of each assignment's grading rubric to match this new focus. These changes will enable us to gather information that clearly separates information about one skill (summarizing objectively) from another skill (evaluating the information). As such, we will Page 2 of 12

3 again gather data on this learning outcome for AY MALL faculty will complete the revision of the grading rubrics by December Data for this assessment will be gathered again in December 2014 and April 2015 and analyzed by May 15, Reporting are not reported to stakeholders at present. However, the MALL faculty will need to meet this academic year to make a plan for disseminating this information to stakeholders. Our tentative plan is to present the results on the program's website. A draft of the report is attached. MALL report to stakeholders AY13-14 Additional Information As further evidence of students' ability to effectively summarize disciplinary literature, one MALL student is presenting her review of literature on the uses of technology to support learning disabled students at ACTFL 2014, a national conference. Her presentation builds off of her research paper in LGL 5830/7830 (Technology in the FL Classroom). 3.0: A3.0 Projects Progress: In Progress Faculty will collect scores from student projects each semester in LGL 5750 and LGL 5830/7830 from the rubric section titled "summary of literature"... Baseline Reporting Additional Information 2.0: MALL Outcome 2_ Evaluation of Existing Pedagogical Materials Providing Department: MA in Language Learning Learning Outcome Description Students apply theory and research to the evaluation of pedagogical materials and activities. Page 3 of 12

4 2.0: MALL Assessment 2_Critiques, essays, presentations 1. Data source: Students critiques of activities, tests,or uses of technology, and Master s essays from LGL courses in which Development/Reinforcement/Practice or Mastery of the outcome is expected (5810/7810, 5820/7820, 5830/7830, 5860/7860, 7999). Scores from one section of the grading rubric will used. (See #4 below.) 2 and 3. Data gathering and timeline: MALL faculty will collect these assignments as part of normal class requirements each semester the courses are offered. 4. Data scoring: Student work will be scored by MALL faculty using shared criteria as part of more comprehensive rubrics for each assignment. The criteria are incorporated into each assignment's grading rubric as an "Evaluation" subscore, and include: an evaluation of the match between the theory and research studied and the pedagogical materials or activity selected for each class an evaluation of students' critical commentary explaining the advantages and disadvantages of the materials or activity based on the theory and research 5. Scale: Scores on the "Evaluation" subscore of each grading rubric for these assignments will serve as the data for this assessment. Because the points possible may vary across classes and assignments, all scores will be reported as a percentage for the purposes of program assessment. 6. Criteria for acceptable performance: The criterion level of performance for successful achievement of the "Evaluation" learning outcome is an 85% average score across all students and assignments on this assessment. 7. Review of results: MALL faculty will conduct an annual review of students performance on this assessment by April 30 each year. Students "Evaluation" subscores from Winter 2013 through Winter 2014 averaged 91%. This average exceeded the target 85% average for this learning outcome. (Future academic year data will be limited to one Fall, Winter, and Spring/Summer cycle; AY included extended information to establish baseline data.) MALL data_lo 2 EValuation Baseline No survey data were collected. from the AY13-14 assessment of LO 1.0 revealed that the grading rubrics used to assess student performance provided ambiguous information. They inadvertently combined into a single score students' ability to objectively report the contents of published research with their ability to evaluate the validity of that research. As a result, we have taken two steps: 1. Redefinition of our original learning outcomes: We have redefined LO 1.0 from "Analysis of theory and research" to "Summarize theory and research" to focus on students' ability to objectively report the contents of published research. We have redefined LO 2.0 to focus on the use of theory and research to effectively evaluate pedagogical materials and activities. The revised outcomes are already entered in Compliance Assist. 2. Revision of our grading rubrics: We have drafted a modified grading rubric for the MA essay, and this year we will revise the comparable sections of each assignment's grading rubric to match this new focus. These changes will enable us to gather information that clearly separates information about one skill (summarizing objectively) from another skill (evaluating the information). As such, we will again gather data on this learning outcome for AY Page 4 of 12

5 MALL faculty will complete the revision of the grading rubrics by December Data for this assessment will be gathered again in December 2014 and April 2015 and analyzed by May 15, Reporting are not reported to stakeholders at present. However, the MALL faculty will need to meet this academic year to make a plan for disseminating this information to stakeholders. Our tentative plan is to present the results on the program's website. A draft of the report is attached. MALL stakeholder report AY13-14 Additional Information None. 1.0: MALL Outcome 1_ Summarize theory and research Learning Outcome Description Students will accurately summarize theory and research in second language acquisition and foreign language pedagogy Page 5 of 12

6 1.0: MALL Assessment 1_Research papers, book reviews, and MA essays 1. Data source: Students research papers, book reviews, and Master s essays in LGL courses in which Development/Reinforcement/Practice or Mastery of the outcome is expected (5810/7810, 5820/7820, 5830/7830, 5860/7860, 7999). Scores from one section of the grading rubric will used. (See #4 below.) 2 and 3. Data gathering and timeline: MALL faculty will collect these assignments as part of normal class requirements each semester the courses are offered. (For AY13-14 data were gathered from W13 through W14 to establish baseline performance. Future academic year data will only include scores from one Fall, Winter, and Spring/Summer.) 4. Data scoring: Student work will be scored by MALL faculty using the shared MALL essay grading rubric, which includes a subscore titled "Review of lit(erature)". 5. Scale: Scores on the "Review of lit" section of each grading rubric for these assignments will serve as the data for this assessment. Because the points possible may vary across classes, all scores will be reported as a percentage for the purposes of program assessment. 6. Criteria for acceptable performance: The criterion level of performance for successful achievement of the "Summarize" learning outcome is an 85% average score across all students and assignments on this assessment. 7. Review of results: MALL faculty will conduct an annual review of students performance on this assessment by April 30 each year. Scores on students summaries of theory and research from Winter 2013 through Winter 2014 averaged 96%. This average exceeds the target 85% average for this learning outcome. MALL assessment data file all LOs Baseline No survey data were collected. from the AY13-14 assessment of LO 1.0 revealed that the grading rubrics used to assess student performance provided ambiguous information. They inadvertently combined into a single score students' ability to objectively report the contents of published research with their ability to evaluate the validity of that research. As a result, we have taken two steps: 1. Redefinition of our original learning outcomes: We have redefined LO 1.0 from "Analysis of theory and research" to "Summarize theory and research" to focus on students' ability to objectively report the contents of published research. We have redefined LO 2.0 to focus on the use of theory and research to effectively evaluate pedagogical materials and activities. The revised outcomes are already entered in Compliance Assist. 2. Revision of our grading rubrics: We have drafted a modified grading rubric for the MA essay, and during AY we will revise the comparable sections of each assignment's grading rubric to match this new focus. These changes will enable us to gather information that clearly separates information about one skill (summarizing objectively) from another skill (evaluating the information). As such, we will again gather data on this learning outcome for AY Page 6 of 12

7 MALL faculty will complete the revision of the grading rubrics by December Data for this assessment will be gathered again in December 2014 and April 2015 and analyzed by May 15, Reporting are not reported to stakeholders at present. However, the MALL faculty will need to meet this academic year to make a plan for disseminating this information to stakeholders. Our tentative plan is to present the results on the program's website. A draft of the report is attached. MALL report to stakeholders AY13-14 Additional Information As further evidence of students' ability to effectively summarize disciplinary literature, one MALL student is presenting her review of literature on the uses of technology to support learning disabled students at ACTFL 2014, a national conference. Her presentation builds off of her research paper in LGL 5830/7830 (Technology in the FL Classroom). 3.0: A3.0 Projects Progress: In Progress Faculty will collect scores from student projects each semester in LGL 5750 and LGL 5830/7830 from the rubric section titled "summary of literature"... Baseline Reporting Additional Information 2.0: MALL Outcome 2_ Evaluation of Existing Pedagogical Materials Learning Outcome Description Students apply theory and research to the evaluation of pedagogical materials and activities. Page 7 of 12

8 2.0: MALL Assessment 2_Critiques, essays, presentations 1. Data source: Students critiques of activities, tests,or uses of technology, and Master s essays from LGL courses in which Development/Reinforcement/Practice or Mastery of the outcome is expected (5810/7810, 5820/7820, 5830/7830, 5860/7860, 7999). Scores from one section of the grading rubric will used. (See #4 below.) 2 and 3. Data gathering and timeline: MALL faculty will collect these assignments as part of normal class requirements each semester the courses are offered. 4. Data scoring: Student work will be scored by MALL faculty using shared criteria as part of more comprehensive rubrics for each assignment. The criteria are incorporated into each assignment's grading rubric as an "Evaluation" subscore, and include: an evaluation of the match between the theory and research studied and the pedagogical materials or activity selected for each class an evaluation of students' critical commentary explaining the advantages and disadvantages of the materials or activity based on the theory and research 5. Scale: Scores on the "Evaluation" subscore of each grading rubric for these assignments will serve as the data for this assessment. Because the points possible may vary across classes and assignments, all scores will be reported as a percentage for the purposes of program assessment. 6. Criteria for acceptable performance: The criterion level of performance for successful achievement of the "Evaluation" learning outcome is an 85% average score across all students and assignments on this assessment. 7. Review of results: MALL faculty will conduct an annual review of students performance on this assessment by April 30 each year. Students "Evaluation" subscores from Winter 2013 through Winter 2014 averaged 91%. This average exceeded the target 85% average for this learning outcome. (Future academic year data will be limited to one Fall, Winter, and Spring/Summer cycle; AY included extended information to establish baseline data.) MALL data_lo 2 EValuation Baseline No survey data were collected. from the AY13-14 assessment of LO 1.0 revealed that the grading rubrics used to assess student performance provided ambiguous information. They inadvertently combined into a single score students' ability to objectively report the contents of published research with their ability to evaluate the validity of that research. As a result, we have taken two steps: 1. Redefinition of our original learning outcomes: We have redefined LO 1.0 from "Analysis of theory and research" to "Summarize theory and research" to focus on students' ability to objectively report the contents of published research. We have redefined LO 2.0 to focus on the use of theory and research to effectively evaluate pedagogical materials and activities. The revised outcomes are already entered in Compliance Assist. 2. Revision of our grading rubrics: We have drafted a modified grading rubric for the MA essay, and this year we will revise the comparable sections of each assignment's grading rubric to match this new focus. These changes will enable us to gather information that clearly separates information about one skill (summarizing objectively) from another skill (evaluating the information). As such, we will again gather data on this learning outcome for AY Page 8 of 12

9 MALL faculty will complete the revision of the grading rubrics by December Data for this assessment will be gathered again in December 2014 and April 2015 and analyzed by May 15, Reporting are not reported to stakeholders at present. However, the MALL faculty will need to meet this academic year to make a plan for disseminating this information to stakeholders. Our tentative plan is to present the results on the program's website. A draft of the report is attached. MALL stakeholder report AY13-14 Additional Information None. MALL Curriculum Map Providing Department: MA in Language Learning Attach Curriculum Map Here MALL Curriculum Map There are no related items. 1.0: MALL Assessment 1_Research papers, book reviews, and MA essays 1. Data source: Students research papers, book reviews, and Master s essays in LGL courses in which Development/Reinforcement/Practice or Mastery of the outcome is expected (5810/7810, 5820/7820, 5830/7830, 5860/7860, 7999). Scores from one section of the grading rubric will used. (See #4 below.) 2 and 3. Data gathering and timeline: MALL faculty will collect these assignments as part of normal class requirements each semester the courses are offered. (For AY13-14 data were gathered from W13 through W14 to establish baseline performance. Future academic year data will only include scores from one Fall, Winter, and Spring/Summer.) 4. Data scoring: Student work will be scored by MALL faculty using the shared MALL essay grading rubric, which includes a subscore titled "Review of lit(erature)". 5. Scale: Scores on the "Review of lit" section of each grading rubric for these assignments will serve as the data for this assessment. Because the points possible may vary across classes, all scores will be reported as a percentage for the purposes of program assessment. 6. Criteria for acceptable performance: The criterion level of performance for successful achievement of the "Summarize" learning outcome is an 85% average score across all students and assignments on this assessment. 7. Review of results: MALL faculty will conduct an annual review of students performance on this assessment by April 30 each year. Scores on students summaries of theory and research from Winter 2013 through Winter 2014 averaged 96%. This average exceeds the target 85% average for this learning outcome. Page 9 of 12

10 MALL assessment data file all LOs Reporting are not reported to stakeholders at present. However, the MALL faculty will need to meet this academic year to make a plan for disseminating this information to stakeholders. Our tentative plan is to present the results on the program's website. A draft of the report is attached. MALL report to stakeholders AY13-14 Baseline No survey data were collected. from the AY13-14 assessment of LO 1.0 revealed that the grading rubrics used to assess student performance provided ambiguous information. They inadvertently combined into a single score students' ability to objectively report the contents of published research with their ability to evaluate the validity of that research. As a result, we have taken two steps: 1. Redefinition of our original learning outcomes: We have redefined LO 1.0 from "Analysis of theory and research" to "Summarize theory and research" to focus on students' ability to objectively report the contents of published research. We have redefined LO 2.0 to focus on the use of theory and research to effectively evaluate pedagogical materials and activities. The revised outcomes are already entered in Compliance Assist. 2. Revision of our grading rubrics: We have drafted a modified grading rubric for the MA essay, and during AY we will revise the comparable sections of each assignment's grading rubric to match this new focus. These changes will enable us to gather information that clearly separates information about one skill (summarizing objectively) from another skill (evaluating the information). As such, we will again gather data on this learning outcome for AY MALL faculty will complete the revision of the grading rubrics by December Data for this assessment will be gathered again in December 2014 and April 2015 and analyzed by May 15, There are no related items. 2.0: MALL Assessment 2_Critiques, essays, presentations 1. Data source: Students critiques of activities, tests,or uses of technology, and Master s essays from LGL courses in which Development/Reinforcement/Practice or Mastery of the outcome is expected (5810/7810, 5820/7820, 5830/7830, 5860/7860, 7999). Scores from one section of the grading rubric will used. (See #4 below.) 2 and 3. Data gathering and timeline: MALL faculty will collect these assignments as part of normal class requirements each semester the courses are offered. 4. Data scoring: Student work will be scored by MALL faculty using shared criteria as part of more comprehensive rubrics for each assignment. The criteria are incorporated into each assignment's grading rubric as an "Evaluation" subscore, and include: Page 10 of 12

11 an evaluation of the match between the theory and research studied and the pedagogical materials or activity selected for each class an evaluation of students' critical commentary explaining the advantages and disadvantages of the materials or activity based on the theory and research 5. Scale: Scores on the "Evaluation" subscore of each grading rubric for these assignments will serve as the data for this assessment. Because the points possible may vary across classes and assignments, all scores will be reported as a percentage for the purposes of program assessment. 6. Criteria for acceptable performance: The criterion level of performance for successful achievement of the "Evaluation" learning outcome is an 85% average score across all students and assignments on this assessment. 7. Review of results: MALL faculty will conduct an annual review of students performance on this assessment by April 30 each year. Students "Evaluation" subscores from Winter 2013 through Winter 2014 averaged 91%. This average exceeded the target 85% average for this learning outcome. (Future academic year data will be limited to one Fall, Winter, and Spring/Summer cycle; AY included extended information to establish baseline data.) MALL data_lo 2 EValuation Reporting are not reported to stakeholders at present. However, the MALL faculty will need to meet this academic year to make a plan for disseminating this information to stakeholders. Our tentative plan is to present the results on the program's website. A draft of the report is attached. MALL stakeholder report AY13-14 Baseline No survey data were collected. from the AY13-14 assessment of LO 1.0 revealed that the grading rubrics used to assess student performance provided ambiguous information. They inadvertently combined into a single score students' ability to objectively report the contents of published research with their ability to evaluate the validity of that research. As a result, we have taken two steps: 1. Redefinition of our original learning outcomes: We have redefined LO 1.0 from "Analysis of theory and research" to "Summarize theory and research" to focus on students' ability to objectively report the contents of published research. We have redefined LO 2.0 to focus on the use of theory and research to effectively evaluate pedagogical materials and activities. The revised outcomes are already entered in Compliance Assist. 2. Revision of our grading rubrics: We have drafted a modified grading rubric for the MA essay, and this year we will revise the comparable sections of each assignment's grading rubric to match this new focus. These changes will enable us to gather information that clearly separates information about one skill (summarizing objectively) from another skill (evaluating the information). As such, we will again gather data on this learning outcome for AY MALL faculty will complete the revision of the grading rubrics by December Data for this assessment will be gathered again in December 2014 and April 2015 and analyzed by May 15, There are no related items. 3.0: A3.0 Projects Progress: In Progress Page 11 of 12

12 Faculty will collect scores from student projects each semester in LGL 5750 and LGL 5830/7830 from the rubric section titled "summary of literature"... Reporting Baseline There are no related items. Page 12 of 12