ESL Reading & Writing SLO Assessments. A brief history and a look ahead...

Transcription

1 ESL Reading & Writing SLO Assessments A brief history and a look ahead...

2 Background: Highlights Discussion of consistency of standards for passing/failing students Development of rubrics for evaluating students writing Occasional common midterms or essay exams with joint grading in 181RW and 183RW (fulltime and part-time faculty participation)

3 Fall 2011: SLO Assessment The first common exams were given in ESL reading/writing courses with multiple sections (ESL 181RW-183RW). A different exam was used for each level (181RW, 182RW, 183RW), but each section at that level took the same exam. Each test contained a reading, a set of comprehension questions designed by the teachers at that level, and a writing topic related to the reading.

4 Fall Fall : The ESL Department conducted course review. SLOs were modified and updated. Common exams were given each semester. Faculty decided, by consensus, to test the lower levels (181RW, 182RW) in fall semesters and upper levels (183RW, 184RW) in spring semesters.

5 ESL Reading and Writing SLOs Approved in December : Reading Skills ESL 181RW ESL 182RW ESL 183RW ESL 184RW 1. Identify main ideas and specific 1. Identify main ideas, specific details, details in reading passages and simple inferences in reading appropriate for students at this passages appropriate for students at level of English proficiency this level of English proficiency (usually (adapted, low-intermediate adapted, intermediate readings, 1-3 readings, 1-2 pages). pages long, on general interest and/or academic topics). 1. Distinguish between main ideas and details in texts appropriate for students at this level of English proficiency (predominantly authentic texts, written for native English speakers, 2-4 pages in length, high intermediate to low advanced, on general and academic topics). 2. Analyze and make inferences based on reading texts described above. 1. Read and analyze texts appropriate for students at this level of English proficiency (authentic texts, 2-5 pages in length, primarily academic English) Writing and Grammar Skills 2. Produce sentences which usually demonstrate correct use of the grammar taught at this level, with a particular focus on sentence structure (complete sentences), verb tenses, pronoun forms, and basic sentence combining (with primary coordinating conjunctions: and, but, so). 3. Write basic paragraphs of words on assigned topics (related to themes of study or the student's life). 2. Produce sentences which usually demonstrate correct use of the grammar taught at this level (and 181RW), with a particular focus on verb tenses (including perfect tenses), modal verbs, and sentence combining (with coordinating and subordinating conjunctions). 3. Write coherent paragraphs of words on assigned topics related to a reading and/or the student's life. 3. Summarize and respond to assigned reading materials in comprehensible and coherent written English. 4. Write coherent and organized paragraphs/short essays of approximately 250 words based on reading materials and other topics appropriate for students at this level of English proficiency. 5. Use the grammar covered in ESL 81RW, 182RW, and 183RW with increased accuracy and fluency. 2. Summarize, respond to, and discuss assigned reading materials in comprehensible and coherent written English. 3. Write coherent and organized paragraphs and short essays of 250 words or more based on reading materials and topics appropriate for students at this level of English proficiency. 4. Use the grammar covered in ESL 181RW-184RW with increased accuracy and fluency.

6 Student Learning Outcomes Tested in Fall : Note: The portions in red font are specific to ESL 181RW. ESL 182RW requires somewhat longer, more challenging readings; additional grammatical targets; and longer, more coherent paragraphs. The student will: Identify main ideas and specific details in reading passages appropriate for students at this level of English proficiency (adapted, low-intermediate readings, 1-2 pages). Produce sentences which usually demonstrate correct use of the grammar taught at this level, with a particular focus on sentence structure (complete sentences), verb tenses, pronoun forms, and basic sentence combining (with primary coordinating conjunctions: and, but, so). Write basic paragraphs of words on assigned topics (related to themes of study or the student's life).

7 Testing: After completing a reading and some objective questions on a related topic, students were asked to... Answer the following question in a paragraph (approximately words). Most people agree that there is a growing problem of too much trash in our world. What can people do differently to produce less trash? Use specific examples from the reading and/or from your own observation or experience. (181RW) Answer the following question in a paragraph (or more). Write as much as you can (or, approximately words). Do you think money can ever bring a person happiness? Why or why not? Explain your answer with reasons and examples. Use examples from the article and from your own life or the life of someone you know. (182RW)

8 Results of ESL Reading/Writing Common Exams, tracked over time (beginning in Fall 2011) Writing Test Results Total Students Tested, Fall 2011 % successful on Writing SLO(s), Fall 2011 Total Students Tested, Spring % successful on Writing SLO(s), Spring Total Students Tested, Fall 181RW 72 74% % 182RW 93 74% %* % successful on Writing SLO(s), Fall 183RW 68 59% 44 70% RW Invalid results** Invalid results** Reading Test Results Total Students Tested, Fall 2011 % successful on Reading SLO(s), Fall 2011 Total Students Tested, Spring % successful on Reading SLO(s), Spring Total Students Tested, Fall 181RW 72 76% % 182RW 91 69% % % successful on Reading SLO(s), Fall 183RW 69 62% 44 75% RW Invalid results** Invalid results** *One section of this course was exceptionally low. If this section is removed and the other three sections are tabulated, the success rate is 72%. **This test was not properly designed or administered. The reading portion of the test was so long and difficult that many students did not even have time to begin or write more than a few sentences for the writing portion.

9 Results of common exams, Fall only (by section) Course & section 181RW Section-01 Section-02 Section RW Section-01 Section-02 Section-03 Section-04 Total Students Tested 19 Students who met the Writing SLO(s) 15 (79%) 7 (44%) 8 (50%) (59%) Students who did not (38%) 13 (87%) 10 (63%) 8 (67%) (60%) 27 Course & section 181RW Section-01 Section-02 Section RW Section-01 Section-02 Section-03 Section-04 Total students Tested 19 Students who met the Reading SLO (71%) (81%) Students who did not

10 Analysis of the Fall Results Faculty readers noted the main problems with each paper that was deemed not passing and found the following: In ESL 181RW, the most common problems were... incomplete sentences / sentence structure length of writing (less than 150 words on the assigned topic) plagiarism (due to some students having access to the reading while writing their papers; some teachers did not follow the prescribed procedure, which disallowed this access). In ESL 182RW, the most common problems were... incomplete sentences / sentence structure basic verb tenses (those included in the 181RW curriculum) incorrect use of conjunctions, especially subordinating conjunctions

11 Possible reasons for poor results: 1. A poorly chosen writing question in 181RW In the case of the 181rw results, I suspect they can be partly attributed to a poorly chosen writing question. Upon reading the students test papers, we (the faculty scoring them) realized that the students at this level had not been taught some of the grammar that would have been most helpful to them in answering the question. While the 182rw classes used a test that had been given the previous year, a new one was made for the 181rw classes, and I think I didn t give due diligence in anticipating how students would respond to the question. 2. An exceptionally weak class bringing down the pass rate in 182RW I think it s noteworthy that just one section of 182rw was exceptionally low in writing skills. You ll see in the chart that if that section is removed from the tabulation, the other three sections have very similar results to the classes tested in Fall Test administration / plagiarism issues There were some test administration problems which resulted in students having access to the reading material during the writing test and then plagiarizing portions of it in their answers. These papers were considered not passing due to plagiarism. 4. Changes in the ESL student population? Are we getting more underprepared students due to shrinking adult schools? More older and/or underprepared students who are disadvantaged when it comes to language learning?

12 Where do we go from here? Ideas related to testing: (under consideration) 1. Don t use an exam for which the students come in cold. It s a lot like what we do for placement testing, but it isn t what teachers do most of the time in their classes. Why not let the students write about a topic they re familiar with? 2. Don t try to test/evaluate so many things at once. Test one SLO and/or one skill at a time. 3. Emphasize to all faculty the importance of following the established procedures for whatever test we administer. Ideas related to improving instruction: (under consideration) 1. Instructors could spend 5-10 minutes of every class period (or 15 min. per week) looking at incomplete sentences and how to fix them, or focusing on sentence structure in some other way 2. Add grammar courses co-requisites for each level in the program 3. Make sure all faculty require students to do some in-class writings cold before the common exams if this is a skill students do really need to develop. 4. Modify course outlines to ensure greater consistency of teachers expectations of students.