Notetaking Strategies and Their Relationship to Performance on Listening Comprehension and Communicative Assessment Tasks

Size: px
Start display at page:

Download "Notetaking Strategies and Their Relationship to Performance on Listening Comprehension and Communicative Assessment Tasks"

Transcription

1 I SSN

2 Notetaking Strategies and Their Relationship to Performance on Listening Comprehension and Communicative Assessment Tasks Patricia L. Carrell Southern Illinois University, Carbondale RR-07-01

3 ETS is an Equal Opportunity/Affirmative Action Employer. As part of its educational and social mission and in fulfilling the organization's non-profit Charter and Bylaws, ETS has and continues to learn from and also to lead research that furthers educational and measurement research to advance quality and equity in education and assessment for all users of the organization's products and services. Copyright 2007 by ETS. All rights reserved. No part of this report may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Violators will be prosecuted in accordance with both U.S. and international copyright laws. ETS, the ETS logos, TOEFL, the TOEFL logo, and TSE are registered trademarks of Educational Testing Service (ETS). THE TEST OF ENGLISH AS A FOREIGN LANGUAGE and the TEST OF SPOKEN ENGLISH are trademarks of ETS. To obtain more information about the TOEFL programs and services, use one of the following: Web site:

4 Foreword The TOEFL Monograph Series features commissioned papers and reports for TOEFL 2000 and other Test of English as a Foreign Language (TOEFL) test development efforts. As part of the foundation for the development of the TOEFL Internet-based test (TOEFL ibt), papers and research reports were commissioned from experts within the fields of measurement, language teaching, and testing through the TOEFL 2000 project. The resulting critical reviews, expert opinions, and research results have helped to inform TOEFL program development efforts with respect to test construct, test user needs, and test delivery. Opinions expressed in these papers are those of the authors and do not necessarily reflect the views or intentions of the TOEFL program. These monographs are also of general scholarly interest, and the TOEFL program is pleased to make them available to colleagues in the fields of language teaching and testing and international student admissions in higher education. The TOEFL 2000 project was a broad effort under which language testing at ETS would evolve into the 21st century. As a first step, the TOEFL program revised the Test of Spoken English (TSE ) and introduced a computer-based version of the TOEFL test. The revised TSE test, introduced in July 1995, is based on an underlying construct of communicative language ability and represents a process approach to test validation. The computer-based TOEFL test, introduced in 1998, took advantage of new forms of assessment and improved services made possible by computer-based testing, while also moving the program toward its longer-range goals, which included: the development of a conceptual framework that takes into account models of communicative competence a research program that informs and supports this emerging framework a better understanding of the kinds of information test users need and want from the TOEFL test a better understanding of the technological capabilities for delivery of TOEFL tests into the next century Monographs 16 through 20 were the working papers that laid out the TOEFL 2000 conceptual frameworks with their accompanying research agendas. The initial framework document, Monograph 16, described the process by which the project was to move from identifying the test domain to building an empirically based interpretation of test scores. The subsequent framework documents, Monographs 17-20, extended the conceptual frameworks to the domains of reading, writing, listening, and speaking (both as independent and interdependent domains). These conceptual frameworks guided the research and prototyping studies described in subsequent monographs that resulted in the final test model. The culmination of the TOEFL 2000 project is the TOEFL ibt, which was introduced in September TOEFL Program ETS iii

5

6 Abstract Utilizing a pre- and posttest research design, with an instructional intervention of good practices in notetaking, the notes taken by examinees during a computer-based listening comprehension test prior to and following the instructional intervention were examined for particular notetaking strategies. Questionnaires probed perceptions of the frequency of use and helpfulness of the notetaking strategies. Also investigated were relationships between the strategies and performance on a multiple-choice listening comprehension measure and integrated listening/writing and listening/speaking tasks and whether the brief instructional intervention affected examinees notes and performance on the three tasks. Results showed that examinees made little use of efficiency and marked organizational notetaking strategies. While the intervention had no effect on examinees notetaking strategies or on their task performance, students posttest notetaking strategies and task performance were significantly related to their pretest notetaking strategies and task performance. Moreover, notetaking and test performance were moderately related: Two notetaking strategies that most consistently related to performance on all three tasks were the number of content words in the notes and the number of test answers recorded in the notes. Efficiency strategies represented a cognitively complex category worthy of further investigation. Key words: Testing, listening comprehension, notetaking, content analysis of notes, integrated tasks v

7 Acknowledgments No study as extensive as this one could be accomplished without the assistance of a number of people. This is especially true for a researcher who is retired from full-time university employment and who, therefore, no longer enjoys institutional support. This project could not have been accomplished without the following individuals. Sara Cushing Weigle, associate professor of applied linguistics/esl and director of TESL test research and development at Georgia State University (GSU), served as overall consultant, providing invaluable advice on all aspects of the research design, instruments (especially the good practices in notetaking guidelines), and interpretation of the results. In addition, she trained the writing and speaking raters and served as the third rater on both the writing and speaking tasks. Matt McCrudden, assistant professor at the University of North Florida, served as statistician and also provided helpful input with design of the questionnaires. Donna Murray and Chizu Jaret, doctoral students at the University of Nevada Las Vegas (UNLV), provided many hours of helpful discussions about the content analysis of the notes and served as the two content analysts of the notes. I also wish to thank the consultants at the cooperating universities who were responsible for overseeing installation of the computer-based test on their respective campus, recruitment of participants, and administration of the study materials to participants: Alfred D. Stover (University of Arizona), John Haught (UNLV), and Heather Boldt and Jennifer Murphy (GSU). I wish to single out for particular thanks Beverly Ruiz (University of Indiana), who singlehandedly tested more participants than anyone else at great personal expenditure of her time. She has my undying thanks for her efforts, which went above and beyond the call of duty. Thanks also go to those at GSU who rated the written and spoken responses: Jennifer Murphy (writing), Margareta Larsson (writing and speaking), and Frank Smith (speaking). At ETS, I wish to thank Mary Enright, Susan Nissan, Tom Florek, Daniel Eignor, and Jan Rotolo. Finally, I wish to acknowledge the support of and thank my husband, J. Craig Carrell, who provided financial accounting support to the project, computer support, assistance in formatting the final report, and, most importantly, helped keep me sane throughout the 3-yearplus tenure of the project. vi

8 Table of Contents Page Introduction... 1 Literature Review... 1 Research Questions... 6 Methodology and Design... 6 Participants... 6 Materials Used in the Study... 8 Procedures Content Analysis of Notes Scoring of Other Instruments Statistical Analyses Results Research Question 1 The Prevalence of the Notetaking Strategies Research Question 2 Effects of Intervention on Notetaking Strategies Research Question 3a Perceptions of Notetaking Strategies Research Question 3b Effects of the Intervention on Perceptions of Notetaking Strategies Research Question 4 Relationships Between Notetaking and Performance on LanguEdge Listening Comprehension Tasks Research Question 5 Effects of the Intervention on LanguEdge Assessment Tasks Summary and Interpretation of Findings Discussion Implications for the Field Discussion Implications for TOEFL References Notes Appendixes A - Session 1 Questionnaire on Notetaking Strategies B - Instructional Intervention Good Practices in Notetaking vii

9 List of Tables Page Table 1 Relevant Participant Characteristics... 7 Table 2 Results of Interrater Reliabilities Tests Table 3 Useful Benchmarks for the Sufficiency of Interrater Agreement With Cohen s Kappa Coefficient Table 4 Ratings of Integrated Speaking and Written Responses Table 5 The 22 Questionnaires Items and Their Composite Categories Table 6 Descriptive Statistics for Notetaking Strategies, by Session and by Group Table 7 Overlap Among Notetaking Strategies as Evidenced by Intercorrelations, by Session and by Group Table 8 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Total Notations Table 9 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Content Words Table 10 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Abbreviations Table 11 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Test Answers Table 12 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Diagrams Table 13 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Arrows viii

10 Table 14 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Lists Table 15 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Notetaking Strategy: Circles Table 16 Perceptions of Frequency and Helpfulness of Notetaking Strategies: Descriptive Statistics, by Session and by Group Table 17 Overlap Among Questionnaire Frequency Subscales as Evidenced by Intercorrelations, by Session and by Group Table 18 Overlap Among Questionnaire Helpfulness Subscales as Evidenced by Intercorrelations, by Session and by Group Table 19 Reliability of the Questionnaire Data as Measured by Coefficient Alpha, by Session and by Group Table 20 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Frequency of General Notetaking Strategies Table 21 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Frequency of Content Notetaking Strategies Table 22 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Frequency of Efficiency Notetaking Strategies Table 23 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Frequency of Organization Notetaking Strategies Table 24 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Frequency of Review Notetaking Strategies ix

11 Table 25 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Helpfulness of General Notetaking Strategies Table 26 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Helpfulness of Content Notetaking Strategies Table 27 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Helpfulness of Efficiency Notetaking Strategies Table 28 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest versus Posttest Session) on the Perceptions of Helpfulness of Organization Notetaking Strategies Table 29 Repeated Measures ANOVA of the Effects of Group (Intervention Versus No Intervention) and Session (Pretest Versus Posttest Session) on the Perceptions of Helpfulness of Review Notetaking Strategies Table 30 Overlap Among the LanguEdge Assessment Measures as Evidenced by Intercorrelations, by Session and by Group Table 31 Pearson Product-Moment Correlations Between the Multiple-Choice Listening Comprehension Task and the Notetaking Strategies, by Session and by Group Table 32 Pearson Product-Moment Correlations Between the Integrated Listening/Writing Task and the Notetaking Strategies, by Session and by Group Table 33 Pearson Product-Moment Correlations Between the Integrated Listening/Speaking Task and the Notetaking Strategies, by Session and by Group Table 34 Estimated Marginal Means and Standard Errors of Posttest Assessment Tasks, by Group...40 Table 35 Effects of Intervention on Performance on LanguEdge Posttest Multiple-Choice Listening Comprehension Task One-Way ANCOVA With Pretest Multiple-Choice Task as Covariate Table 36 Effects of Intervention on Performance on LanguEdge Posttest Integrated Listening/Speaking Comprehension Task One-Way ANCOVA With Pretest Integrated Listening/Speaking Task as Covariate x

12 Table 37 Effects of Intervention on Performance on LanguEdge Posttest Integrated Listening/Writing Comprehension Task One-Way ANCOVA With Pretest Integrated Listening/Writing Task as Covariate xi

13 Introduction Mini-lectures comprise a substantial portion of the listening materials on the TOEFL and other listening comprehension tests. Previous research has demonstrated the potential benefit of notetaking during listening to lectures (Carrell, Dunkel, & Mollaun, 2002, 2004). Given that the TOEFL program permits notetaking on the listening comprehension section of the new TOEFL-iBT (in fact, notetaking will be permitted throughout the test see Frequently Asked Questions About the Internet-Based Test, n.d.), the relationship between notetaking and lecture comprehension is relevant for assessment as well as instruction. The purpose of this study was to examine the relationship between the content of notes taken by examinees during mini-lectures and their performance on a listening comprehension measure and integrated listening/speaking and listening/writing tasks. There were four research objectives. The first was to examine the content of notes taken by English as a second language/english as a foreign language (ESL/EFL) examinees while listening to mini-lectures on a computer-based listening comprehension test. The second objective was to examine the relationship between the content of examinees notes and performance on multiple-choice listening comprehension tasks and integrated listening/writing and listening/speaking tasks. The third objective was to examine the effects of a brief notetaking instructional intervention on the notes, as well as on task performance. The fourth objective was to investigate examinees perceptions of their notetaking strategies as well as their perceptions of the helpfulness of those notetaking strategies in their performance on the listening comprehension measure and speaking and writing tasks based on the lectures. The computer-based test used in this study was drawn from LanguEdge (ETS, 2002a), instructional courseware intended to provide teachers and students experience with communicative assessments. LanguEdge included tasks that require the integration of language skills and represented potential prototypes of future TOEFL assessment tasks. The existence of these integrated tasks provided an opportunity to explore how notetaking impacts performance on a multiple-choice listening comprehension measure as well as on integrated listening/writing and listening/speaking tasks. Literature Review Taking lecture notes is widely accepted as a useful strategy for augmenting student attention and retention of academic discourse. Notetaking is intuitively appealing to the lecturelistener and is generally viewed as a way to facilitate the process of learning and remembering 1

14 lecture material. According to Kiewra (1989), notetaking is beneficial for at least two reasons. First, notetaking aids lecture learning by activating attentional mechanisms and engaging the learner s cognitive processes of coding, integrating, synthesizing, and transforming aurally received input into a personally meaningful form. Second, notetaking is seen as beneficial because the notes taken serve as an external repository of information that permits later revision and review to stimulate recall of the information heard. A number of researchers, focusing principally on native English-speaking (L1) students, have investigated the issue of lectures and student learning (Armel & Shrock, 1996; Barnett, DiVesta, & Rogozinski, 1981; Cohn, Cohn, & Bradley, 1995; Fisher & Harris, 1973; Kiewra, 1983, 1987, 1989; Kiewra, Benton, Kim, Risch, & Christensen, 1995; Kiewra, DuBois, Christensen, Kim, & Lindberg, 1989; Kiewra, Mayer, Christensen, Kim, & Risch, 1991). Researchers studying nonnative-english speakers (L2) have also examined whether or not taking notes facilitates lecture learning and what kind of notes listeners take when listening to minilectures (Bilbow, 1989; Carrell, Dunkel, & Mollaun, 2002, 2004; Chaudron, Cook, & Loschky, 1988; Chaudron, Loschky, & Cook, 1994; Clerehan, 1995; Cushing, 1991, 1993; Dunkel, 1985, 1988a, 1988b; Dunkel & Davy, 1989; Hale & Courtney, 1994; Liu, 2001; Faraco, Barbier, & Piolat, 2002). A few researchers have looked at cultural aspects of lecture listening from an ethnographic perspective, noting that notetaking is only one of many strategies used for lecture comprehension and may not be an important strategy in many contexts (e.g, Benson, 1994; Flowerdew & Miller, 1992, 1995). In one of the first investigations of the relationship between L2 learners notetaking and listening comprehension, Chaudron, Cook, and Loschky (1988) investigated the effect on comprehension tests of the availability of notes taken during lecture listening. Chaudron et al. found that retaining or not retaining notes taken while listening to short (six- or seven-minute) lectures did not affect performance on multiple-choice or cloze listening comprehension measures. They further investigated a number of quantitative and qualitative measures of the notes and their relationships to successful short-term recall. Utilizing multiple choice and cloze comprehension tests, Chaudron et al. concluded that certain [notetaking] measures are more related to successful comprehension performance than others, on a lecture-specific basis (p. 15). For example, three measures (symbols, abbreviations, and total words) were significantly 2

15 correlated with multiple-choice test scores on one lecture, but not with multiple-choice test scores on two other lectures. In another early investigation, Dunkel (1988a) examined the relationship between the content of L2 students lecture notes and test performance. Dunkel scored the notes taken by L1 and L2 listeners during a 23-minute mini-lecture for five quality indices. Dunkel s results suggested that, when considered together as one group, L1 and L2 notetakers who scored high on the postlistening test had incorporated a large amount of information in their notes and had detected and noted information that subsequently appeared on the post lecture quiz. Interestingly, for both L1 and L2 test takers, the index total number of words was inversely related to the dependent variable, suggesting that test achievement was not directly related to the quantity of the notes taken but rather to (a) terseness of notations (embodied in the information units) and (b) inclusion of potential test information (embodied in the test-answerability index)... (pp ). For the L2 learners considered as a separate group, with respect to questions about general concepts from the lecture (not details), only total number of words and information units were statistically significantly related to test performance, the former negatively and the latter positively. With respect to questions about details from the lecture, only the number of information units and completeness were statistically significantly related to test performance, the former positively and the latter negatively. These results demonstrated that the quality of notes taken by L2 learners affects performance on tests that assess different types of learning. Notetaking is apparently also related to individual differences among L2 learners. Cushing (1993) examined the relationship between academic status (noncredit, undergraduate, or graduate student), language proficiency, and notetaking. Similar to Dunkel s (1988a) findings, good notes were those that contained the most information in the fewest number of words. These results suggested that individual differences, such as listening comprehension proficiency (high versus low) and academic status affect the content and quality of notes taken and the retention of information heard. In another study of the content of notes, Cushing (1991) employed a qualitative analysis of sets of the notes taken by the high and low proficiency L2 listeners when students were provided an outline of the lecture to guide their notetaking. Her results indicated the following: (a) high proficiency students tended to take more complete notes than low proficiency students; (b) high proficiency students made somewhat better use of the notetaking guide than did low 3

16 proficiency students; (c) overall, there was not a great deal of incorrect information in the notes, although 40% of the low proficiency students had written wrong or incomplete information in blanks in the notetaking guide; (d) lower proficiency students did not distinguish between relevant and irrelevant information as well as higher proficiency students; (e) graduate students tended to fill in the blanks on the notetaking guide more completely (although not always more accurately) than did either undergraduate or noncredit students, suggesting perhaps that they approached the task more seriously. Cushing provided suggestions for future qualitative analyses of notes. She suggested that future investigations of the content of notes should include analysis of: (a) content (main ideas, important details, mention of the connection between ideas, and inclusion of the presence/absence of trivial details and unconnected words from the lecture); (b) organization (i.e., the macro-level strategies or organization that indicates progression of ideas throughout the text, as well as the micro-level strategies or organization that suggest the relationship between the main ideas and details; and (c) notetaking efficiency (abbreviations, symbols, content versus function words, paraphrasing versus verbatim transcription, complete sentences versus key phrases); and (d) features of notetaking specific to L2 speakers (e.g., use of the L1, problems with understanding content, and problems of understanding the structure of the text). Noting that with the exception of Dunkel s work, the literature has not offered a cross cultural focus on the lecture notetaking of groups of L1 and L2 students (p. 137), Clerehan (1995) pointed out that after years of research on a vast array of features of students academic English, we actually have little idea of what is being taken down in lectures, and whether it matters. To address the situation, Clerehan analyzed the hierarchical propositional structure of notes taken by L1 and L2 students from a lecture on commercial law and compared those to the hierarchical propositional structure of the original lecture. The investigation was based on the identification of the different levels of the lecture discourse to see how much of the propositional structure (Level 1: headings; Level 2: propositions related to the headings (rules of law); Level 3: illustrative case examples) students were recording. In general, L2 notetakers recorded significantly fewer of the top-level elements than did L1 students. Moreover, Clerehan found that the L2 group failed to adequately record 19% of Level 1, 43% of Level 2, and 43% of Level 3 elements. The researcher posited if the L2 law notetakers were revising the topic from their lecture notes and from the textbook, these students, unlike their L1 peers, would need to perform 4

17 what resembles a jigsaw operation to retrieve the full meaning (p. 145). By contrast, the L1 students consistently recorded % of the principal elements, and they did not show an increase in their omissions down the hierarchy from Level 1 to Level 3, as the L2 students did. The study by Carrell, Dunkel, and Mollaun (2002, 2004) was the first to show a facilitating effect on L2 listening comprehension when a group of examinees with heterogeneous L1s was allowed to take and refer to notes during mini-lecture listening. This study, however, did not examine the content of the notes taken by examinees, nor, therefore, the relationship of the content of the notes to test performance. In a more recent study of the functions of notetaking and the content of L2 listeners notes in the context of a group of Chinese EFL learners, Liu (2001) found that taking and having one s notes available during question answering had a significant effect on the recognition of specific information (but not general information) in both immediate and delayed multiplechoice test performance. A follow-up study reported in the same thesis investigated the relationship between three qualitative features of the Chinese students notes and test performance: number of content words, number of words spelled out fully, and number of notations. Liu found significant positive correlations between number of content words and lecture-specific information (as opposed to general information), as well as significant negative correlations between number of words in full spelling and lecture-specific information (again as opposed to general information). Liu concluded that learners should be encouraged to take down more content words when required to recall specific information, and as it might be a lost cause to spell out words fully, to establish a personalized shorthand system (2001, abstract). In another recent study comparing notetaking in French by L1 and L2 undergraduate students from the perspective of cognitive load and working memory, Faraco, Barbier, and Piolat (2002) scored notes taken during students listening to a 12-minute audiotaped lecture for volume and content (i.e., number of words as a percentage of total words spoken by the lecturer and as percentage of basic units, conceptual units, and major units) and procedures (i.e., abbreviations, icons [arrows, hyphens, etc.] and reformulations with words other than those produced by the lecturer). For all participants taken together (native speakers of French, high proficiency second language learners of French, and lower proficiency second language learners of French) they found significant positive correlations between comprehension scores and both abbreviations and icons and a significant negative correlation between comprehension and 5

18 reformulations. Presumably, if notetakers are taking time to reformulate or paraphrase they are missing what s coming next. Faraco et al. also found that high proficiency nonnative speakers actually took down more content than native speakers or low proficiency nonnative speakers. Research Questions Research Question 1: What strategies do examinees use in taking notes on the academic lectures in LanguEdge? Research Question 2: Are these notetaking strategies affected by the instructional intervention on notetaking? Research Question 3: (a) What strategies do examinees report using in taking notes and how helpful do they think these strategies are for their performance on the multiple-choice listening comprehension task and the integrated listening/writing and listening/speaking tasks in LanguEdge? (b) Are these reports affected by the instructional intervention? Research Question 4: Are examinees notetaking strategies related to their performance on the multiple-choice listening comprehension task and the integrated listening/writing and listening/speaking tasks in LanguEdge? Research Question 5: Is examinees performance on the multiple-choice listening comprehension task and the integrated listening/writing and listening/speaking tasks in LanguEdge affected by the instructional intervention? Methodology and Design Participants A total of 216 participants generated sufficiently complete data to be included in the study. Participants were international students studying in the United States; they ranged in listening comprehension proficiency from low-intermediate to high, as measured by their scores on the pretest listening comprehension measure (mean scale score = 13.89, S.D. = 5.221, range 1-25, maximum possible scale score = 25). Eligible participants were any nonnative speakers of English who were ready to attempt TOEFL. Participants, who were paid for their participation, were drawn from the international student enrollees in the intensive English program (IEP) or matriculated ESL programs of several cooperating universities: The University of Indiana ( IU), The University of Arizona ( UAZ), Georgia State University ( GSU), and the University of Nevada Las Vegas ( UNLV). Relevant participant characteristics are detailed in Table 1. 6

19 Table 1 Relevant Participant Characteristics Gender Age Countries Native languages Anticipated study Characteristic N Years % Female 128 Male 88 Mean Range Korea 61 Japan 22 PRC 22 Taiwan 18 Mexico 14 Venezuela 6 Indonesia 6 Brazil 5 Other 56 Korean 61 Chinese 42 Spanish 27 Japanese 22 Other 164 Engineering, Science, & Mathematics 29 Social Sciences & Education 27 Business & Economics 23 Arts & Fine Arts 12 Other & undecided 7 Not specified 2 Time in 1 month or less 87 U.S. 3 months or less 112 To ensure that newly arrived foreign students (i.e., students such as TOEFL s EFL examinees) as well as longer-time resident foreign students (i.e., students such as TOEFL s ESL examinees) were included in the sample, testing was done at the beginning of an instructional 7

20 term, when new students were arriving in the United States. This timing was also intended to mitigate the effects of any instruction on notetaking in the institution's IEP/ESL curriculum. Testing was done in the summer and fall of 2003 and winter of Participants were each paid an incentive of $25 for their participation, in the form of a gift certificate at their university bookstore. To motivate participants to do their best on the posttest, they were told at the end of pretesting that if their posttest score equaled or exceeded their pretest scores, they would receive an additional $25 bonus. 1 Materials Used in the Study The computer-based test. Pre- and posttests consisted of the listening comprehension sections of two forms of the computer-based LanguEdge tests. Each test form consisted of two conversations and four academic lectures; both the conversations and the lectures are based on speech commonly found in North American colleges and universities and reflect that found in introductory-level academic settings on a variety of academic subject matters (ETS, 2002a). Each conversation was followed by five multiple-choice questions, and each lecture was followed by six multiple-choice questions. In Form 2 of the test one lecture was followed by only five multiple-choice questions, but one of those questions, with more than one correct response, was worth 2 points. All other multiple-choice items were worth 1 point. Multiple-choice questions included conventional four-choice questions; most of those were one-correct-response multiple-choice items, but some were innovative items that required more than one correct response. For example, one such item required the examinee to click on two correct answers. The question, which was worth 2 points, required participants to correctly categorize four examples. Partial credit (1 point) was given for correctly categorizing two of the four examples. In addition, an integrated speaking question followed the multiple-choice questions after the third lecture, and an integrated writing question followed the multiple-choice questions after the fourth lecture. The topics of the conversations and lectures were as follows: Form 1: Conversation 1 (Applying to Graduate School); Conversation 2 (Review for Biology Exam on Genes); Lecture 1 (Botany/Root Systems); Lecture 2 (Business/Organization of Management); Lecture 3 (Environmental Science/Water Resources); Lecture 4 (Philosophy/ Ethics/Plato). Form 2: 8

Community Colleges. Measuring Internationalization. AMERICAN COUNCIL ON EDUCATION The Unifying Voice for Higher Education

Community Colleges. Measuring Internationalization. AMERICAN COUNCIL ON EDUCATION The Unifying Voice for Higher Education Measuring Internationalization at Community Colleges Funded by the Ford Foundation AMERICAN COUNCIL ON EDUCATION The Unifying Voice for Higher Education Center for Institutional and International Initiatives

More information

Evaluating Analytical Writing for Admission to Graduate Business Programs

Evaluating Analytical Writing for Admission to Graduate Business Programs Evaluating Analytical Writing for Admission to Graduate Business Programs Eileen Talento-Miller, Kara O. Siegert, Hillary Taliaferro GMAC Research Reports RR-11-03 March 15, 2011 Abstract The assessment

More information

Liberal Arts Colleges

Liberal Arts Colleges Measuring Internationalization at Liberal Arts Colleges Funded by the Ford Foundation AMERICAN COUNCIL ON EDUCATION The Unifying Voice for Higher Education Center for Institutional and International Initiatives

More information

Insight. TOEFL ibt TM Test Framework and Test Development. TOEFL ibt TM Research. ETS Listening. Learning. Leading.

Insight. TOEFL ibt TM Test Framework and Test Development. TOEFL ibt TM Research. ETS Listening. Learning. Leading. Insight TOEFL ibt TM Research Series 1, Volume 1 TOEFL ibt TM Test Framework and Test Development oooooooo ETS Listening. Learning. Leading. ETS Listening. Learning. Leading. TOEFL ibt TM Research Series

More information

January 26, 2009 The Faculty Center for Teaching and Learning

January 26, 2009 The Faculty Center for Teaching and Learning THE BASICS OF DATA MANAGEMENT AND ANALYSIS A USER GUIDE January 26, 2009 The Faculty Center for Teaching and Learning THE BASICS OF DATA MANAGEMENT AND ANALYSIS Table of Contents Table of Contents... i

More information

ETS Research Spotlight. Measuring Learning Outcomes in Higher Education: The Role of Motivation

ETS Research Spotlight. Measuring Learning Outcomes in Higher Education: The Role of Motivation ETS Research Spotlight Measuring Learning Outcomes in Higher Education: The Role of Motivation ISSUE 9, AUGUST 2013 Foreword ETS Research Spotlight ISSUE 9 AUGUST 2013 Senior Editor: Jeff Johnson Contributing

More information

Validity, reliability, and concordance of the Duolingo English Test

Validity, reliability, and concordance of the Duolingo English Test Validity, reliability, and concordance of the Duolingo English Test May 2014 Feifei Ye, PhD Assistant Professor University of Pittsburgh School of Education feifeiye@pitt.edu Validity, reliability, and

More information

Journal of Educational Psychology

Journal of Educational Psychology Journal of Educational Psychology Note-Taking With Computers: Exploring Alternative Strategies for Improved Recall Dung C. Bui, Joel Myerson, and Sandra Hale Online First Publication, October 8, 2012.

More information

Monograph Series. The Reading, Writing, Speaking, and Listening Tasks Important for Academic Success at the Undergraduate and Graduate Levels

Monograph Series. The Reading, Writing, Speaking, and Listening Tasks Important for Academic Success at the Undergraduate and Graduate Levels Monograph Series MS - 21 NOVEMBER 2001 The Reading, Writing, Speaking, and Listening Tasks Important for Academic Success at the Undergraduate and Graduate Levels Michael Rosenfeld Susan Leung Philip K.

More information

Dr. Wei Wei Royal Melbourne Institute of Technology, Vietnam Campus January 2013

Dr. Wei Wei Royal Melbourne Institute of Technology, Vietnam Campus January 2013 Research Summary: Can integrated skills tasks change students use of learning strategies and materials? A case study using PTE Academic integrated skills items Dr. Wei Wei Royal Melbourne Institute of

More information

MSW Application Requirements. General Requirements...2. Additional Requirements by Program 4. Admissions Essays...5. International Applicants 7

MSW Application Requirements. General Requirements...2. Additional Requirements by Program 4. Admissions Essays...5. International Applicants 7 MSW Application Requirements Page General Requirements...2 Additional Requirements by Program 4 Admissions Essays.....5 International Applicants 7 Statistics Requirement.... 8 1 General Requirements All

More information

Northern Illinois University Office of Assessment Services

Northern Illinois University Office of Assessment Services Northern Illinois University Office of Assessment Services 2011 University Writing Project Report Analysis of College-Level Writing Ability/Skills Fall 2011 Distributed Fall 2011 For Additional Information

More information

Indiana University Request for a New Certificate Program. Proposed Title of Certificate Program: Certificate in Instructional Systems Technology

Indiana University Request for a New Certificate Program. Proposed Title of Certificate Program: Certificate in Instructional Systems Technology Campus: Indiana University-Bloomington Proposed Title of Certificate Program: Certificate in Instructional Systems Technology Projected Date of Implementation: Fall 2003 I. TYPE OF CERTIFICATE: REGULAR

More information

OVERVIEW AND ADMISSION REQUIREMENTS

OVERVIEW AND ADMISSION REQUIREMENTS PH.D. PROGRAM IN SPECIAL EDUCATION EMPHASIS IN APPLIED BEHAVIOR ANALYSIS AND POSITIVE BEHAVIORAL SUPPORT (ABA/PBS; INCLUDING BCBA) (THIS IS A SPECIAL FEDERALLY FUNDED PROJECT) OVERVIEW AND ADMISSION REQUIREMENTS

More information

Insight. TOEFL Program History. Series I, Volume 6. ETS Listening. Learning. Leading.

Insight. TOEFL Program History. Series I, Volume 6. ETS Listening. Learning. Leading. TOEFL ibt Research Series I, Volume 6 TOEFL Program History ETS Listening. Learning. Leading. TOEFL ibt Research Series 1, Volume 6 Foreword We are very excited to announce the TOEFL ibt Research Series,

More information

A Comparison of Training & Scoring in Distributed & Regional Contexts Writing

A Comparison of Training & Scoring in Distributed & Regional Contexts Writing A Comparison of Training & Scoring in Distributed & Regional Contexts Writing Edward W. Wolfe Staci Matthews Daisy Vickers Pearson July 2009 Abstract This study examined the influence of rater training

More information

MSSW Application Requirements

MSSW Application Requirements MSSW Application Requirements General Requirements... 2 Additional Requirements by Program... 4 Admissions Essays... 5 International Applicants... 8 Statistics Requirement... 9 1 General Requirements All

More information

EXCEPTIONAL NEEDS SPECIALIST

EXCEPTIONAL NEEDS SPECIALIST IMPORTANT NOTICE TO CANDIDATES: The assessment information in this document is aligned with NBPTS Exceptional Needs Standards, Second Edition (for teachers of students ages birth 21+). If you applied for

More information

TOEFL. Test Fourth. The Official Guide to the. Edition

TOEFL. Test Fourth. The Official Guide to the. Edition The Official Guide to the TOEFL Test Fourth Edition New York Chicago San Francisco Lisbon London Madrid Mexico City Milan New Delhi San Juan Seoul Singapore Sydney Toronto Copyright 2012, 2009 by Educational

More information

THE BARRIERS AND NEEDS OF ONLINE LEARNERS

THE BARRIERS AND NEEDS OF ONLINE LEARNERS Turkish Online Journal of Distance Education-TOJDE July 2014 ISSN 1302-6488 Volume: 15 Number: 3 Article 5 THE BARRIERS AND NEEDS OF ONLINE LEARNERS ABSTRACT Asst. Prof. Napaporn SRICHANYACHON Language

More information

An Analysis of College Mathematics Departments Credit Granting Policies for Students with High School Calculus Experience

An Analysis of College Mathematics Departments Credit Granting Policies for Students with High School Calculus Experience An Analysis of College Mathematics Departments Credit Granting Policies for Students with High School Calculus Experience Theresa A. Laurent St. Louis College of Pharmacy tlaurent@stlcop.edu Longitudinal

More information

An Investigation on Learning of College Students and the Current Application Situation of the Web-based Courses

An Investigation on Learning of College Students and the Current Application Situation of the Web-based Courses 2011 International Conference on Computer Science and Information Technology (ICCSIT 2011) IPCSIT vol. 51 (2012) (2012) IACSIT Press, Singapore DOI: 10.7763/IPCSIT.2012.V51.127 An Investigation on Learning

More information

Dept. of Applied Linguistics M.A. TESOL Program Description with Application and Recommendation Forms (Revised 7-April-2014, TH)

Dept. of Applied Linguistics M.A. TESOL Program Description with Application and Recommendation Forms (Revised 7-April-2014, TH) I. ADMISSION Dept. of Applied Linguistics M.A. TESOL Program Description with Application and Recommendation Forms (Revised 7-April-2014, TH) PLEASE NOTE: INTERNATIONAL STUDENTS NEED TO CHECK WITH THE

More information

Physics Graduate Program Handbook

Physics Graduate Program Handbook Carnegie Mellon University Department of Physics November 14, 2012 Version 1.1 Physics Graduate Program Handbook This document presents the rules and requirements governing the Graduate Program in the

More information

Understanding Your Praxis Scores

Understanding Your Praxis Scores 2014 15 Understanding Your Praxis Scores The Praxis Series Assessments are developed and administered by Educational Testing Service (E T S ). Praxis Core Academic Skills for Educators (Core) tests measure

More information

ACT Research Explains New ACT Test Writing Scores and Their Relationship to Other Test Scores

ACT Research Explains New ACT Test Writing Scores and Their Relationship to Other Test Scores ACT Research Explains New ACT Test Writing Scores and Their Relationship to Other Test Scores Wayne J. Camara, Dongmei Li, Deborah J. Harris, Benjamin Andrews, Qing Yi, and Yong He ACT Research Explains

More information

Feifei Ye, PhD Assistant Professor School of Education University of Pittsburgh feifeiye@pitt.edu

Feifei Ye, PhD Assistant Professor School of Education University of Pittsburgh feifeiye@pitt.edu Feifei Ye, PhD Assistant Professor School of Education University of Pittsburgh feifeiye@pitt.edu Validity, reliability, and concordance of the Duolingo English Test Ye (2014), p. 1 Duolingo has developed

More information

The Effects of Guided Planning on L2 Oral Discourse Management

The Effects of Guided Planning on L2 Oral Discourse Management The Effects of Guided Planning on L2 Oral Discourse Management Euijung Cheong Department of English, Northern Arizona University Abstract The role of pre-task planning in L2 learners oral performances

More information

APEC Online Consumer Checklist for English Language Programs

APEC Online Consumer Checklist for English Language Programs APEC Online Consumer Checklist for English Language Programs The APEC Online Consumer Checklist For English Language Programs will serve the training needs of government officials, businesspeople, students,

More information

High School Psychology and its Impact on University Psychology Performance: Some Early Data

High School Psychology and its Impact on University Psychology Performance: Some Early Data High School Psychology and its Impact on University Psychology Performance: Some Early Data John Reece Discipline of Psychology School of Health Sciences Impetus for This Research Oh, can you study psychology

More information

Achievement and Satisfaction in a Computer-assisted Versus a Traditional Lecturing of an Introductory Statistics Course

Achievement and Satisfaction in a Computer-assisted Versus a Traditional Lecturing of an Introductory Statistics Course Australian Journal of Basic and Applied Sciences, 3(3): 1875-1878, 2009 ISSN 1991-8178 2009, INSInet Publication Achievement and Satisfaction in a Computer-assisted Versus a Traditional Lecturing of an

More information

TAKING NOTES. II. Instructors usually give clues to what is important to take down:

TAKING NOTES. II. Instructors usually give clues to what is important to take down: TAKING NOTES I. There are many reasons for taking lecture notes. A. Making yourself take notes forces you to listen carefully and test your understanding of the material. B. When you are reviewing, notes

More information

3. How can you improve your ability to understand and record information presented in lectures?

3. How can you improve your ability to understand and record information presented in lectures? GETTING THE MOST OUT OF LECTURES Use this sheet to help you: prepare for and learn from lectures manage note-taking during lectures 5 minute self test 1. What are some of the purposes of lectures? 2. How

More information

Organization and Bylaws College of Education and Human Development

Organization and Bylaws College of Education and Human Development Organization and Bylaws College of Education and Human Development I. Mission The College of Education and Human Development (EHD) has a special mission within the University of fostering healthy human

More information

DICTATION IN THE ESP CLASSROOM: A TOOL TO IMPROVE LANGUAGE PROFICIENCY

DICTATION IN THE ESP CLASSROOM: A TOOL TO IMPROVE LANGUAGE PROFICIENCY DICTATION IN THE ESP CLASSROOM: A TOOL TO IMPROVE LANGUAGE PROFICIENCY Galina Kavaliauskienė and Irena Darginavičienė Mykolas Romeris University, Vilnius, Lithuania Introduction Dictation has been used

More information

Mathematics Concepts and Applications Administration and Scoring Guide

Mathematics Concepts and Applications Administration and Scoring Guide Mathematics Concepts and Applications Administration and Scoring Guide Pearson Executive Office 560 Green Valley Drive Blmington, MN 55437 800.627.727 www.psychcorp.com Copyright 20 2 NCS Pearson, Inc.

More information

Measuring the response of students to assessment: the Assessment Experience Questionnaire

Measuring the response of students to assessment: the Assessment Experience Questionnaire 11 th Improving Student Learning Symposium, 2003 Measuring the response of students to assessment: the Assessment Experience Questionnaire Graham Gibbs and Claire Simpson, Open University Abstract A review

More information

psychology and its role in comprehension of the text has been explored and employed

psychology and its role in comprehension of the text has been explored and employed 2 The role of background knowledge in language comprehension has been formalized as schema theory, any text, either spoken or written, does not by itself carry meaning. Rather, according to schema theory,

More information

The Impact of Input Enhancement through Multimedia on the Improvement of Writing Ability

The Impact of Input Enhancement through Multimedia on the Improvement of Writing Ability The Impact of Input Enhancement through Multimedia on the Improvement of Writing Ability Roya Khoii 1, Azin Tabrizi 2 1 Islamic Azad University, Tehran North Branch, Tehran, Iran, 2 Young Researchers Club,

More information

GMAC. Predicting Success in Graduate Management Doctoral Programs

GMAC. Predicting Success in Graduate Management Doctoral Programs GMAC Predicting Success in Graduate Management Doctoral Programs Kara O. Siegert GMAC Research Reports RR-07-10 July 12, 2007 Abstract An integral part of the test evaluation and improvement process involves

More information

Calculating, Interpreting, and Reporting Estimates of Effect Size (Magnitude of an Effect or the Strength of a Relationship)

Calculating, Interpreting, and Reporting Estimates of Effect Size (Magnitude of an Effect or the Strength of a Relationship) 1 Calculating, Interpreting, and Reporting Estimates of Effect Size (Magnitude of an Effect or the Strength of a Relationship) I. Authors should report effect sizes in the manuscript and tables when reporting

More information

English Language Learners

English Language Learners English Language Learners English as a Second Language 2012-2013 Objectives: To identify LEP students and provide appropriate ESL services to students enrolled in ESL program Activities Person(s) Responsible

More information

Data Coding and Entry Lessons Learned

Data Coding and Entry Lessons Learned Chapter 7 Data Coding and Entry Lessons Learned Pércsich Richárd Introduction In this chapter we give an overview of the process of coding and entry of the 1999 pilot test data for the English examination

More information

Interpretive Report of WMS IV Testing

Interpretive Report of WMS IV Testing Interpretive Report of WMS IV Testing Examinee and Testing Information Examinee Name Date of Report 7/1/2009 Examinee ID 12345 Years of Education 11 Date of Birth 3/24/1988 Home Language English Gender

More information

Best Practices in Implementing ACCUPLACER//MyFoundationsLab

Best Practices in Implementing ACCUPLACER//MyFoundationsLab Best Practices in Implementing ACCUPLACER//MyFoundationsLab June 2013 2013 The College Board. College Board, ACCUPLACER, SAT and the acorn logo are registered trademarks of the College Board. PSAT/NMSQT

More information

Developing Measurable Program Goals and Objectives

Developing Measurable Program Goals and Objectives Developing Measurable Program Goals and Objectives Adapted from Presentation Developed by: Sharon T. Wilburn, Ph.D. and d Kenneth T. Wilburn, Ph.D. University of North Florida Florida Department of Education

More information

Co-Curricular Activities and Academic Performance -A Study of the Student Leadership Initiative Programs. Office of Institutional Research

Co-Curricular Activities and Academic Performance -A Study of the Student Leadership Initiative Programs. Office of Institutional Research Co-Curricular Activities and Academic Performance -A Study of the Student Leadership Initiative Programs Office of Institutional Research July 2014 Introduction The Leadership Initiative (LI) is a certificate

More information

Department of Political Science. College of Social Science. Undergraduate Bachelor s Degree in Political Science

Department of Political Science. College of Social Science. Undergraduate Bachelor s Degree in Political Science Student Outcomes Assessment Plan (SOAP) I. Mission Statement Department of Political Science College of Social Science Undergraduate Bachelor s Degree in Political Science The Department of Political Science

More information

Department of Accounting, Finance, & Economics

Department of Accounting, Finance, & Economics Department of Accounting, Finance, & Economics Assessment Report 2010-2011 Mission/Purpose The mission of the Department of Accounting, Finance and Economics is to provide a quality education in accounting,

More information

D.P. Abeysooriya a, P.A.P. Samantha Kumara b a Faculty of Information Technology, ICBT Campus, Sri Lanka. deloosha@icbtcampus.edu.lk.

D.P. Abeysooriya a, P.A.P. Samantha Kumara b a Faculty of Information Technology, ICBT Campus, Sri Lanka. deloosha@icbtcampus.edu.lk. Reshaping Management and Economic Thinking through Integrating Eco-Friendly and Ethical Practices Proceedings of the 3 rd International Conference on Management and Economics 26-27 February 2014 Faculty

More information

University of Oregon College of Education. Cynthia Herr, Ph.D. 368 HEDCO Education Building Tuesdays 10:30 11:30, others by appointment

University of Oregon College of Education. Cynthia Herr, Ph.D. 368 HEDCO Education Building Tuesdays 10:30 11:30, others by appointment 1 University of Oregon College of Education Course Number/CRN: SPED 610/42714 Summer, 2011 Course Title: Instructor/Office: Office Hours: Contact Information: Time and Location: Credits: Assessment in

More information

Summary of the Case University of Virginia Reading Education Progra m1 October 27-30, 2014 Authorship and approval of the Inquiry Brief

Summary of the Case University of Virginia Reading Education Progra m1 October 27-30, 2014 Authorship and approval of the Inquiry Brief Summary of the Case University of Virginia Reading Education Program 1 October 27-30, 2014 The Summary of the Case is written by the auditors and approved by program faculty on October 28, 2014. The Summary

More information

INFORMATION FOR LEARNING: MAKING IT YOUR OWN

INFORMATION FOR LEARNING: MAKING IT YOUR OWN INFORMATION FOR LEARNING: MAKING IT YOUR OWN Active listening: notetaking in lectures DCU Student Learning Resources Office of the Vice-President for Learning Innovation and Registrar 2 INFORMATION FOR

More information

E-Learning and Credit Recovery: Strategies for Success

E-Learning and Credit Recovery: Strategies for Success E-Learning and Credit Recovery: Strategies for Success Brenda Humphrey Valdosta State University GA, United States brhumphrey@valdosta.edu Abstract: The purpose of this study was to describe the effects

More information

GUIDELINES FOR M.P.H. STUDENTS

GUIDELINES FOR M.P.H. STUDENTS Master of Public Health (M.P.H.) Public Health and Health Education Programs School of Nursing and Health Studies College of Health and Human Sciences Northern Illinois University DeKalb, IL 60115-2854

More information

Instructional Designer Standards: Competencies & Performance Statements

Instructional Designer Standards: Competencies & Performance Statements Standards Set 2012 ibstpi Instructional Designer Standards: Competencies & Performance Statements The 2012 ibstpi Instructional Designer Competencies and Performance statements are copyrighted by the International

More information

Evaluation of Increased Flexibility for 14 to 16 Year Olds Programme: The Second Year

Evaluation of Increased Flexibility for 14 to 16 Year Olds Programme: The Second Year RESEARCH Evaluation of Increased Flexibility for 14 to 16 Year Olds Programme: The Second Year Sarah Golden, Lisa O Donnell and Peter Rudd National Foundation for Educational Research Research Report RR609

More information

Principals Use of Computer Technology

Principals Use of Computer Technology Principals Use of Computer Technology 85 Lantry L. Brockmeier James L. Pate Don Leech Abstract: The Principal s Computer Technology Survey, a 40-item instrument, was employed to collect data on Georgia

More information

English Language Institute. New York City and Westchester County. www.pace.edu/eli

English Language Institute. New York City and Westchester County. www.pace.edu/eli English Language Institute New York City and Westchester County www.pace.edu/eli Table of Contents 2 - Campus Maps (New York City and Pleasantville) 3 - About the English Language Institute 4 - Intensive

More information

TEACHING UNDERGRADUATES DATA MINING IN ENGINEERING PROGRAMS

TEACHING UNDERGRADUATES DATA MINING IN ENGINEERING PROGRAMS TEACHING UNDERGRADUATES DATA MINING IN ENGINEERING PROGRAMS Debra L. Banks 1, Guozhu Dong 2, Huan Liu 3 and Amit Mandvikar 4 Abstract - The Teaching Undergraduates Data Mining in Engineering Programs project

More information

Test Anxiety, Student Preferences and Performance on Different Exam Types in Introductory Psychology

Test Anxiety, Student Preferences and Performance on Different Exam Types in Introductory Psychology Test Anxiety, Student Preferences and Performance on Different Exam Types in Introductory Psychology Afshin Gharib and William Phillips Abstract The differences between cheat sheet and open book exams

More information

ORIGINAL RESEARCH AND PEER REVIEW USING WEB-BASED COLLABORATIVE TOOLS BY COLLEGE STUDENTS

ORIGINAL RESEARCH AND PEER REVIEW USING WEB-BASED COLLABORATIVE TOOLS BY COLLEGE STUDENTS ORIGINAL RESEARCH AND PEER REVIEW USING WEB-BASED COLLABORATIVE TOOLS BY COLLEGE STUDENTS Mustafa Çakır 1, William S. Carlsen 2 1 Marmara Üniversitesi, Atatürk Eğitim Fakültesi, Orta Öğretim Fen ve Matematik

More information

2. SUMMER ADVISEMENT AND ORIENTATION PERIODS FOR NEWLY ADMITTED FRESHMEN AND TRANSFER STUDENTS

2. SUMMER ADVISEMENT AND ORIENTATION PERIODS FOR NEWLY ADMITTED FRESHMEN AND TRANSFER STUDENTS Chemistry Department Policy Assessment: Undergraduate Programs 1. MISSION STATEMENT The Chemistry Department offers academic programs which provide students with a liberal arts background and the theoretical

More information

SCHOOL COUNSELING. Assessment at a Glance. Early Childhood through Young Adulthood. Choosing the Right Certificate. Reviewing the Standards

SCHOOL COUNSELING. Assessment at a Glance. Early Childhood through Young Adulthood. Choosing the Right Certificate. Reviewing the Standards Early Childhood through Young Adulthood SCHOOL COUNSELING Choosing the Right Certificate Reviewing the Standards Demonstrating Your Teaching Practice and Content Knowledge Understanding the National Board

More information

DEPARTMENT OF POLITICAL SCIENCE MICHIGAN STATE UNIVERSITY

DEPARTMENT OF POLITICAL SCIENCE MICHIGAN STATE UNIVERSITY DEPARTMENT OF POLITICAL SCIENCE MICHIGAN STATE UNIVERSITY PH.D. HANDBOOK Last Updated, September 2010 1 INTRODUCTION One of the main missions of the Department of Political Science at MSU is the education

More information

Department of Curriculum and Instruction Master s Programs Handbook May 2013

Department of Curriculum and Instruction Master s Programs Handbook May 2013 Department of Curriculum and Instruction Master s Programs Handbook May 2013 TABLE OF CONTENTS INTRODUCTION...1 I. MASTER S DEGREE PROGRAM...2 M.S. in Curriculum and Instruction...2 II. ADMISSION...2 Requirements

More information

FROM: TJ Rivard, Dean, School of Humanities & Social Sciences

FROM: TJ Rivard, Dean, School of Humanities & Social Sciences TO: Mary Blakefield, Associate Vice Chancellor FROM: TJ Rivard, Dean, School of Humanities & Social Sciences DATE: June 17, 2009 RE: General Education Assessment for 2008 09 Introduction In the spring

More information

Handbook on Test Development: Helpful Tips for Creating Reliable and Valid Classroom Tests. Allan S. Cohen. and. James A. Wollack

Handbook on Test Development: Helpful Tips for Creating Reliable and Valid Classroom Tests. Allan S. Cohen. and. James A. Wollack Handbook on Test Development: Helpful Tips for Creating Reliable and Valid Classroom Tests Allan S. Cohen and James A. Wollack Testing & Evaluation Services University of Wisconsin-Madison 1. Terminology

More information

Concordia University Course Evaluation Survey: Summary Report

Concordia University Course Evaluation Survey: Summary Report Concordia University Course Evaluation Survey: Summary Report Introduction In fall 2010, the Vice-Provost, Teaching and Learning engaged the Institutional Planning Office to conduct a survey of student

More information

1. Is the program overall effective in preparing candidates to meet the expected outcomes?

1. Is the program overall effective in preparing candidates to meet the expected outcomes? Wichita State University Educational Psychology Program Committee Annual Report Calendar Year 2007 Template Adopted by Unit Assessment Committee 3-11-05, Last Revised 6-25-07, 2-15-08 Submitted April 2008,

More information

The new Verbal Reasoning and Quantitative Reasoning score scales:

The new Verbal Reasoning and Quantitative Reasoning score scales: Better by Design The new Verbal Reasoning and Quantitative Reasoning score scales: A helpful overview of what you need to know Inside: An in-depth look at the new 130 170 score scales Guidance on making

More information

Understanding Your Texas Educator Certification Program Test Scores

Understanding Your Texas Educator Certification Program Test Scores Understanding Your Texas Educator Certification Program Test Scores April 2015 Texas educator certification is governed by the State Board for Educator Certification (SBEC). The tests required for educator

More information

Effects of Different Response Types on Iranian EFL Test Takers Performance

Effects of Different Response Types on Iranian EFL Test Takers Performance Effects of Different Response Types on Iranian EFL Test Takers Performance Mohammad Hassan Chehrazad PhD Candidate, University of Tabriz chehrazad88@ms.tabrizu.ac.ir Parviz Ajideh Professor, University

More information

Significant Change to a Graduate Degree Program PhD Program in Economics

Significant Change to a Graduate Degree Program PhD Program in Economics Significant Change to a Graduate Degree Program PhD Program in Economics I. Identifying Information a) Academic unit: College of Arts and Sciences b) Teaching unit Department of Economics c) Degree program

More information

Department of Communication Studies M.A. Program Annual Report 2010-2011

Department of Communication Studies M.A. Program Annual Report 2010-2011 Department of Communication Studies M.A. Program Annual Report 2010-2011 This report documents the Department of Communication Studies efforts in assessing student learning outcomes in its graduate Masters

More information

Mathematics Computation Administration and Scoring Guide

Mathematics Computation Administration and Scoring Guide Mathematics Computation Administration and Scoring Guide Pearson Executive Office 560 Green Valley Drive Blmington, MN 55437 800.627.727 www.psychcorp.com Copyright 20 2 NCS Pearson, Inc. All rights reserved.

More information

Descriptive Statistics

Descriptive Statistics Descriptive Statistics Primer Descriptive statistics Central tendency Variation Relative position Relationships Calculating descriptive statistics Descriptive Statistics Purpose to describe or summarize

More information

Southwest Baptist University

Southwest Baptist University Doctoral Program in Educational Leadership Application Packet Southwest Baptist University College of Education and Social Sciences Department of Graduate Studies in Education Page 1 Overview of Program

More information

National assessment of foreign languages in Sweden

National assessment of foreign languages in Sweden National assessment of foreign languages in Sweden Gudrun Erickson University of Gothenburg, Sweden Gudrun.Erickson@ped.gu.se This text was originally published in 2004. Revisions and additions were made

More information

Chapter 2. Education and Human Resource Development for Science and Technology

Chapter 2. Education and Human Resource Development for Science and Technology Chapter 2 Education and Human Resource Development for Science and Technology 2.1 Evironment for Basic Human Resource Development... 53 2.1.1 Science education in primary and secondary schools... 53 2.1.2

More information

Understanding Your GACE Scores

Understanding Your GACE Scores Understanding Your GACE Scores July 2015 Georgia educator certification is governed by the Georgia Professional Standards Commission (GaPSC). The assessments required for educator certification are administered

More information

Basic Concepts in Research and Data Analysis

Basic Concepts in Research and Data Analysis Basic Concepts in Research and Data Analysis Introduction: A Common Language for Researchers...2 Steps to Follow When Conducting Research...3 The Research Question... 3 The Hypothesis... 4 Defining the

More information

Masters Program in Political Science:

Masters Program in Political Science: Masters Program in Political Science: Frequently Asked Questions Division of Political Science Mark O. Hatfield School of Government www.pdx.edu/hatfieldschool/political- science (Updated, September 2011)

More information

Introduction to. Hypothesis Testing CHAPTER LEARNING OBJECTIVES. 1 Identify the four steps of hypothesis testing.

Introduction to. Hypothesis Testing CHAPTER LEARNING OBJECTIVES. 1 Identify the four steps of hypothesis testing. Introduction to Hypothesis Testing CHAPTER 8 LEARNING OBJECTIVES After reading this chapter, you should be able to: 1 Identify the four steps of hypothesis testing. 2 Define null hypothesis, alternative

More information

AP Statistics 2002 Scoring Guidelines

AP Statistics 2002 Scoring Guidelines AP Statistics 2002 Scoring Guidelines The materials included in these files are intended for use by AP teachers for course and exam preparation in the classroom; permission for any other use must be sought

More information

An Analysis of IDEA Student Ratings of Instruction in Traditional Versus Online Courses 2002-2008 Data

An Analysis of IDEA Student Ratings of Instruction in Traditional Versus Online Courses 2002-2008 Data Technical Report No. 15 An Analysis of IDEA Student Ratings of Instruction in Traditional Versus Online Courses 2002-2008 Data Stephen L. Benton Russell Webster Amy B. Gross William H. Pallett September

More information

Guide to school authorization: Diploma Programme

Guide to school authorization: Diploma Programme Diploma Programme Guide to school authorization: Diploma Programme Diploma Programme Guide to school authorization: Diploma Programme Published October 2010 Updated February 2015 International Baccalaureate

More information

We provide the following resources online at http:// compstorylab.org/share/papers/dodds2014a/ and at

We provide the following resources online at http:// compstorylab.org/share/papers/dodds2014a/ and at S1 SUPPLEMENTARY INFORMATION Online, interactive visualizations: We provide the following resources online at http:// compstorylab.org/share/papers/dodds2014a/ and at http://hedonometer.org. Links to example

More information

Email to set up appointments at other times. SYLLABUS

Email to set up appointments at other times. SYLLABUS PROVIDENCE UNIVERSITY COLLEGE 471.12 Business Communications 3 credit hours 2015 Winter Professor: Office: Website: Jeremy Funk, PhD jeremy.funk@prov.ca 2H22 Providence Student Portal Classes: Mon. & Thurs.

More information

A Comparative Study of Reading Strategies among College Students

A Comparative Study of Reading Strategies among College Students A Comparative Study of Reading Strategies among College Students Liu Mingxu Jinan University, Zhuhai Campus Abstract Among the four language skills, reading is perhaps the most frequently used one by speakers

More information

College Choice Report

College Choice Report A Profile of 0 ACT-tested high school graduates College Choice Report Part Preferences and Prospects ACT is an independent, nonprofit organization that provides assessment, research, information, and program

More information

OPERATING STANDARDS FOR IDENTIFYING AND SERVING GIFTED STUDENTS

OPERATING STANDARDS FOR IDENTIFYING AND SERVING GIFTED STUDENTS OPERATING STANDARDS FOR IDENTIFYING AND SERVING GIFTED STUDENTS Ohio Administrative Code 3301-51-15 March 2008 3301-51-15 Operating standards for identifying and serving gifted students. (A) Definitions

More information

Math Science Partnership (MSP) Program: Title II, Part B

Math Science Partnership (MSP) Program: Title II, Part B Math Science Partnership (MSP) Program: Title II, Part B FLOYD COUNTY: COLLABORATIVE SYMPOSIUM FOR MATH IMPROVEMENT IN FLOYD COUNTY SCHOOLS ANNUAL EVALUATION REPORT: YEAR TWO Report Prepared by: Tiffany

More information

NEW YORK STATE TEACHER CERTIFICATION EXAMINATIONS

NEW YORK STATE TEACHER CERTIFICATION EXAMINATIONS NEW YORK STATE TEACHER CERTIFICATION EXAMINATIONS TEST DESIGN AND FRAMEWORK September 2014 Authorized for Distribution by the New York State Education Department This test design and framework document

More information

A National Study of School Effectiveness for Language Minority Students Long-Term Academic Achievement

A National Study of School Effectiveness for Language Minority Students Long-Term Academic Achievement A National Study of School Effectiveness for Language Minority Students Long-Term Academic Achievement Principal Investigators: Wayne P. Thomas, George Mason University Virginia P. Collier, George Mason

More information

2015-2016 Articulation Manual

2015-2016 Articulation Manual 2015-2016 Articulation Manual Ringling College of Art and Design For more information please visit: www.ringling.edu Mission Ringling College of Art and Design recognizes that artists and designers play

More information

CPA Firms Hiring Preferences and the 150-Hour Requirement

CPA Firms Hiring Preferences and the 150-Hour Requirement CPA Firms Hiring Preferences and the 150-Hour Requirement A. Bruce Caster, Wanda Causseaux, and Courtney Droms A. Bruce Caster is Associate Professor of Accounting, Utica College. Wanda Causseaux is Assistant

More information

CHAPTER 4 RESULTS. four research questions. The first section demonstrates the effects of the strategy

CHAPTER 4 RESULTS. four research questions. The first section demonstrates the effects of the strategy CHAPTER 4 RESULTS This chapter presents the statistical analysis of the collected data based on the four research questions. The first section demonstrates the effects of the strategy instruction on the

More information

Examining Students Performance and Attitudes Towards the Use of Information Technology in a Virtual and Conventional Setting

Examining Students Performance and Attitudes Towards the Use of Information Technology in a Virtual and Conventional Setting The Journal of Interactive Online Learning Volume 2, Number 3, Winter 2004 www.ncolr.org ISSN: 1541-4914 Examining Students Performance and Attitudes Towards the Use of Information Technology in a Virtual

More information