RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS)

Transcription

1 RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS) STUDENT SUPPORTS AND OPPORTUNITIES IN SCHOOL: ENGAGEMENT, BELIEFS ABOUT SELF, AND EXPERIENCES OF INTERPERSONAL SUPPORT MANUAL FOR ELEMENTARY AND MIDDLE SCHOOL ASSESSMENTS 1998 Edition

2 i RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS) Student Supports and Opportunities in School: Engagement, Beliefs about Self, and Experiences of Interpersonal Support MANUAL FOR ELEMENTARY and MIDDLE SCHOOL ASSESSMENTS TABLE OF CONTENTS I. GENERAL INTRODUCTION TO THE RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS) A. Introduction and theoretical framework for the RAPS...I-1 B. What is in the RAPS?...I-2 1. RAPS-R: An approach to the use of school records data 2. Student Engagement, Beliefs about Self, and Experiences of Interpersonal Support a. Self-Report: RAPS-S b. Teacher Report: RAPS-T c. Parent Report: RAPS-P 3. T-RAPS: Teacher Engagement, Beliefs about Self, and Experiences of Interpersonal Support 4. Critical Features of School-Site Reform: Perspectives of students (RAPS-SCF) and teachers (RAPS-TCF C. How should the RAPS be used?...i-3 II. INTRODUCTION TO RAPS COMPONENT 2: Student Supports And Opportunities in School: Engagement, Beliefs About Self, and Experiences of Interpersonal Support A. Organization of the Manual...II-1 B. Theoretical background...ii-1 1. Engagement as a Precursor of Student Performance and Adjustment 2. Beliefs About Self and Engagement in School 3. Students Experiences of Interpersonal Support and the Development of Student Self-Beliefs 4. Summary

3 ii III. DESCRIPTION OF RAPS QUESTIONNAIRES FOR SCHOOLS: RAPS-T and RAPS-S A. Overview...III-1 B. The Teacher Report of Student Engagement for Elementary School (RAPS-TE)...III-3 1. Item Description 2. Administration Information 3. The RAPS-TE Questionnaire C. The Student Self-Report for Middle School (RAPS-SE).III-8 1. Item by Construct Description 2. Administration information 3. The RAPS-SE Questionnaire D. The Teacher Report of Student Engagement for Middle School (RAPS-TM)...III Item Description 2. Administration Information 3. The RAPS-TM Questionnaire E. The Student Self-Report for Middle School (RAPS-SM)...III Item by Construct Description 2. Administration information 3. The RAPS-SM Questionnaire IV. SCORING AND INTERPRETATION OPTIONS FOR RAPS SCHOOL QUESTIONNAIRES: RAPS-T and RAPS-S A. Elementary Options I: Composite Scores...IV-1 1. The RAPS-TE Composite 2. RAPS-SE Composites B. Elementary Options II: High Risk and Optimal Indicators...IV-5 1. RAPS-TE Indicators 2. RAPS-SE Indicators C. Middle School Options I: Composite Scores...IV-7 1. The RAPS-TM Composite 2. RAPS-SM Composites

4 iii D. Middle School Options II: High Risk and Optimal Indicators...IV RAPS-TM Indicators 2. RAPS-SM Indicators V. RELIABILITY AND VALIDITY INFORMATION FOR RAPS COMPOSITES AND INDICATORS A. RAPS-TE...V-1 1. Characteristics of reliability sample 2. Reliability of the RAPS-TE Student Engagement Composite 3. Assessing the validity of the RAPS-TE Student Engagement Composite and Indicators B. RAPS-SE...V Characteristics of the reliability sample 2. Reliabilities of the RAPS-SE Composites 3. Assessing the validity of the RAPS-SE Composites and Indicators C. RAPS-TM...V Characteristics of reliability sample 2. Reliability of the RAPS-TM Student Engagement Composite 3. Assessing the validity of the RAPS-TM Student Engagement Composite and Indicators D. RAPS-SM...V Characteristics of the reliability sample 2. Reliabilities of the RAPS-SM Composites 3. Assessing the validity of the RAPS-SM Composites and Indicators VI. APPENDICES Biblography for Component 2: Student Supports and Opportunities in School..VI-1 RAPS-TE Reliability and Validity.. VI-9 RAPS-TM Reliability and Validity. VI-13 RAPS-SE Reliability and Validity..VI-17 RAPS-SM Reliability and Validity..VI-33

5 GENERAL INTRODUCTION TO THE RESEARCH ASSESSMENT PACKAGE FOR SCHOOLS (RAPS) I- 1 In this general introduction we will provide a brief overview of the theoretical framework that forms the foundation of the Research Assessment Package for Schools (RAPS), and of each of the separate components of the RAPS. Further, we will review what is in the RAPS, why and how the RAPS measurement tools were developed, and how the assessment package can be used for diverse purposes. The RAPS manual is designed in a modular format, with different components of the RAPS contained within separate modules that can be obtained either together or individually from IRRE. Each module of the RAPS manual goes into considerably more depth on both the theoretical and the research background supporting the individual components. A. Introduction and Theoretical Framework for the RAPS The Institute for Research and Reform has developed this assessment package as part of its overarching approach to the reform of public schools and school systems. This approach has three key elements: Seven critical features of successful school-site reform that research on adult and youth development, organizational and educational research and the experience of successful urban schools have shown to yield positive outcomes for diverse groups of students; A change framework that links implementation of these changes in schools to changes in the supports and opportunities experienced by adults and students in schools to changes in student performance and adjustment across elementary, middle and high school, and finally, to increased life chances of youth beyond high school; and The Research Assessment Package for Schools (RAPS), that includes strategies and instruments for measuring four major components of the school-site reform framework: 1) key indicators of student performance and adjustment; 2) the supports and opportunities experienced by students in schools; 3) the supports and opportunities experienced by adults in schools (i.e. teachers, administrators, and other staff members); and 4) the implementation of the seven critical features for school-site reform. The first two elements of the Institute s approach to educational reform are described in detail in two other IRRE publications: the seven critical features in First Things First: A Framework for Successful School-site Reform; and the change framework in A Theory of Change Approach to Evaluating Investments in Public Education. Both publications are available from the Institute for Research and Reform. The seven critical features of successful school-site reform are

6 I-2 summarized in Figure 1 and a simplified illustration of the change framework is presented in Figure 2. B. What Is In The RAPS? Currently, the RAPS includes five separate but integrated measurement tools assessing aspects of three of the four major components described above: RAPS-R, a strategy for analyzing and reporting data from student records (component 1); RAPS-S, a survey given to students to assess their levels of engagement in school, their beliefs about themselves and the interpersonal supports they receive from adults at home and at school (component 2); RAPS- T, a survey given to teachers to obtain their reports of student engagement (component 2); RAPS-P, a parent-report of student engagement in school and the support that the student receives from his or her teachers (component 2); and T-RAPS, a survey given to teachers to assess their levels of engagement and the professional and interpersonal supports they perceive in their schools (component 3). The RAPS-R. The RAPS-R was developed in response to school districts need to have a simple, compelling and scientifically credible way to track their students performance and adjustment across elementary, middle and high school. The RAPS-R is a strategy for combining information from readily available student records to fill this need. The Student Performance and Commitment Index (SPCI) includes two categories -- high risk and optimal for each school level that were developed based on longitudinal data from the Institute s data banks and from detailed reviews of other longitudinal research on markers of student performance and adjustment. The RAPS-S. The survey measures included in the RAPS-S were originally designed by a team of researchers to be used in studying specific psychological and interpersonal processes affecting students school performance and adjustment. These measures been used by researchers in education, psychology and sociology and results from these measures have been published in numerous peer-reviewed journals and other scholarly articles (see Appendices for sample references). Over the past two years, Institute research staff have revised and abbreviated these measures for use by evaluators and school districts. Measures were retained that showed utility as interim markers of progress toward subsequent outcomes in the change framework shown in Figure 2. The RAPS-T. This teacher survey was originally developed for research purposes to get information about student engagement from teachers that could then be compared to student reports of their own engagement. Subsequently, Institute research staff have abbreviated and revised the measure for use in other applications, including as an important outcome measure for educational

7 I-3 program and reform efforts targeting student motivation. Again, RAPS-T survey items were selected based on their predictive validity with respect to subsequent student performance and adjustment. The RAPS-P. The RAPS-P is a questionnaire designed to provide additional information about student engagement as well as about how much support the student is receiving from teachers at school. Additional items regarding the parent s self-perceptions of the support they provided to the student, as well as their own experiences of being supported by the school have been omitted due to low reliability and poor prediction of student self-perceptions, academic performance and personal adjustment. The RAPS-P is currently undergoing additional reliability and validity assessment. The T-RAPS. This measure, tapping the fourth component of the RAPS, was originally developed to assess the full range of supports and opportunities experienced by teachers working in schools. The scales included in the T-RAPS were based on the same motivational theory that guided the construction of the student measures included in the RAPS-S. Subsequent revisions have focused on two components: teachers experience of supports for their engagement and their reports of their own levels of engagement. The revised RAPS-T is also currently undergoing additional field testing and validation. The RAPS-CFT and RAPS-CFS. The newest measurement tools included in the assessment package, RAPS-CF (Critical Features) for Teachers and Students, will address the fourth component of the RAPS framework by assessing the extent to which the seven critical features of successful school-site reform are being implemented in schools from the perspectives of students and teachers. These two survey instruments are still being field tested and validated by the Institute. C. How Should the RAPS Be Used? The RAPS is designed to be used in at least four ways: As a diagnostic instrument that can provide simple, valid and compelling information about the current status of a particular population of students or adults; a school or set of schools; or an entire district; As a program or initiative evaluation tool to examine whether or not a particular program that focuses on one or more component of the change framework is having its intended effects; As a management tool for tracking whether particular change thresholds on key outcomes have been achieved by a group of individuals, a program or an institution; or

8 I-4 As a basic research tool to study change at the individual, classsroom, school or district level with respect to the relevant elements of the change framework. Figure 2

9 II- 1 INTRODUCTION TO RAPS COMPONENT 2 Student Supports and Opportunities in School: Engagement, Beliefs about Self, and Experiences of Interpersonal Support A. Organization of the Manual The next four sections of this manual focus specifically on the intra- and interpersonal supports students need in order to do well in school. These supports comprise those provided by teachers, parents, and peers; positive beliefs about oneself; and engagement with school and schoolwork. The manual is organized to offer easy access to varying levels of theoretical and empirical information about two RAPS measures of these three constructs: 1) A student self-report measure (RAPS-S), and 2) a teacher-report measure of student engagement (RAPS-T). This section provides theoretical background that explains the importance of the constructs as they relate to educational outcomes for students. The following section includes a brief description of each of the constructs covered by the two measures, including the specific items used to tap each construct. This is followed by a presentation of two approaches to scoring and interpreting responses on each of the measures. The last section pertaining to the RAPS-S and the RAPS-T includes reliability and validity information for the student- and teacher-report measures. Finally, three appendices provide additional information on the RAPS-S and RAPS-T, including a bibliography of related research and theoretical work, and detailed information on the reliability and validity of the RAPS-S and RAPS-T. B. Theoretical Background A substantial body of existing research explores the conditions that promote school success. Such research has found that students academic performance and adjustment are enhanced when the following three conditions exist: 1. Students experience a supportive interpersonal environment that helps them to believe in themselves and their abilities. A supportive environment is characterized by: a. a structure that is consistent and fair in its rules and expectations for student performance and conduct; b. support for student autonomy, reflected in the provision of opportunities for choice and respect for students perspectives; and c. adults who demonstrate high levels of interest in and involvement with students.

10 II Students have strongly positive feelings about themselves. This includes feeling competent and in control of their environment and of their own behavior. It also includes having generally positive feelings about their relationships with important others such as parents, teachers, and peers. 3. Students are engaged in school. Engagement is reflected in the amount of attention and effort students apply to schoolwork and homework, the emotions they experience while doing it, and the importance they place on doing well in school. These three conditions are fundamentally interrelated. They build on each other to create an interpersonal context in the school environment that supports the development of positive self-beliefs, which in turn support engagement in school activities. Student engagement has been found to be the single most robust predictor of students performance and personal adjustment in school. This conclusion holds regardless of whether students come from families that are i, ii relatively advantaged or disadvantaged economically or socially. The following pages provide a brief review of the literature regarding these three conditions. We will begin with the condition that has been most directly associated with student academic performance and personal adjustment in school student engagement and move backward through our model to student beliefs about themselves and, finally, student experience of interpersonal support. 1. Engagement as a Precursor of Student Performance and Adjustment Defining Engagement: Ongoing Engagement and Reaction to Challenge. Students engagement with school is evident both in the flow of ongoing daily activity (e.g., during a particular classroom activity, in the effort expended on homework) and in students reactions to challenge (e.g., being given a bad grade, responding incorrectly to a question in class). Ongoing Engagement. Ongoing engagement versus disaffection has to do with students typical emotions, thought processes, and behavior evidenced within the course of the school day. Behavioral engagement includes the amount of time students spend on work, the intensity of their concentration and effort, the tendency to stay on task, the level of difficulty of the tasks selected, the demonstration of flexible problem-solving, and the propensity to initiate action when given the opportunity to do so. Emotional components of engagement include heightened levels of positive emotion during the completion of an activity, demonstrated by enthusiasm, optimism, curiosity, and interest. Disaffection from classroom activities is demonstrated by feelings of boredom, discouragement, and anger. iii

11 II- 3 Reaction to Challenge. Engagement and disaffection in the face of challenge are seen in students methods of coping with perceived failures in school. Situations can be perceived as either a threat or a challenge depending on students beliefs about their own competence. Students who perceive a situation as challenging actively persist in the face of failure through the use of effort, strategizing, problem-solving, information-seeking, and experimentation. These behaviors tend to be accompanied by an optimistic attitude and attempts to plan and prevent problems from occurring in the future. Conversely, students who feel threatened tend to feel incompetent and full of self-doubt. Typical reactions to a perceived threatening situation include attempting to escape the situation either by mentally or physically withdrawing and by avoiding or putting off the activity as long as possible when it is encountered in the future. These behaviors are invariably accompanied by negative emotions that include anger, blame, denial, anxiety, and hopelessness. iv Linking Engagement to Important Educational Outcomes. A wide array of studies has concluded that school engagement is of utmost importance in minimizing the probability of school dropout and in maximizing the probability of school completion. v Regardless of socioeconomic or family background, student engagement predicts positive student outcomes in such areas as achievement test scores, attendance, grades, disciplinary referrals, retention, and graduation. vi Engaged students are more likely than their disengaged peers to stay in school, learn, demonstrate high levels of achievement, feel good about themselves, and continue on to higher education after high school. vii High levels of engagement can even explain why some high-risk students succeed academically -- a finding confirmed for males and females and for White, Hispanic, and African-American students alike. viii In contrast, students with low levels of engagement who do not develop a sense of belonging are at risk for a wide range of long-term, adverse consequences, including disruptive behavior in class, absenteeism, truancy, juvenile delinquency, and dropping out of school. ix 2. Beliefs about Self and Engagement in School Many researchers have suggested that high levels of engagement are associated with students perceptions of a) competence; b) autonomous selfregulation; and c) relatedness to significant others. Competence. Perceived competence, interacting effectively with the environment, has been found to be strongly associated with student performance in school. x In order to succeed in school, students have to know what it takes to be competent and they must believe that they have what it takes to carry out those strategies. Such competence beliefs within the academic domain are strong predictors of students behavioral and emotional engagement in the classroom. For example, when competence beliefs are examined as predictors of children s engagement in the classroom, they account for more than 25% of

12 II- 4 the variance in teachers ratings of children s engagement and disaffection. xi In addition, many studies have explored the relationship between various aspects of perceived competence (i.e., locus of control, mastery versus helplessness beliefs, causal attributions, and self-efficacy) and performance (i.e., academic achievement). Changes enhancing these aspects of competence beliefs have been found to produce positive changes in the quality and persistence of students problem-solving and performance on academic tasks. xii Autonomy. The psychological need for autonomy refers to the need to be in control of one s own behavior, or to be self-regulating. xiii One way in which perceptions of autonomy can be manifested in the academic domain is through students reasons for engaging in classroom activities. If students participate in school activities because they feel forced to or feel guilty about not participating, they will be less motivated and less engaged in the activity than if they choose to take part because it is important to them personally, or because they truly enjoy the activity. Students who perform primarily to please others, avoid guilt, or avoid punishment demonstrate strategies that are timid, stressed, and defensive and even attempt to evade the activity altogether. Conversely, students who participate in school activities because they enjoy them and/or see their value react to challenge with persistence, strategy-testing, and optimism. xiv Relatedness. The psychological need for relatedness includes feelings of being respected by, connected to, and cared for by others. xv In a school setting, students are concerned with whether their teachers, parents, and peers care about and respect them. Perceptions of belonging, of relatedness, and similar social constructs have been found to be positively associated with school adjustment (i.e., less absenteeism, greater social competence, fewer conduct problems, reduced drug use and delinquency), xvi as well as with academic motivation, effort, and achievement. xvii Indeed, some researchers view relatedness as being the most crucial factor in the motivation and engagement of certain categories of at-risk students. xviii 3. Students Experiences of Interpersonal Support and the Development of Student Self-Beliefs The development of competence, autonomy, and relatedness. The development of heightened levels of competence, autonomy, and relatedness in youth requires a variety of interpersonal supports from the school environment. xix For instance, in order to develop high levels of perceived competence, students must be provided with expectations and consequences that are clearly defined and consistently communicated by adults. xx Environments high in structure provide students with this kind of information. xxi In addition, the extent to which students perceive the classroom as autonomy-supportive is a function of their teachers ability to create an optimally challenging climate, offer choice, incorporate relevance into the curriculum, and minimize the use of external

13 II- 5 rewards, controls, and pressures. xxii Finally, the quality of students relationships with their teachers is significantly related to students beliefs about themselves and their experience of school. xxiii If students perceive their parents, teachers, and peers as dedicating resources such as time and interest to them, and being accepting and affectionate toward them, they will feel satisfied in terms of relatedness they will express more optimistic and enthusiastic responses when engaged in academic work. If students perceive their teachers and classmates as uncaring and uninterested, they will feel rejected and neglected, and will experience school as frustrating to their sense of relatedness. xxiv Interpersonal supports, student self-beliefs, and engagement. Both research and practice point to the effectiveness of meeting students psychological needs relative to alternative strategies for promoting student engagement. Factors such as personal involvement with teachers, academic and conduct standards that are high, clear, fair, and consistent, and being given a voice in the classroom, are forms of social control that influence at-risk students far more than negative consequences such as detention, suspension, retention, and so on. Relative to at-risk students who are ruled by the carrot and stick technique, students whose psychological needs are met are far more likely to demonstrate heightened levels of engagement. xxv Independent of poverty level, students whose psychological needs for competence, autonomy, and relatedness are supported by their interpersonal environments are not only more likely to score high on measures of academic engagement, xxvi but also on measures of personal and social development (e.g., self-esteem, concern for others, liking for school, educational aspirations, and academic aspirations). xxvii These students are also more likely to be successful in school. xxviii Conversely, students who do not receive the contextual supports they need to develop these positive self-beliefs lose their motivation toward and interest in learning, while demonstrating increases in problem behaviors. xxix Furthermore, because the classroom is a social setting in which students and teachers interact with each other, a reciprocal relationship has been found between student engagement and a teacher s provision of interpersonal supports such as structure, autonomy support, and involvement. Students who are engaged in their learning activities elicit enhanced levels of positive teacher behaviors, which, in turn, produce higher levels of positive student behaviors. In contrast, students who enter school with lower levels of engagement (as is the case with many at-risk students) tend to receive less of the contextual support necessary to enhance motivation, and, therefore, are more likely to disengage completely. xxx It is only a short step between disengagement and dysfunctional behaviors such as absenteeism, disruptive behaviors, suspension, and retention. xxxi Once students begin to engage in such dysfunctional behaviors, dropping out and the negative life outcomes associated with dropping out are highly probable.

14 II Summary For students to complete school with high levels of academic performance and low levels of at-risk behaviors, they must be engaged in school. For high levels of engagement to occur, students must develop positive perceptions of themselves and their abilities. Interpersonal supports within the school community allow youth to develop more effective strategies for performing in school as well as the beliefs that they have the capacity to execute those strategies. Interpersonal supports can also further students understanding of their own and others roles in regulating their behavior and can lead to the accumulation of experiences in which significant others are there when students need support. When these supports are consistently available within the school environment and allow positive self-beliefs to develop, students will be more likely to engage fully in school and, ultimately, to graduate creditably.

15 III- 1 DESCRIPTION OF RAPS QUESTIONNAIRES: RAPS-T AND RAPS-S A. OVERVIEW Two types of RAPS instrument are described in this manual: The first and most extensive of these, the RAPS-S, is a self-report measure tapping student engagement in school, student beliefs about self, and student perceptions of interpersonal support. The second measure, the RAPS-T, is a brief teacherreport measure tapping student engagement in school. Both of the instruments are available in forms for middle school (RAPS-TM and RAPS-SM) and for elementary school (RAPS-TE and RAPS-SE). 1 The RAPS-S and the RAPS-T are both designed to be used as part of comprehensive school evaluation programs, and they can also be used in basic educational research. They are not designed as diagnostic tests for at-risk students. Both of these measures are described in detail in the following sections. The table below summarizes the constructs tapped by the two measures: Construct Student-Report Teacher-Report 1. ENGAGEMENT: 1a. Ongoing Engagement X X 1b. Reaction to Challenge X 2. BELIEFS ABOUT SELF: 2a. Perceived Competence X 2b. Perceived Autonomy (Self- X Regulation) 2c. Perceived Relatedness X 3. EXPERIENCES OF INTERPERSONAL SUPPORT: 3a. Parental Support X 3b. Teacher Support X 1 No separate high school versions of the RAPS are currently available. However, the middle school versions have recently been used successfully in a number of school evaluation projects in urban school districts.

16 III- 2 A Word About RAPS Questionnaire Administration In addition to describing the RAPS measures, this section provides information about questionnaire administration. The information is based on the experience of IRRE staff and other researchers and educators who have administered these questionnaires in a variety of school settings. Because each setting in which the RAPS might be administered is unique, the information is necessarily very general. IRRE staff are available to provide more detailed information pertaining to RAPS administration upon request by teachers, researchers, administrators, and community members. Requests should include as much specific information as possible about the purposes for which the measures are to be used, and the conditions under which administration will take place. Contact numbers for IRRE may be found in the front of this manual. Confidentiality, Voluntary Participation, and Value Regardless of the purposes for which these questionnaires are being administered, or of who is conducting actual questionnaire administration, data management, data analysis, and report writing, is extremely important that individuals understand that their responses are confidential and voluntary, and that the information that is being provided is valuable. Honest responses can only be expected when the informants completing a questionnaire believe that their answers cannot be used against them, or in their favor, in any way. Similarly, honest and complete responding can only be expected from individuals who do not feel that they are being unduly coerced by those administering the questionnaire. Teachers, parents, and students should be informed that there are no repercussions for refusing to participate, or for skipping questions that make them feel uncomfortable. Parents also need to be given the opportunity to consent to their children s participation, and a mechanism for obtaining parental consent must be established. Finally, the best way to ensure full and honest responding to RAPS questionnaires is to make it clear to respondents that the information they are providing serves a larger purpose (e.g., helping the school community to establish goals for change, or evaluate how well the school is meeting its established goals).

17 III- 3 B. THE TEACHER-REPORT OF STUDENT ENGAGEMENT FOR ELEMENTARY SCHOOL (RAPS-TE) 1. Item Description 2. Administration Information 3. The RAPS-TE Questionnaire

18 III Item Description The RAPS-TE is a very brief (3-item) teacher-report measure of student engagement in school. The three items are: 1. In my class, this student seems tuned in. (teexs5 2 ) 2. This student comes to class unprepared. (tebxf2) 3. This student does more than required. (tebxs1) Each of these items is responded to on the following scale: Very True Sort of True Not Very True Not At All True (4) (3) (2) (1) 2. Administration Information Instructions for completing the RAPS-TE are fairly straightforward. Typically, teachers are given sufficient forms to provide information for all students in their class. All teachers who are being asked to participate should be given the questionnaires at the same time and should be asked to return the questionnaires by a fixed deadline. The brevity of this measure makes it possible to have teachers report on numerous students without presenting an excessive time burden that could undermine teachers willingness to participate. 2 Codes in parentheses are item names that have been used in all IRRE-sponsored and facilitated administrations of this measure. They are included here primarily to facilitate communication with IRRE staff.

19 III The RAPS-TE Questionnaire Teacher ID: Research Assessment Package for Schools - Teacher Report On the following pages, please write in a student name or identification number for each of the students in your class who are participating in this study. Then, for EACH STUDENT, tell us how true each of the three items is for THAT STUDENT by circling one of the four answers: VERY TRUE, SORT OF TRUE, NOT VERY TRUE, or NOT AT ALL TRUE. Student ID: 1. In my class, this student seems tuned-in. Very Sort of Not Very Not At All 2. This student comes to class unprepared. Very Sort of Not Very Not At All 3. This student does more than required. Very Sort of Not Very Not At All Student ID: 1. In my class, this student seems tuned-in. Very Sort of Not Very Not At All 2. This student comes to class unprepared. Very Sort of Not Very Not At All 3. This student does more than required. Very Sort of Not Very Not At All

20 III- 6 Student ID: 1. In my class, this student seems tuned-in. Very Sort of Not Very Not At All 2. This student comes to class unprepared. Very Sort of Not Very Not At All 3. This student does more than required. Very Sort of Not Very Not At All Student ID: 1. In my class, this student seems tuned-in. Very Sort of Not Very Not At All 2. This student comes to class unprepared. Very Sort of Not Very Not At All 3. This student does more than required. Very Sort of Not Very Not At All Student ID: 1. In my class, this student seems tuned-in. Very Sort of Not Very Not At All 2. This student comes to class unprepared. Very Sort of Not Very Not At All 3. This student does more than required. Very Sort of Not Very Not At All

21 III- 7 C. THE STUDENT SELF-REPORT FOR ELEMENTARY SCHOOL (RAPS-SE) 1. Item by Construct Description 2. Administration Information 3. The RAPS-SE Questionnaire

22 III Item by Construct Description This multidimensional measure includes a total of 79 items, tapping three major domains and seven separate subdomains, in a questionnaire format that can be administered to students during a single class period. The majority of these items reflect negative feelings or beliefs. In order to reduce the overall negative tone of the questionnaire, nine positive items have been added. These nine items were selected from the middle school version of the questionnaire, but are not used in constructing summary scores for elementary school students. With one exception, all items are responded to on the following scale: Very True Sort of True Not Very True Not At All True (4) (3) (2) (1) An item by construct list, including brief descriptions of each construct, is presented on the following pages. The nine items that are not included in the summary scores are presented together in the final table. Engagement Domain The Engagement domain as measured by the RAPS-SE incorporates two separate but interrelated subdomains of student adjustment in school: 1) Ongoing Engagement with school; and 2) Reactions to Challenge (i.e., the strategies students use when faced with negative or stressful school events). Ongoing Engagement Subdomain Ongoing Engagement includes the extent to which the student exerts effort on schoolwork, pays attention in class, prepares for class, and believes that doing well in school is personally important. The RAPS-SE includes a total of six items tapping these aspects of student engagement. These items are listed below: Item Item Code 3 Questionnaire Item Number I work very hard on my schoolwork. eoxs7 3 I don t try very hard in school. eoxf3 19 I pay attention in class. eaxs2 35 I don t work very hard in school. eoxf4 50 When I m in class I just act as if I m working. eaxf3 60 How important is it to you to do the best you can in school? eyxx As with the RAPS-TE, codes are item names that have been used in all IRRE-sponsored and facilitated administrations of this measure. They are included to facilitate communication among users of these measures and with IRRE staff.

23 III- 9 Eyxx3 is the single item that has a response set that differs from the other items in the questionnaire. It is responded to on the following scale: Very True Sort of True Not Very True Not At All True (4) (3) (2) (1) Reaction to Challenge Subdomain There are a number of ways in which students may cope with, or react to, negative school-related events. Students may blame negative events on the teacher or other individuals (Projection). Students may cope with negative events by downplaying their importance (Denial). At other times, they may perseverate on the events and worry about them without necessarily doing anything to ensure that such events won t occur again (Anxiety Amplification). Finally, students may cope by examining their own behavior and attempting to make changes to prevent similar negative events from occurring in the future (Positive Coping). Of these four different reactions to challenge, Projection and Denial appear to be particularly related to positive or negative outcomes for elementary school students and are included in this revised version of the RAPS-SE. One Anxiety Amplification item is included as well. The items are listed below: Item Item Code Questionnaire Item Number Projection Items: When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say the teacher didn t cover the things on the test. When something bad happens to me in school, (like not doing well on a test or not being able to answer an important question in class), I get angry at the teacher. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say it was the teacher s fault. Denial Items: When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class),, I tell myself it didn t matter. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say it wasn t important. When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I say I didn t care about it. Anxiety Amplification Item: When something bad happens to me in school (like not doing well on a test or not being able to answer an important question in class), I worry that the other students will think I m dumb. ecjx3 14 ecjx4 33 ecjx1 65 ecdx3 22 ecdx4 42 ecdx7 71 ecax1 53

24 III- 10 Beliefs about Self Domain The Beliefs about Self domain incorporates three separate subdomains: 1) Perceived Competence, 2) Perceived Autonomy, and 3) Perceived Relatedness. Perceived Competence Subdomain The Competence subdomain of the RAPS-SE consists of a total of 16 items. One of these items reflects perceptions of control in the school setting. Seven items reflect the student's beliefs about what kinds of strategies are necessary in order to achieve desirable, or avoid undesirable, outcomes. There are four different strategies represented: 1) Unknown, 2) Powerful s, 3) Effort, and 5) Luck. A fifth strategy that appeared in earlier versions of the RAPS-SE, Ability, has been omitted because students endorsement of ability as a strategy for doing well in school did not appear to be associated with student performance and adjustment in elementary school. The final eight Perceived Competence items reflect the student's belief that he or she has the capacity to enact strategies. There are two capacity items for each strategy except Unknown, including Ability. Thus, although students beliefs that ability is necessary to do well in school do not appear to differ among students doing more or less well in school, students do differ in the extent to which they believe that they are smart in school. The Unknown strategy items reflect the student's inability to determine what it takes to do well in school. Thus, it more accurately reflects the lack of a strategy and hence would have no associated feeling of being more or less capable of enacting the strategy in order to do well. The 16 items included in the revised RAPS-SE are listed below: Item Item Code Questionnaire Item Number Perceptions of Control Item I can t do well in school. ccxf3 72 Unknown Strategy Items I don t know how to keep myself from getting bad grades. csuf2 5 I don t know what it takes to get good grades in school. csus3 48 Powerful s Strategy Items I won t do well in school if the teacher doesn t like me. cspf1 36 The best way for me to get good grades is to get my teacher to like csps2 61 me. Effort Strategy Items Trying hard is the best way for me to do well in school. cses2 43 Luck Strategy Items I have to be luck to do well in school. csls2 1 If I m unlucky, I won t do well in school. cslf1 31

25 III- 11 Item Item Code Questionnaire Item Number Powerful s Capacity Items I can get my teacher to like me. cpps2 29 I can t get my teacher to like me. cppf2 51 Effort Capacity Items I can work really hard in school. cpes2 32 I can t work very hard in school. cpef3 80 Ability Capacity Items I m pretty smart in school. cpas4 10 I m not very smart in school. cpaf2 40 Luck Capacity Items I m pretty lucky at getting good grades. cpls2 17 I am unlucky in school. cplf1 45 Perceived Autonomy Subdomain The Perceived Autonomy subdomain of the RAPS-SE is made up of nine items that tap two different types of self-regulation, or reasons for doing schoolwork or homework. Five items tap identified self-regulation (i.e., doing schoolwork because it is seen as important), and four items tap intrinsic self-regulation (i.e., doing schoolwork because it is fun or interesting). These two forms of self-regulation are positively related to each other and are all expected to contribute to students academic performance. Two other forms of self-regulation external (i.e., doing schoolwork to avoid punishment or because it is the rule) and introjected (i.e. doing schoolwork or homework to avoid feeling embarrassed or guilty) were included in previous versions of the RAPS-SE but were omitted from the current revised version because they were unrelated to other indices of student performance and adjustment. Autonomy, or Self-regulation, items are listed below: Item Item Code Questionnaire Item Number Identified Self-Regulation Items: I do my homework because I want to understand the subject. ardx1 20 I work on my classwork because I think it is important. ardx9 26 I work on my classwork because I want to learn new things. ardx2 39 I do my homework because I want to learn new things. ardx7 52 I work on my classwork because doing well in school is important to me. ardx8 64 Intrinsic Self-Regulation Items: I do my homework because I like to do it. arix2 6 I work on my classwork because it s interesting. arix8 13 I do my homework because it s fun. arix1 46 I work on my classwork because it s fun. arix3 70

26 III- 12 Perceived Relatedness Subdomain The Relatedness subdomain of the RAPS-SE is made up of 14 items that tap four constructs: 1) Feelings about Self (5 items), 2) Parental Emotional Security (3 items), 3) Teacher Emotional Security (3 items), and 4) Peer Emotional Security (3 items). These four constructs differ from those presented in the middle school version, RAPS-SM in that items reflecting Emotional Security with Self and Satisfaction with Self, two separate constructs on the RAPS-SM, are combined into the single Feelings about Self construct. Three additional constructs were included in previous versions of the RAPS-SE: Parental Psychological Proximity Seeking, Teacher Psychological Proximity Seeking, and Peer Psychological Proximity Seeking. Due to low reliabilities and lack of associations with other indices of student performance and adjustment, however, these three components were excluded from the current revision. The Emotional Security constructs each reflect the extent to which positive emotions are present, and negative emotions absent, when in the presence of a relationship partner (i.e., parent(s), teacher, or peers). Feelings about Self reflects the extent to which students feel negatively about themselves and wish that they were different or were someone else. The 14 Perceived Relatedness items are as follows: Item Item Code Questionnaire Item Number Feelings about Self Items I wish I were someone else. rswx4 12 When I think about myself, I feel bad. rsef11 18 I wish I felt better about myself. rswx3 37 I wish I liked myself better. rswx5 57 When I think about myself, I feel unhappy. rsef10 69 Parent Emotional Security Items: When I m with my parents, I feel mad. rpef2 8 When I m with my parents, I feel unhappy. rpef4 47 When I m with my parents, I feel ignored. rpef1 83 Teacher Emotional Security Items: When I m with my teacher, I feel ignored rtef1 2 When I m with my teacher, I feel mad. rtef2 15 When I m with my teacher, I feel unhappy. rtef4 41 Peer Emotional Security Items: When I m with my classmates, I feel ignored. rref1 25 When I m with my classmates, I feel mad. rref2 44 When I m with my classmates, I feel unhappy. rref4 78

27 III- 13 Experiences of Interpersonal Support Domain The Experiences of Interpersonal Support domain of the RAPS-SE includes a total of 27 items tapping two subdomains: 1) Parental Support; and 2) Teacher Support. Items within each subdomain reflect the extent to which the student feels that the adult(s): 1) are involved with them, 2) provide support for student autonomy, and 3) provide structure. Involvement items reflect the extent to which the student feels that the adult(s) know and care about them. Autonomy Support items concern the extent to which students feel that they are allowed to make decisions for themselves, and the extent to which the real-life value of the work that they are asked to do is made clear by the adult(s). Structure items reflect the clarity of the adult(s) expectations regarding student conduct, the extent to which consequences are consistent and predictable by the student when expectations are not met, and the student s perception of the fairness of the adult(s) expectations. In the RAPS- SE, Parental Support and Teacher Support constitute two separate subdomains, each encompassing involvement, autonomy support, and structure. All items referring to the teacher are specifically about the school setting. Items referring to the parent are either about school and schoolwork (School Context) or do not refer to any specific context (General Context). Experiences of Support from Parents Subdomain Item Item Code Questionnaire Item Number Parent Involvement - School Context Items My parents never have enough time to hear about what happens to me in school. ipxf6 4 Parent Autonomy Support - School Context Items My parents don t explain why school is important. ypxf15 16 When it comes to school, my parents try to control everything I do. ypxf4 30 When it comes to school, my parents are always telling me what to ypxf5 66 do. My parents don t talk about how schoolwork is related to what I want to be. ypxf14 75 Parent Structure - School Context Items My parents don t make it clear what they expect of me in school. spxf10 7 My parents want me to do better than I can in school. spxf12 24 I don t know what my parents expect of me in school. spxf4 85 Parent Involvement - General Context Items My parents don t seem to have enough time for me. ipgf1 62 My parents don t seem to know how I feel about things. ipgf3 87 Parent Autonomy Support - General Context Items My parents don t let me make any of my own decisions. ypgf2 54 My parents trust me. ypgs1 73 My parents try to control everything I do. ypgf1 77

28 III- 14 Experiences of Support from Parents Subdomain - continued Item Item Code Questionnaire Item Number Parent Structure - General Context Items My parents don t think I can do very much. spgf1 11 I never know what my parents are going to do. spgf3 27 I don t know what my parents want from me. spgf2 34 My parents are fair with me. spgs4 55 Experiences of Support from Teachers Subdomain Item Item Code Questionnaire Item Number Teacher Involvement Items: My teacher doesn t seem to have enough time for me. itxf5 38 My teacher likes the other kids in my class better than me. itxf3 68 Teacher Autonomy Support Items: My teacher doesn t explain why we have to learn certain things in ytxf3 28 school. My teacher doesn t talk about how schoolwork is related to what I ytxf14 56 want to be. My teacher interrupts me when I have something to say. ytxf6 63 My teacher tries to control everything I do. ytxf1 81 Teacher Structure Items: My teacher is fair with me. stxs11 9 My teacher s expectations for me are way off base. stxf12 21 My teacher isn t fair with me. stxf11 59 My teacher doesn t make clear what he/she expects of me in school. stxf8 74 Additional Items The following items were selected from the middle school version of the RAPS-S (the RAPS-SM) in order to balance the predominantly negative orientation of the RAPS-SE items. Each item is presented along with the domain and subdomain with which it is associated on the RAPS-SM.

29 III- 15 Item Item Code Questionnaire Item Number Beliefs About Self Domain, Perceived Competence Subdomain: 1 item I can do well in school if I want to. ccxs5 23 Beliefs About Self Domain, Perceived Relatedness Subdomain: 4 items When I m with my parents, I feel good. rpes1 58 When I m with my classmates, I feel good. rres1 67 When I think about myself, I feel proud. rses8 76 When I m with my teacher, I feel good. rtes5 84 Experiences of Interpersonal Support Domain, Support from Parents Subdomain: 3 items My parents like to talk to me about school. ipxs2 79 My parents know just how well I can do in school. spxs11 82 My parents encourage me to find out how schoolwork could be useful to me. ypxs14 86 Experiences of Interpersonal Support Domain, Support from Teachers Subdomain: 1 item My teacher cares about how I do in school. itxs Administration Information Time The RAPS-SE has been designed to be administered to groups of students during one class period (approximately 50 minutes). All materials should be ready to distribute immediately at the beginning of the allotted time, in order to ensure that all students will finish. Oral Questionnaire Administration and Training of Administrators Many of the items on the RAPS-SE ask students to discuss their feelings of relatedness with, and experiences of support from teachers. Because of this, students may be hesitant to respond honestly if the administrator of the RAPS- SE is their teacher. If teachers are administering the RAPS-SE, a plan should be established so that each teacher administers the questionnaire to students other than their own. Questionnaire administrators (i.e. teachers, researchers, school administrators or other members of the school community) should read instructions and all items aloud to students, in addition to providing each student with a copy of the questionnaire. In our work in diverse school settings, we have found that reading to students at all levels speeds up the administration process and decreases the possibility that students will misread questions.