Alberta Children's Hospital, Calgary, Alberta, Canada b University of Calgary, Calgary, Alberta, Canada PLEASE SCROLL DOWN FOR ARTICLE

Transcription

1 This article was downloaded by: [Brooks, Brian L.][Canadian Research Knowledge Network] On: 26 August 2010 Access details: Access Details: [subscription number ] Publisher Psychology Press Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK Child Neuropsychology Publication details, including instructions for authors and subscription information: Behavior Rating Inventory of Executive Function - Preschool Version (BRIEF-P): Test Review and Clinical Guidelines for Use Elisabeth M. S. Sherman ab ; Brian L. Brooks ab a Alberta Children's Hospital, Calgary, Alberta, Canada b University of Calgary, Calgary, Alberta, Canada First published on: 04 May 2010 To cite this Article Sherman, Elisabeth M. S. and Brooks, Brian L.(2010) 'Behavior Rating Inventory of Executive Function - Preschool Version (BRIEF-P): Test Review and Clinical Guidelines for Use', Child Neuropsychology, 16: 5, , First published on: 04 May 2010 (ifirst) To link to this Article: DOI: / URL: PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

2 Child Neuropsychology, 16: , ISSN: print / online DOI: / BEHAVIOR RATING INVENTORY OF EXECUTIVE FUNCTION PRESCHOOL VERSION (BRIEF-P): TEST REVIEW AND CLINICAL GUIDELINES FOR USE Elisabeth M. S. Sherman 1,2 and Brian L. Brooks 1,2 1 Alberta Children s Hospital, Calgary, Alberta, Canada and 2 University of Calgary, Calgary, Alberta, Canada The Behavior Rating Inventory of Executive Function-Preschool Version (BRIEF-P) is a standardized questionnaire that measures executive functioning in preschoolers. This test review provides an overview of the BRIEF-P, including scale structure, administration, norms, score interpretation, current reliability and validity evidence, and provides general guidelines for clinical use. Keywords: BRIEF-P; Preschoolers; Executive function; Test review; Behavior Rating Inventory of Executive Function. OVERVIEW Executive functions are not always as amenable to objective testing as are other domain-specific functions such as language, memory, and visual-spatial skills (Gioia, Isquith, Guy, & Kenworthy, 2000). In addition, because measures of executive functions are administered in a structured, novel, quiet, and one-on-one testing environment, standard performance-based tests do not always allow executive deficits to emerge during administration. Standardized questionnaires that measure executive functions outside of the clinic or laboratory setting completed by the individual or by family members therefore provide important information for the assessment of executive deficits. These questionnaires provide unique information on the degree to which executive deficits are noticed by others and the severity of these deficits in everyday life. The first standardized rating scale designed specifically for use with children was the Behavior Rating Inventory of Executive Function (BRIEF), which appeared in 2000 (Gioia et al., 2000). It served as the basis for the development of the Behavior Rating Inventory of Executive Function - Preschool Version (BRIEF-P; Gioia, Espy & Isquith, 2003), the only standardized executive functioning scale designed specifically for use with preschoolers. The BRIEF-P is intended as one component of a comprehensive evaluation to assess a broad range of childhood disorders (Gioia et al., 2003). Unlike the BRIEF, which has The authors wish to thank Helen Carlson, PhD, for assistance with references and manuscript preparation. Address correspondence to Elisabeth M. S. Sherman, PhD, Neurosciences Program, Alberta Children s Hospital, 2888 Shaganappi Trail NW, Calgary, AB Canada T3B 6A8. elisabeth.sherman@ albertahealthservices.ca 2010 Psychology Press, an imprint of the Taylor & Francis Group, an Informa business

3 504 E. M. S. SHERMAN AND B. L. BROOKS Table 1 Overview of BRIEF-P Scales. Scale Description Number of Items Clinical Scales Inhibit Impulse control; stopping/modulating behavior 16 Shift Ability to shift from one activity to another or solve problems 10 flexibly Emotional Control The ability to modulate emotional responses 10 Working Memory The process of holding information in mind for the purposes of 17 completing a task; staying with an activity Plan/Organize The ability to anticipate future events, to set goals, and to 10 develop steps ahead of time to complete tasks Indexes and Composites Inhibitory Self-Control Ability to modulate actions/responses/emotions/behavior via inhibitory control Flexibility Ability to move flexibly among actions/responses/emotions/ behavior Emergent Metacognition Ability to initiate, to plan, to organize, to implement, and to sustain future-oriented problem solving Global Executive Composite Overall executive function level (summary score based on the five clinical scales) Validity Scales Inconsistency Extent of inconsistency of respondent on selected similar items 10 item pairs Negativity Extent to which respondent responded to selected items in a negative manner 10 Adapted from Gioia et al. (2003). separate parent and teacher forms, the BRIEF-P consists of a single form that can be completed by different raters such as parents, caregivers, preschool teachers, and/or childcare workers. It yields five nonoverlapping theoretically and empirically derived clinical scales, each of which reflects a specific aspect of executive functioning: Inhibit, Shift, Emotional Control, Working Memory, and Plan/Organize (Table 1). These clinical scales yield three composite indexes: the Inhibitory Self-Control Index (ISCI), Flexibility Index (FI), and Emergent Metacognition Index (EMI). The overall composite index is the Global Executive Composite (GEC; Figure 1). In addition, the BRIEF-P includes two scales designed to assess validity of responses (Inconsistency and Negativity). ADMINISTRATION AND SCORING Age Range The scale can be administered to children aged 2 years 0 months to 5 years 11 months. Administration Time The BRIEF-P takes approximately 10 to 15 minutes to complete. Reading Level Respondents should have a fifth-grade reading level and be proficient in English.

4 TEST REVIEW: BRIEF-P 505 Figure 1 Clinical scales, indexes, and composite score for the BRIEF-P. Materials The BRIEF-P consists of a professional manual, rating form, and tools for scoring the protocols (i.e., software package for computerized scoring, and scoring sheet and profile form for hand-scoring). Instructions for respondents are printed on the front of the rating form, which can be administered to both parents and other raters such as teachers or childcare workers. Format, Instructions, and Administration Respondents are asked to rate the child s behavior on a 3-point scale ( Never, Sometimes, and Often ) in terms of how often, in the last 6 months, the particular behavior has been a problem (this differs from the BRIEF, which asks how often the behavior occurs; Gioia et al., 2000). This specific instruction is repeated inside the BRIEF-P test form at the top of each page. Given that most preschoolers exhibit some of the behaviors included in the form, the authors also supply additional instructions to the rater that emphasize it is the degree to which behaviors are a problem that should be rated, rather than the frequency with which the behaviors occur. The manual includes additional detailed instructions that may be used as a guide for instructing raters (Gioia et al., 2003, p. 6). It is not uncommon for raters to inadvertently omit responses on the rating form. The manual indicates that it is important to check the form for missing items at the end of the administration and have the rater fill in any missing responses. Choosing the Parent Rater The manual suggests that both parents be administered the BRIEF-P separately to obtain more information and to identify areas of disagreement. When only one parent is available, it should be the parent with the most contact with the child in the last 6 months.

5 506 E. M. S. SHERMAN AND B. L. BROOKS Users should note that the vast majority of the parent normative data was gathered from mothers (89%). Mothers and fathers yielded equivalent scores in the normative sample (Gioia et al., 2003), which suggests that either parent would be a suitable rater. Nevertheless, given the parental differences reported on other scales (e.g., Mulligan, Etheridge, Kassoumeri, Wedderburn, & Newman, 2009), maternal ratings would be preferable to be consistent with the normative sample unless the father is the primary caregiver or the child is being raised by other family members. Data from raters with low representation in the normative sample (e.g., grandparents) should be used with some caution in making clinical inferences. The manual does not indicate what to do with regard to diagnostic decision making when parents disagree, or when only one parent provides clinical-range ratings. Using the same rater at baseline and retest makes clinical sense so as not to introduce rater effects when interpreting changes over time. Choosing the Caregiver/Teacher Rater The manual indicates that one month of daily contact by a teacher or child care worker should be sufficient to serve as a rater. The median time caregivers reported knowing a child in the normative sample was 6.5 months. There is a slight effect of familiarity in that caregivers who indicated they knew a child Moderately Well produced slightly higher ratings (signifying more impairment) than those who reported knowing the child Very Well, but overall, teacher/caregiver familiarity appears to account for less than 2% 3% of the variance in BRIEF-P ratings (Gioia et al., 2003). Of note, there is no research on whether different kinds of raters yield similar information or whether some raters have more sensitivity at detecting problems than others (e.g., daycare workers vs. preschool teachers). Scores Clinical scales and composite index scores yield T scores and percentiles based on gender and age, and according to rater (parent or teacher/caregiver). Validity scales (Inconsistency and Negativity) are scored with regard to specific cutoffs indicated on the scoring form (see below for interpretation of these scales). Omitted response can be assigned a rating of 1 ( Never ) in order to calculate the T score for the clinical scale, but the presence of three or more omitted items from a clinical scale invalidates that particular scale. Furthermore, having 13 or more omitted items from the entire protocol invalidates the BRIEF-P. Confidence Intervals Of note, 90% confidence intervals can be calculated for scales and indexes according to the procedure outlined in the manual and summarized in Table 2. For clinical scales, 90% confidence intervals typically range from 4 to 8 T-score points. The indexes have 90% confidence intervals ranging from 3 to 6 T-score points, whereas the GEC ranges from 3 T-score points for parent ratings to 4 T-score points for teacher ratings. Cutoffs for Clinical Significance The recommended cutoff for abnormal elevation and potential clinical significance is a T score of 65, which represents a score 1.5 SDs above the normative mean, or a

6 TEST REVIEW: BRIEF-P 507 Table 2 Confidence Interval Values for Calculating 90% Confidence Intervals for the BRIEF-P. VALUES FOR CONSTRUCTING CONFIDENCE INTERVALS Parent Ratings Teacher Ratings Boys Girls Boys Girls Age 2:0 3:11 Age 4:0 5:11 Age 2:0 3:11 Age 4:0 5:11 Age 2:0 3:11 Age 4:0 5:11 Age 2:0 3:11 Age 4:0 5:11 Inhibit Shift Emotional Control Working Memory Plan/Organize ISCI FI EMI GEC To calculate confidence intervals, subtract the above value from the child s obtained T score to obtain the low end of the interval; add the above value to the obtained T score to obtain the high end of the interval. Adapted from Tables A1 B12 in Gioia et al. (2003). percentile of 93 or higher. Notably, T score to percentile correspondence is not uniform across all subscales because raw score distributions (and resulting skew) differ. Consequently, a T score of 65 may not always correspond to the normal-distribution-based 93rd percentile, and a T score of 65 on one scale might not yield the same percentile as a T score of 65 on another scale. Because the percentiles involve direct translation from raw score distributions, it makes sense to use percentiles as the benchmark for detecting clinically significant elevations rather than the T score. This suggestion, however, differs from the authors recommendation to use a cutoff of T score > 65 for clinical significance. Interpretation and Score Discrepancies According to the manual, the GEC is best interpreted when the three index scales are not significantly different from each other. To assess this, a table of cumulative percentages of T-score differences for the three indexes is shown in the BRIEF-P manual on p. 20 (Gioia et al., 2003), and discrepancy cutoffs occurring in less than 10% of the normative sample are summarized here in Table 3. Discrepancies of this magnitude suggest that the Table 3 Index Score Discrepancies Occurring in less than 10% of Children in the BRIEF-P Normative Sample. Parent Ratings Teacher Ratings ISCI vs. FI ISCI vs. EMI EMI vs. FI Note. ISCI = Inhibitory Self-Control Index; EMI = Emergent Metacognition Index; FI = Flexibility Index. Adapted from Gioia et al. (2003).

7 508 E. M. S. SHERMAN AND B. L. BROOKS child exhibits marked differences between index scores instead of a more uniform index score profile. In cases with such discrepancies, the respective index scores may provide a more accurate clinical picture than the GEC. The next level of discrepancy analysis (i.e., whether the individual index scores can be interpreted if the clinical scales are very discrepant) has not been formally established, but the decision to interpret single clinical scales over composite index scores should be based, in part, on the reliability of that particular scale. For most tests and scales in common use, reliabilities for the index scores exceed those of the subscales that comprise them (Strauss, Sherman, & Spreen, 2006). In the case of the BRIEF-P, many of the clinical-scale reliabilities rival or exceed some index scores (see Evidence for Reliability, below). Importantly, BRIEF-P scoring can be accomplished by persons without formal training in psychology, but the authors emphasize that interpretation of scores and profiles requires graduate training in neuropsychology, psychology, or related fields, as well as postsecondary training in the interpretation of psychological tests (Gioia et al., 2003). Interpreting the Inconsistency Scale The Inconsistency scale consists of 10 item pairs with similar but not identical content. High scores on this scale indicate that the rater responded in an inconsistent manner within item pairs compared to the normative sample and the clinical sample. The manual notes that calculation of the Inconsistency scale must be made carefully because it is somewhat complex. For each pair, the absolute value of the difference between items is calculated, and the absolute differences for each item pair are summed. Based on this total, protocols are rated as either Acceptable or Inconsistent. Acceptable protocols are those with raw scores up to and including 7 (parent ratings) or 6 (teacher/caregiver ratings). When a BRIEF-P protocol yields an elevated Inconsistency score, it is important to review the items that led to the high score. Although relatively uncommon in clinical practice, this kind of protocol is not necessarily invalid. High Inconsistency scores could potentially be obtained if the rater was very detail oriented or concrete, or if the child s behavior is very situation specific. More research is needed on interpreting this scale in clinical samples. Interpreting the Negativity Scale The Negativity Scale reflects the extent of negative bias on the part of the respondent compared to those in the BRIEF-P normative and clinical samples. A high Negativity score indicates that the respondent endorsed Often for a specific number of items on the scale (four for the parent and three for the teacher), and that this score was found in less than 1% (parent scale) or 2% (teacher scale) of the combined normative and clinical samples. The raw score is computed for these items, with raw scores being Acceptable up to and including three for parent ratings and two for teacher ratings. The authors recommend that protocols should only be deemed invalid after a careful review of items in the context of other test scores, history, and observations, because severe executive dysfunction can be associated with an elevated Negativity scale. An illustrative case example of severe attention deficit/hyperactivity disorder (ADHD) is shown in the manual. As is the case for the Inconsistency scale, more research is needed on validity in clinical samples.

8 TEST REVIEW: BRIEF-P 509 Of note, the BRIEF-P does not include a validity scale for detecting overly positive responding or denial of the presence of any dysexecutive-type behavior. Protocols with endorsement of Not at all for all items should be interpreted with caution, as they may reflect a rater who was not fully engaged in the task, who misunderstood the instructions, who is minimizing problems, or who has an overly positive view of the child. NORMATIVE DATA Standardization Sample The BRIEF-P item tryout phase involved administration of a longer version (93 items) to children in preschools in the Midwest, Mid-Atlantic, and Northeast regions of the United States. The actual standardization edition (63 items) was administered in Maryland, Illinois, Vermont, New Hampshire, Florida, and Texas. Children were recruited through public and private school recruitment and pediatric well-child visits; in total, 20 preschool programs were sampled. The try-out and standardization samples were combined to form the final normative sample. According to the manual, gender and race/ethnicity for the normative sample was based on the 1999 US Census bureau information. A total of 460 Parent Forms were completed by respondents in the normative sample; the Teacher Form was completed for 302 of these. The normative sample contains a fairly high proportion of upper middle and upper class families (about 48%) and the mean parental education was high (15.7 years; SD = 2.84). The majority of respondents in the normative sample for parent ratings were mothers (i.e., n = 408; 89%), with fathers representing 7.1% of the sample and the remainder consisting of grandparents and foster/adoptive parents. See Table 4 for other sample characteristics. Age Scales measuring executive function in children should show evidence of increases in executive skills with age, and this is true of the BRIEF-P. Age effects are found on parent ratings of Inhibit, Emotional Control, and Plan/Organize, and on the ISCI, FI, and GEC. This is also the case for teacher ratings of Shift, Emotional Control, and on the FI. The two normative age groups classifications used in the BRIEF-P (2 3 years vs. 4 5 years) are based on the distributions across age that show a peak in mean scale scores for 2 3 year olds (Gioia et al., 2003). Sex Sex is related to BRIEF-P ratings, with boys generally obtaining higher ratings (i.e., worse problems) than girls. In parent ratings, sex effects are found for Inhibit; in teacher ratings, sex effects are found for Inhibit, Working Memory, and Plan/Organize and on the ISCI, EMI, and GEC, according to the manual. Therefore, separate BRIEF-P norms are provided for gender, along with age. Education/Socioeconomic Status (SES) The manual indicates that there is a low, negative correlation between parental education and BRIEF-P ratings (range =.05 to.22), with the highest correlation accounting for 5% of the variance. SES is only mildly related to BRIEF scores in the

9 510 E. M. S. SHERMAN AND B. L. BROOKS Table 4 Characteristics of the BRIEF-P Normative Sample. Sample Type Parent Ratings Teacher Ratings n Age 2:00 to 5:11 Same Geographic Location Described as including Maryland, Illinois, Vermont, New Hampshire, Florida, and Texas Gender Boys 53.5% 54.3% Girls 46.5% 45.7% Race/Ethnicity White 73.0% 71.9% African American 13.9% 12.3% Hispanic 4.8% 4.6% Asian/Pacific Islander 3.0% 2.0% Native American/Eskimo 0.7% 0.7% Unspecified 4.6% 8.6% SES Upper 18.9% 23.5% Upper-Middle 28.9% 29.1% Middle-Middle 26.3% 24.2% Lower-Middle 15.7% 12.3% Lower 10.0% 6.6% Unassigned 0.2% 4.3% Screening Excluded if history of special education, attention problems, developmental difficulties, psychotropic medication usage, or more than 10% of items missing. Norms are organized along two age groupings (ages 2 3, and 4 5) and are based on US Census bureau information for Adapted from Gioia et al. (2003). standardization sample (i.e., <2% of the variance), with children from low-ses families tending to be rated as having more executive problems. Caution is nevertheless recommended in interpreting BRIEF-P scores of disadvantaged children and children from families with low parental education (see Conclusions and General Guidelines for Clinical Use). Ethnicity Ethnicity did not have a significant impact on BRIEF-P ratings in the standardization sample (see manual; see also Conclusions and General Guidelines for Clinical Use). Normative Cell Sizes Overall, the age- and sex-based cell sizes are relatively uniform (see p. 43 in Gioia et al., 2003). For parent ratings, the smallest cell size is for 2-year-old boys (n = 30), and the largest is for 4-year-old boys (n = 85). For teacher ratings, the smallest cell size is for 2- year-olds (boys, n = 16; girls, n = 17) and the largest is for 4-year-old boys (n = 66). However, combining age groups for the normative scores results in larger sample sizes, ranging from n = 63 (teacher ratings for girls years 0 months to 3 years 11 months) to n = 133 (parent ratings for boys 4 years 0 months to 5 years 11 months).

10 TEST REVIEW: BRIEF-P 511 EVIDENCE FOR RELIABILITY Evidence for Internal Consistency Cronbach s alphas for the BRIEF-P scales are high for the parent version (r =.80 to.90) and very high for the teacher version (r =.90 to.97 on the scales; Gioia et al., 2003). Reliabilities are very high for the GEC for both parent and teacher ratings in the normative sample (r =.95 and.97, respectively). For both parent and teacher ratings, FI has marginally lower internal reliability (r =.89 and.93, respectively) compared to EMI (r =.91 and.95, respectively); both, however, are strong. Individual reliabilities for all the clinical scales and index scores are shown in the manual on p. 47. In the normative sample, the lowest reliabilities are for Plan/Organize for parent ratings (r =.80) and Shift for teacher ratings (r =.90); both are within acceptable levels for clinical use. Evidence for Test-Retest Reliability The magnitude of test-retest reliability coefficients is presented in Table 5, organized according to size. Higher correlations, particularly those over.90, are optimal for clinical decision making, whereas lower correlations (e.g.,.70), may be suitable for other purposes such as research (Strauss et al., 2006). Score stability over time is very high for the Inhibit scale, the ISC Index, and the GEC, based on 52 parent ratings and 67 teacher ratings obtained after a mean interval of about 4 weeks (Gioia et al., 2003). The FI from teacher ratings demonstrates somewhat lower reliability than the other index scores, and the reliability of the Shift scale is only marginal. Most scales, however, demonstrate at least high reliability and are therefore of sufficient stability for clinical use. Reliable Change Scores Interpretation of change scores using the 80% and 90% confidence intervals is facilitated with the reliable change information presented in Table 6. This table presents both reliable change and adjusted reliable change values (adjusted for exposure effects, see below) that can be used for determining whether a change is reliable (i.e., whether the change exceeds measurement and exposure effects), based on the sample used to determine Table 5 Test-Retest Reliability Coefficients for the BRIEF-P. Magnitude of Coefficient Parent Ratings Teacher Ratings Very High (.90+) High (.80 to.89) Inhibit Inhibitory Self-Control Index Global Executive Composite Shift Emotional Control Working Memory FlexibilityIndex Emergent Metacognition Index Inhibit Inhibitory Self-Control Index Emotional Control Working Memory Plan/Organize Emergent Metacognition Index Global Executive Composite Adequate (.70 to.79) Plan/Organize Flexibility Index Marginal (.60 to.69) Shift Low (<.59) Note. Index and composite scores shown in italics. Adapted from Gioia et al. (2003).

11 Table 6 Reliable Change Scores for the BRIEF-P Parent and Teacher Ratings. Unadjusted Change Scores Adjusted Change Scores 80% CI 90% CI BRIEF-P Scores r 12 SE diff 80% CI 90% CI Exposure Effect (Mean Time 2 Mean Time 1) Increase T score/ More Problems Decrease T score/ Fewer Problems Increase T score/ More Problems Decrease T score/ Fewer Problems Parent Ratings Inhibit Shift Emotional Control Working Memory Plan/Organize ISCI FI EMI GEC Teacher Ratings Inhibit Shift Emotional Control Working Memory Plan/Organize ISCI FI EMI GEC Note. Change scores are T scores, with higher T scores representing increased severity of problems. Parent retest sample, n = 52 and retest interval = 4.5 weeks (range = weeks). Teacher retest sample, n = 67 and retest interval = 4.2 weeks (range = weeks). r 12 = retest correlation. SE diff = standard error of the difference, which is calculated using the following formula: SE diff = [(SEM 1 ) 2 +(SEM 2 ) 2 ], where SEM = SD* (1-r 12 ). CI = confidence interval. ISCI = Inhibitory Self-Control. FI = Flexibility. EMI = Emergent Metacognition. GEC = Global Executive Composite. Unadjusted change scores do not include exposure effects. 80% CI = SE diff * % CI = SE diff * Exposure effect is the difference between mean scores (time 2 time 1): Negative values indicate a decrease in mean T-score ratings (and improvement in behavior) and positive values indicate an increase in mean T-score ratings (and worsening of behavior). Adjusted change scores are corrected for exposure effects. Change scores greater than the values presented are considered statistically reliable at the presented confidence interval. Adapated and reproduced by special permission of the Publisher, Psychological Assessment Resources, Inc., North Florida Avenue, Lutz, Florida 33549, from the Behavior Rating Inventory of Executive Function-Preschool Version by Gerard A. Gioia, PhD, Kimberley Andrews Espy, PhD, Peter K. Isquith, PhD, Copyright 1996, 1998, 2000, 2001, 2003 by PAR, Inc. Further reproduction is prohibited without permission from PAR, Inc. 512

12 TEST REVIEW: BRIEF-P 513 test-retest reliability, as described above. This sample was retested after a mean interval of 4 weeks, which makes it suitable for assessing changes over relatively short test-retest intervals. Exposure Effects The authors examined the magnitude of T-score changes over time for the retest groups to see whether ratings increase or decrease after re-administration (i.e., exposure effect). For the parent and teacher ratings, T-score changes were minimal, on average (i.e., 2.0 points or less). Scales showing the largest changes included the Plan/Organize scale for both parent and teacher ratings (2.9 and 3.0 points lower at retest, respectively). Overall, these results indicate that the BRIEF-P is suitable for repeat assessments. Of note, users should be cautious about making clinical inferences about very small decreases in scores over time, as incremental decreases in scores (i.e., improvements in behavior) are typical of most child-rating scales administered repeatedly over time (e.g., Twenge & Nolen-Hoeksema, 2002). Evidence for Interrater Reliability Because different kinds of raters perceive child behavior differently (e.g., parents vs. teachers), interrater reliability of child behavior scales should ideally be provided through multiple ratings of the same kind of rater (e.g., two daycare providers). Interrater reliability of this type is not provided in the manual. Information on the concordance of parent and teacher/caregiver ratings is discussed below. EVIDENCE FOR VALIDITY Evidence for Content Validity The BRIEF-P was constructed in much the same way as the original BRIEF. In the case of the BRIEF-P, the original BRIEF items were rewritten for younger children and additional items were developed based on reviews of clinical interview notes from the three authors clinical work. In particular, two broad kinds of items were included: items reflecting concrete instances of particular behavioral domains along with general descriptions of the same domains. According to the authors, scale structure was initially derived through literature review and consultation with colleagues and then verified with factor analysis. Item content was based on review of interview notes of actual clinical cases in order to generate items reflecting specific behaviors and characteristics. Common behavior-rating scales were also reviewed to minimize redundancy with other general scales. Item tryouts were carried out in three regions of the United States for 372 parents and 201 teachers, followed by item analysis involving item-total correlations and then principal factor analysis with orthogonal rotation. Following this procedure, two entire clinical scales derived originally from the BRIEF were eliminated: Initiate and Monitor, and the BRIEF Organization of Materials scale was subsumed under Plan/Organize in the BRIEF-P. A third scale developed specifically for the BRIEF-P was also discarded (Regulation of Stimulation) based on insufficient psychometric evidence. A full description of item derivation procedures is described in the manual.

13 514 E. M. S. SHERMAN AND B. L. BROOKS Relationship to Demographic Variables Associations with demographic variables (age, gender, ethnicity, SES) are presented above, under Normative Data. Parent and Teacher/Caregiver Agreement Correlations between parent and teachers/caregiver ratings are presented in the manual for 302 children. Overall, parent-teacher correlations are moderate at best (average r =.19), which is not unusual for parent/teacher-rating agreement for a behavior scale but does highlight the need for multiple informants before assigning clinical diagnoses. In particular, the authors note that agreement was stronger for Inhibit, Shift, and Emotional Control (and consequently ISCI and FI), with rs between.24 and.28. Of note, agreement for Plan/Organize was particularly low (r =.06). Parents consistently rated children as having more difficulties on all scales than did teachers; a finding that differs from research on older children including research on scales designed to measure ADHD, where teachers typically give higher ratings than parents (Mares, McLuckie, Schwartz, & Saini, 2007). Factor Structure The manual describes results from principal factor analysis with oblique rotation (Promax) on the normative sample. A three-factor solution for the parent ratings emerged that accounted for 87% of the variance. The first factor was comprised of Working Memory and Plan/Organize (with a secondary loading on Inhibit), the second factor consisted of Shift and Emotional Control, and a third factor consisted of Inhibit and Emotional Control. A similar factor solution was replicated in a mixed clinical sample (N = 50). For the teacher ratings, a three-factor solution accounted for 92% of the variance. The same factor composition was derived. See below for results of factor analyses involving other behavior-rating scales. Concordance with Performance-Based Executive Tests and Nonexecutive Neuropsychological Tests To date, there is only a single study on the association between BRIEF-P scores and performance-based measures of executive functioning. Mahone and Hoffman (2007) reported that BRIEF-P scores in preschoolers with ADHD have low, nonsignificant correlations with self-ordered pointing, auditory digit span, inhibition and motor persistence, visual attention, and sustained attention. However, these correlations were no higher than correlations with receptive vocabulary or sensorimotor measures. The authors concluded that the BRIEF-P measures different components of executive functioning than those measured by performance-based executive functioning tests. With regard to IQ, the manual indicates that there is a modest correlation between BRIEF-P Working Memory and IQ as measured by the Differential Abilities Scale (DAS). This mirrors data from clinical samples on the original BRIEF, where scale elevations on Working Memory are common in children with intellectual deficits (Slick, Lautzenhiser, Sherman, & Eyrl, 2006).

14 TEST REVIEW: BRIEF-P 515 Relationship to Behavior Rating Scales According to the manual, agreement between parent ratings derived from BRIEF-P and other scales show good evidence for the test s construct validity; although this evidence is based on only a small sample size (N = 20). Specifically, there are large correlations between parent-rated BRIEF-P Working Memory and Plan/Organize scales (and EMI Index) and Attention Problems, Withdrawn, and Emotionally Reactive scales on the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2000). Correlations with the Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1998) indicate that the Inhibit and Plan/Organize scales are highly related to Hyperactivity (r >.70), whereas the Shift and Emotional Control dimensions tap more of affective/adaptive domains (BASC Withdrawal, BASC Depression, BASC Adaptability; rs >.70). Factor analyses of the BRIEF-P along with the behavioral rating scales indicate that the test contributes unique information not covered by these scales. Relationship to ADHD Screening Scales The BRIEF-P measures dimensions of behavior that are very similar to those assessed by ADHD screening scales, according to data presented in the manual. There is a high degree of overlap between the BRIEF-P and conceptually related scales from the preschool version of a well-known ADHD screening scale, the ADHD Rating Scale IV Preschool (rs >.80, N = 135). A similar pattern is reported for teacher/caregiver ratings. In a sample of children with ADHD, BRIEF-P scores were very highly correlated with ADHD symptoms as measured by the Conners Parent Rating Scale (r =.81, GEC and ADHD Index; Gioia et al., 2003). Sensitivity to Clinical Problems Although more research is needed, there is emerging support for the clinical utility and sensitivity of the test to different manifestations of executive dysfunction in the context of identified clinical disorders. The manual includes information on BRIEF-P profiles in children with ADHD (n = 17), premature birth (n = 43), language disorder (n = 21), autism (n = 16), and in a mixed clinical group who were referred for a neuropsychological assessment (n = 50). The manual highlights how children from these samples obtained almost uniformly and significantly higher raw scores on BRIEF-P scales than matched controls. In particular, every BRIEF-P clinical scale was highly elevated in the autism group, consistent with the conceptualization of autism as a disorder with prominent executive dysfunction. Although these data suggest that the BRIEF-P may yield signature profiles that might discriminate between these groups, the authors indicate that BRIEF-P profiles should not be considered the only possible scale configuration in these groups and that a comprehensive assessment is needed for making diagnoses. Although the manual spends considerable time on the mean raw score differences between groups, what is of particular clinical utility is the proportion of children in each clinical group obtaining scores above a clinically significant cutoff within the different diagnostic groups. For example, clinically significant elevations on the GEC are most common in autism (81%) and ADHD (71%), but significantly less common in other groups. This lends support for the clinical utility and sensitivity of the test, because these children would be expected to have the most severe problems with executive functioning compared to the other clinical groups.

15 516 E. M. S. SHERMAN AND B. L. BROOKS Independent studies on the BRIEF-P in clinical groups are needed. In the only study we could find as of this writing, preschoolers with ADHD had elevations on all five of the clinical scales versus matched controls, but the largest difference was on the Working Memory scale, followed by Plan/Organize (Mahone & Hoffman, 2007). CONCLUSIONS AND GENERAL GUIDELINES FOR CLINICAL USE Measuring executive functioning in the structured testing environment of neuropsychological evaluation presents a challenge at any age, but this challenge is especially acute in the assessment of preschoolers whose proficiency in language, memory, and motor skills are not yet established and who have difficulty staying on task for prolonged periods. As a way of capturing parent and caregiver ratings of preschoolers across everyday settings, the BRIEF-P appears uniquely suited to providing estimates of executive functioning in this age group and can be seen as a crucial component of the neuropsychological assessment of preschoolers. Overall, the BRIEF-P is an extremely well-designed, psychometrically sound instrument that has great potential in refining the assessment of preschool children. The BRIEF-P manual is well written, comprehensive, and detailed, and the scale s reliability evidence supports its use for clinical diagnostic decision making involving the presence of executive deficits in preschoolers. Although further research is needed, the validity scales (Inconsistency, Negativity) are also a welcome feature. One very important consideration for busy clinicians strapped for time and resources is whether the BRIEF-P replaces or supplements commonly used tests and scales in the assessment of preschoolers. In particular, does the scale measure something distinct and unique compared to performance-based executive tests, and what is the degree of overlap of the scale with commonly used behavioral rating scales, such as those designed to measure ADHD? First, whether the BRIEF-P is more or less sensitive than performance-based executive tests remains an open question, but, at present, BRIEF-P scores and performance-based executive testing appear to have minimal, nonspecific overlap, insofar as can be determined from a single study (i.e., Mahone & Hoffman, 2007). Thus, the issue for clinicians may not be whether to administer performance-based or behavior scales of executive functioning but rather to be cognizant of the strengths and weaknesses of both approaches for assessing executive deficits in children (see Strauss et al., 2006, for a review of the BRIEF and discussion of this question). The concepts of two separate but interlacing domains of cognitive and behavioral executive functions, assessed respectively by performance-based objective tests and standardized questionnaires, is a model of executive functioning that may have neuroanatomical underpinnings in the cortical-cortical and cortical-subcortical frontal circuits, respectively (Jacobs, Harvey, & Anderson, 2007). Until further research clarifies the differential sensitivity of performance-based and observer-based ratings of executive functioning, both appear to have their place in the neuropsychological work-up of preschoolers. Second, the scale appears to contribute something unique not covered by other behavioral rating scales such as the CBCL and BASC. In addition, of the five BRIEF-P clinical scales, Shift and Emotional Control appear to be the scales that most tap into affective dimensions such as depression and anxiety. Whether the dimension of Emotional Control is purely executive or rather reflects aspects of mood and affect is a question worthy of further study. It does suggest to clinicians that a careful assessment of mood and affective problems may be warranted in children presenting with sole elevations on the

16 TEST REVIEW: BRIEF-P 517 Emotional Control scale, and that a primary problem with executive dysfunction may not be the best descriptor for these children. More research into the commonalities and differences of the BRIEF-P and those designed to detect ADHD symptoms would be of utility. The BRIEF-P has very high conceptual and correlational overlap with ADHD scales, and more information on whether the scale can be used in ADHD screening would be helpful. Although the BRIEF-P has few limitations, these deserve mention. Caution is recommended in interpreting BRIEF-P scores and diagnosing executive deficits in children from disadvantaged backgrounds. The number of parents from high-ses backgrounds approaches 50% in the normative sample, and mean parental education is high (almost 16 years). Because parental lower education/ses is associated with worse executive dysfunction in the normative sample, this may introduce a potential risk of overdiagnosis of executive dysfunction in children of families with low SES/parental education. This issue is complex, however, because diagnoses involving executive dysfunction (such as ADHD) may be more frequent in low-ses children (Dopfner, Breuer, Wille, Erhart, & Ravens-Sieberer, 2008) and in some minority groups (Miller, Nigg, & Miller, 2009). In the same vein, the relative underrepresentation of some minority groups in the normative sample also suggests caution in making diagnostic decisions involving minority children. Hispanic children comprise 5% of the BRIEF-P normative sample, but Hispanic children now represent over 10% of children under age 5 in the United States (US Census Bureau, 2008) with trends projecting rapid increases in this age group over time. More information on how to integrate results from different raters is needed because discrepancies frequently arise in clinical settings and because the agreement between parent and teacher ratings, particularly for some scales, is fairly low (see previous discussion in Parent and Teacher/Caregiver Agreement). Unfortunately, guidelines do not exist on what to do when faced with discrepant information from two raters, or on whether some raters are more accurate in detecting some components of behavior than others. When disagreement exists across raters, the clinician will need to carefully consider the potential reasons for these differences and may consider administering additional ratings scales that tap executive dysfunction (e.g., ADHD scales). Teachers may have more access and exposure to a larger range of normative behaviors in children, which may have implications when faced with discrepancies across raters. However, teacher scales may not necessarily be more accurate than parent scales, particularly if the teacher is unfamiliar with the child or if subtle or nondisruptive aspects of executive dysfunction such as working memory problems are prominent. In our experience, when parent and teacher scales are very discrepant, performance-based measure of executive functioning, behavioral observations during the assessment, and corroborating evidence from other sources (e.g., other questionnaires, school records, in-classroom observations) become even more important when clarifying the diagnostic conceptualization. Of note, the problem of how to interpret discrepant information from parents and teachers is not specific to the BRIEF-P but represents one of the complexities inherent in assessing children s behavior for diagnostic purposes (e.g., Power, Costigan, Leff, Eiraldi, & Landau, 2001). Lack of concordance between the BRIEF-P and performance-based measures of executive function, in contrast, should not be interpreted as lack of evidence for executive deficits in this age group, because these appear to measure relatively independent (or possibly, situation-specific) constructs, insofar as the limited BRIEF-P research on this question suggests. Additional independent research involving clinical groups is needed, as is information on classification accuracy statistics (e.g., positive and negative predictive power) and

17 518 E. M. S. SHERMAN AND B. L. BROOKS replication of the factor structure. From a clinical-interpretive standpoint, users are reminded that the three indexes (ISCI, FI, and EMI) are not completely independent. FI and ISCI both include ratings from the Emotional Control scale. Therefore, caution is recommended in interpreting BRIEF-P profiles of children with specific weaknesses on Emotional Control, as this single deficit could potentially affect scores on both the FI and ISCI. The authors guidelines on interpreting profiles involving significant scale discrepancies should be considered before interpreting specific index scores. The BRIEF-P and original BRIEF can both be given to 5-year-olds; consequently, the clinician has a choice of scales when assessing children in this age group. One of the main differences between the original BRIEF and BRIEF-P is that the latter does not include a finer measurement of metacognitive processes (e.g., Inhibit, Monitor), likely because these abilities are not fully developed in preschoolers. This was determined both empirically by examination of factor analytic results and by expert consensus in the initial stages of scale development by the authors. The BRIEF might therefore be a better choice for 5-year-olds because it may yield a more comprehensive assessment of these different components. On the other hand, the BRIEF-P normative sample is superior to that of the original BRIEF in terms of geographical representativeness and in terms of sample sizes for parent and teacher forms. Although the BRIEF-P samples from four different US regions, the original BRIEF normative sample originates only in Maryland, along with a small number of cases from a research study. As well, because of its school-based focus, some items on the BRIEF may not be suitable for all 5-year-olds. This may include children who have not yet started kindergarten, or children attending unstructured or alternative-type kindergarten classes. The BRIEF-P may therefore be preferable in these cases. In addition, the BRIEF-P and BRIEF differ slightly in terms of the instructions given to the rater. Specifically, the BRIEF-P measures the degree to which behaviors are perceived to be a problem by the rater compared to other children, rather than the frequency of behaviors. This is a subtle distinction that may or may not affect scores, but one that could potentially affect retest scores involving the administration of a BRIEF-P followed by a BRIEF. In sum, the BRIEF-P is a major advancement in the assessment of executive functioning in preschoolers, and its routine inclusion in most neuropsychological evaluations of preschoolers is highly recommended. The BRIEF-P is particularly useful in the assessment of preschoolers because norm-referenced, reliable, performance-based measures of executive functioning in this age group are limited. Moreover, the BRIEF-P provides information on executive functions in daily life, which may have more external validity than performance on a laboratory-based measure. The test has been adopted enthusiastically by pediatric neuropsychologists, particularly those who assess children who may present with a high base rate of executive disorders, such as children with ADHD, autism, and traumatic brain injury, but is equally appropriate for screening in most settings where children are assessed for cognitive or behavioral problems. Original manuscript received January 27, 2010 Revised manuscript accepted January 29, 2010 First published online May 4, 2010 REFERENCES Achenbach, T. M., & Rescorla, L. A. (2000). Manual for ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.