8 Best Practices in Problem Analysis

Transcription

1 8 Best Practices in Problem Analysis Theodore J. Christ Yvette Anne Arañas University of Minnesota OVERVIEW The purpose of this chapter is to describe the theoretical foundations and practical significance of problem analysis. While a problem is defined as an unacceptable discrepancy between expected and observed performance, problem analysis is the systematic process of assessment and evaluation to better understand the nature and possible solution for the problem. Problem analysis includes the collection, summary, and use of information to verify or reject relevant hypotheses related to both the cause and solution of a problem. Problem analysis contributes to the Data-Based Decision Making and Accountability domain in the National Association of School Psychologists (NASP) Model for Comprehensive and Integrated School Psychological Services (NASP, 2010), which requires that practitioners competently use assessments and data to assess and meet the needs of students. This chapter describes the theoretical foundation of problem analysis and describes its relevance at the individual, group, and systems level. School psychologists who are not experts in a specific content area will learn how to use a content-specific hypothesis-testing framework as a guide to formulate intervention recommendations. This chapter encourages school psychologists to seek or develop additional contentspecific frameworks to organize hypothesis testing and make concrete connections to the relevant research. BASIC CONSIDERATIONS Problem analysis is best guided by context- and contentspecific expertise. The context is often the school and classroom setting. The content is often related to academic or social-emotional skills and performance. A school psychologist who lacks expertise in either the context or content might still function as a successful problem analyst if he or she relies on science and evidence to make decisions. This section establishes that problem analysis is scientific, relies on low-level inferences, and focuses on alterable variables. Scientific Method An understanding and appreciation for science is inherent to problem analysis. Both the scientific method and scientific body of evidence inform problem analysis. In brief, the scientific method is the process of inquiry and discovery. The scientific body of evidence is the knowledge and interventions that emerge for systematic research. The steps for problem analysis are consistent with that of the scientific method. They are (a) identify a problem; (b) hypothesize likely causes; (c) select methods for assessment such as reviews, interviews, observations, and tests; (d) collect data; (e) review data; and (f) revise hypotheses regarding likely causes or, if the cause is isolated, form a hypothesized solution. Table 8.1 presents an application of the scientific method to an example of a student with a reading problem. There are two types of hypotheses associated with problem analysis: analytic and intervention (see Table 8.2). An analytic hypothesis is developed in the second step of the scientific method and relates to the likely causes of the problem. Analytic hypotheses guide analysis with improved articulation of the purpose for data collection. That is, data are collected for a reason and analytic hypotheses define those reasons throughout problem analysis. An example of an analytic hypothesis 1

2 Best Practices in School Psychology Table 8.1. The Six Steps of the Scientific Method and Problem Analysis Scientific Method Problem Analysis Example Observe and identify the Identify the problem Teacher observes that a student is struggling in reading problem of interest Develop and identify relevant hypotheses Design and select procedures to test relevant hypotheses Hypothesize likely causes and maintaining conditions (analytic hypotheses) Select methods for assessment such as reviews, interviews, observations, and tests General:. Poor instructional match. Low exposure to instruction. Poor curricular match. Need for more practice. Inaccurate critical skills. Low rate critical skills. Low motivation/incentive Specific:. Inaccurate performance oral reading performance. Low rate or automaticity on oral reading General:. Review educational records. Interview the student and teacher. Observe student performance within instructional conditions and interacting with the curriculum. Test with oral reading assessments to calculate the percent of words read correctly (accuracy) and the number of correct words per minute (rate); use vocabulary and comprehension measures as necessary. Test the student response to a targeted intervention aimed at the maintaining variable (e.g., incentivized performance, repeated practice, timed practice, instruction and curricular match) Collect data Collect the data Review, interview, observe, test Analyze and synthesize Review data data Devise a tentative conclusion Revise hypotheses regarding likely causes or form an hypothesized solution (intervention hypotheses) Synthesize the data to evaluate and address possible causes and maintain conditions so to inform intervention actions. Intervention: The student requires targeted practice of vowel and consonant diagraphs to establish accuracy and automaticity for work attached and word identification. Goal: The student identifies four words correctly in 1 minute (35% accuracy) when presented with a list of words with vowel and consonant diagraphs; if the intervention is effective, then the student will gain three words per minute per week with increased levels of accuracy until the student reaches 65 words read correctly in 1 minute (95% accuracy) is, The student cannot read because he or she struggles to decode words. The second type of hypothesis is an intervention hypothesis, which is developed during the final step of the scientific method. This hypothesis relates to the likely solutions for a problem. An example of an intervention hypothesis is, A phonics interview will teach the student how to match sounds to letters and subsequently improve reading performance. Intervention recommendations are defined as hypotheses because problem solutions are tentative until demonstrated as effective. Intervention hypotheses emerge from analytic hypotheses. A common error in problem analysis is omitting analytical hypothesis testing and immediately performing intervention hypothesis testing. This error frequently leads to failed intervention attempts, wasted resources, and frustrated staff. Analytic hypotheses function to formulate and test the supposed cause or causes of a problem. For example, there are common analytic hypotheses that should be tested when there are reading problems in a particular grade. These analytic hypotheses may address prior exposure to effective instruction, instructional match, and the achievement of relevant foundational skills. When formulating analytic hypotheses for academic 2 National Association of School Psychologists

3 Best Practices in Problem Analysis Table 8.2. Example Analytic and Intervention Hypotheses for Reading Analytic Hypotheses There is a poor instructional match (e.g., pacing, feedback) that contributes to insufficient growth. For example: the level of the material is too difficult. There is a poor curricular match that contributes to insufficient growth. For example: level of the material is too difficult. There are inaccurate critical skills (e.g., does not know letter sounds, diagraphs, or sight words) that contribute to insufficient growth. There is a low performance rate on critical skills that contributes to insufficient growth. For example: accurate and slow decoding and word identification. There is a low motivation or lack of incentives that contribute to insufficient growth. For example: the student can perform when provided sufficient incentives, which might be more interesting materials, activities, or tangibles. Intervention Hypotheses If provided instruction at a slower pace with increased feedback, then the student will demonstrate more rapid growth. If provided with curriculum materials that match or more closely approximate the student s skills, then the student will demonstrate more rapid growth. If provided with demonstration and slow deliberate practice, then the student will demonstrate more rapid growth toward performance at 95% or greater accuracy. If provided with incrementally faster paced practice with feedback, then the student will demonstrate more rapid growth toward automatic performance. If provided with an incentive to improve performance, then the student will demonstrate more rapid growth toward work completion and higher levels of performance on classroom assessments. problems, it may be helpful for school psychologists and educators to refer to a set of research- and evidence-based standards such as the Common Core State Standards (National Governors Association Center for Best Practices & Council of Chief State School Officers, 2010) for reading, mathematics, and other subjects. Finally, it might is also useful to reference hierarchies of skill development so to discern the likely sequence of skill acquisition and instruction. Such hierarchies can help to conceptualize a sequence of analytical hypotheses. Poor instruction might have caused the problem. Deficit skills that are inconsistent with the task demands for third-grade reading might maintain the problem. The process that begins with analytic hypotheses must progress to intervention hypotheses so that problem analysis concludes with useful recommendations. Some examples are presented in Table 8.2, which illustrate how analytic hypotheses (supposed causes) are associated with intervention hypotheses (possible solutions). It is more helpful to organize hypothesis statements into a hypothesis-testing framework, which can effectively guide the process. Figure 8.1 presents an example of a hypothesis-testing framework for reading, which incorporates some information about reading interventions from Scammaca, Vaughn, Roberts, Wanzek, and Torgesen (2007) and unpublished meetings with K. Bollman, M. C. Coolong-Chaffin, and D. Wagner (personal communication, March 21, 2013). This example is discussed in more detail below and is only intended to serve as an illustration. We strongly encourage researchers and practitioners to develop, evaluate, and refine their own hypothesis-testing frameworks. They have the potential to both refine the analytic process for professionals and document the process, which requires substantial expertise in the content and context of school-based problems. Levels of Inference An inference is a tentative conclusion or assumption that lacks explicit support from available data. Hypotheses are inferences by definition. It is critical that problem analysts make distinctions between what is known and what is inferred, or hypothesized. There are two types of approaches in making inferences: high inference and low inference. A high inference approach typically relies on theoretical, within-person constructs such as personality, psychopathology, or aptitude profiles. Such constructs are difficult to observe directly and they tend to rely on the assumption that outward behaviors are merely symptoms of an internal trait. Evidently, this approach requires a number of assumptions whose veracity is often unknown. A low inference approach relies on direct observation of explicit behaviors and testable hypotheses. In contrast to the high inference approach, this requires relatively few untested assumptions. Examples of low inference hypotheses may include a student s strengths and weaknesses as measured by a standardized test, intervention recommendations based on multisourced data, and the relevance of a published assessment to a school. These hypotheses are preferred over high inference hypotheses because they are based on testable assumptions. Thus, low-level inferences should take precedence and should be exhausted prior to the use of Data-Based and Collaborative Decision Making, Ch. 8 3

4 Best Practices in School Psychology Figure 8.1. An Example of a Problem Analysis Hypothesis Framework for Reading high-level inferences. Moreover, the most relevant analytic hypotheses and approaches will emphasize those variables that can be altered to have an impact in solving a problem. Low-level inferences and alterable hypotheses are most closely linked with successful problem solutions. 4 National Association of School Psychologists

5 Best Practices in Problem Analysis Alterable Causal and Maintaining Variables Problem analysis aims to identify variables that both cause and maintain problems. Causal variables are usually historical, such as insufficient nutrition, disadvantaged experiences, or lack of prior exposure to effective instruction. Maintaining variables persist in the present context to influence and sustain the problem. Although there are both biological and ecological variables that interact to cause and maintain problems, the goal of analysis is to promote problem solutions rather than ruminate about unalterable conditions. Problem analysis aims to examine causal and maintaining variables that can be manipulated by an intervention team. Variables such as instruction, classroom rules, and seating arrangements are considered to be alterable variables. In contrast, inalterable variables are irrelevant unless they interact with the alterable variables. For example, physical impairments such as blindness or deafness are inalterable. However, such impairments might help explain why previous instruction was ineffective and consequently guide future strategies, thus interacting with alterable variables. Alterable variables are usually in the ecology (e.g., class instruction and curriculum) and not within the learner. The learner s environment is composed of a large number of alterable variables that influence performance, such as seating arrangements and classroom rules. A problem analyst can hypothesize that the arrangement of the ecology will promote desirable outcomes and can monitor student s progress to test this hypothesis. Characteristics of Novice and Expert Analysts In school psychology, an effective and efficient problem analyst is expected to have context-specific expertise in instruction, curriculum, and ecological variables that interact with learner variables. A problem analyst needs content-specific expertise in reading, writing, mathematics, classroom behavior, and social development. Unfortunately, many school psychologists develop substantial expertise in test interpretation with modest knowledge of the context and content of education. For that reason, many school psychologists are novices at problem analysis because they lack domain specific expertise (i.e., context and content knowledge). Those with area-specific knowledge and experience are likely to employ a top-down approach to problem analysis. Experts use their content- and context-specific knowledge and experiences for problem analysis. They are more likely than novices to sample information about a problem strategically and search for familiar patterns, principles, and concepts associated with possible problem solutions. Familiarity with area-specific patterns and procedures enables experts to identify the most relevant variables, ignore trivial details, and generally combine complex patterns into meaningful chunks of information. For example, when a student is disruptive in the classroom, an expert behavior analyst is more likely to focus on the causal mechanisms and functions of the problem behavior. The expert may find that the behaviors function to access peer attention, access teacher attention, or escape task demands. When addressing a problem, the expert is likely to extract the most relevant information to analyze and represent a problem using a goal-directed, top-down approach. In contrast, a novice is likely to rely on a bottom-up approach for problem analysis because he or she has limited knowledge and experience. The novice often focuses on the extraneous or surface-level variables associated with a problem. It is more difficult for the novice to strategically identify and attend to the most relevant information. A novice behavior analyst might focus on the topography of a target behavior or irrelevant antecedents rather than on the function of the behavior. Despite these difficulties, novice school psychologists can solve problems effectively without having content- and context-specific knowledge. They can use systematic hypothesis-testing frameworks to solve a given problem. In the example of the disruptive student, a novice problem analyst can develop a functionally based set of analytic hypotheses to guide problem analysis. These hypotheses also can yield intervention hypotheses. For example, if problem analysis supported the hypothesis that disruption functioned to escape from task demands (e.g., difficult math work), then the likely intervention hypothesis might include a replacement behavior for escape (e.g., escape cards) with modified task demands (e.g., ensure instructional match). BEST PRACTICES IN PROBLEM ANALYSIS The prior section established that problem analysis by school psychologists incorporates the scientific method, low levels of inference, alterable variables, and expert knowledge of both the context and content. In practice, it takes time for school psychologists to develop these skills and an appreciation of their value. Therefore, best practices in problem analysis is facilitated by selected implementation of problem analysis with a multitiered Data-Based and Collaborative Decision Making, Ch. 8 5

6 Best Practices in School Psychology system of support. It is also facilitated by selective use of systematic hypothesis testing frameworks, which make the process explicit and enhances the potential for widescale implementation. Problem Solving and Multitiered Systems of Support A school psychologist must identify the problem, classify it, and analyze it at the systems, group, or individual level before selecting or implementing an intervention. That is, it is necessary to have information about a problem to understand why it occurs. It is a common error to select an evidenced-based or a well-marketed intervention. However, this well-marketed intervention might be improperly aligned with a local problem. Evidence-based interventions work for particular populations with specified deficits. No intervention works generically to solve all problems. It is problem analysis that helps align problem solutions with the problem at hand. Problem Analysis for the System, Group, and Individual A multitiered system of support is highly inefficient if all problems are analyzed at the individual student level. It is much more efficient to evaluate the prevalence of a particular problem to align the scope of the intervention with the scope of the problem. Therefore, problem solving and problem analysis occur at the systems level so that core curriculum, instruction, and support are assessed, evaluated, and refined. It is very common for school psychologists to observe reading performance, math performance, or behavior as a prevalent problem that is best addressed with systemic changes. Those systemic changes are guided by careful analysis. Regardless of the quality of systemic supports, some groups and individuals will require supplemental and intensive supports. Most problems in education are not individual student problems. Rather, they are typically more pervasive. Because problem analysis on the individual student level is the most inefficient use of resources, it is worthwhile to consider systemic and group analysis first and reserve individualized problem analysis for uncommon cases. The value of efficiency is consistent with the goals of working smarter, not harder; doing less, but doing it better; and doing it once, but for extended amounts of time. Problem analysis should enhance decisions about what is done, how it is done, for whom it is done, and for what reasons it is done. Ultimately, problem analysis is about an alignment of problem characteristics with problem solutions or interventions. Problem Analysis Precedes Standard Protocol It is common for school psychologists to employ a standard protocol approach as part of a multitiered system of supports. This standard protocol provides a default approach to intervention for common problems. This approach is relatively easy to implement because it uses the same intervention or treatment for all students who have similar problems in a particular area (Fuchs, Mock, Morgan, & Young, 2003). Standard protocols, however, do not address individual differences and do not emphasize problem analysis. To address this gap, school psychologists can use problem analysis to refine standard protocols so that they understand the type of problems and solutions before selecting a standard intervention program. For example, students with routine deficits in early reading are likely to benefit from an evidence-based phonics intervention. Initially, that intervention must be selected or developed with problem-specific knowledge that emerges through problem analysis. After implementation and evaluation, a high quality solution is readily available in the education system. That intervention becomes standard protocol for use in the future. Thereafter, problem identification and problem analysis function to match identified problems efficiently with standardized interventions. Theoretical Orientations Guide Systematic Hypothesis-Testing Frameworks A variety of theoretical orientations can support hypothesis testing and support problem analysis. For instance, developmental theories provide information about typical development among children and can inform what kind of instruction and interventions are needed to promote learning and growth in students who are developing differently from these trajectories. Other theoretical orientations can inform interventions specifically for academic skill deficits. The instructional hierarchy, for example, provides a framework that postulates that skills progress in four stages: acquisition, fluency, generalization, and adaptation (Daly, Lentz, & Boyer, 1996; Haring & Eaton, 1978). An applied behavioral analysis orientation can be useful to develop hypotheses about which environmental events predict and maintain a target behavior, especially when a student has a performance deficit. It is also useful to guide task analysis and the construction of skills hierarchies. 6 National Association of School Psychologists

7 Best Practices in Problem Analysis Curriculum-based assessment and curriculum-based evaluation can inform problem analysis. Curriculumbased assessment is particularly relevant to problem analysis because it is used to assess students within the context of their curriculum content and learning needs (Hintze, Christ, & Methe, 2006). Curriculum-based evaluation provides an intricate hypothesis-testing framework across both academic and social-skill domains such as reading, mathematics, written expression, language, social skills, and task-related behavior. The curriculum-based evaluation hypothesis-testing framework usually is presented in a series of flowcharts with corresponding decision-making rules and assumes that performance discrepancies exist because prior knowledge or skills have yet to be established. For more details about curriculum-based evaluation, see Hosp, Hosp, Howell, and Allison (2014). Understanding the limitations of applying certain theories is important to effective problem analysis. For instance, hypotheses related to general intelligence are usually limited to diagnosing developmental disabilities, but the diagnoses are not necessarily useful in selecting instruction and interventions. The traditional framework of using test-based trait and aptitude profiles relies on high inference approaches and distal measurements and thus is not very consistent with effective problem analysis. Development and Use of Systematic Hypothesis-Testing Frameworks School psychologists are expected to use context- and content-specific expertise to solve academic and social behavior problems. Unfortunately, many school psychologists lack training and expertise across the many context and content domains within educational systems. The breadth and depth of academic, behavioral, and socioemotional problems that school psychologists address are substantial. Because it is not always possible to be independent experts for all the problems that might arise in schools, practitioners can develop and use content-specific systematic hypothesis-testing frameworks to address problems. Such frameworks can provide explicit guidance, particularly when a school psychologist lacks expertise in a specific content area. In problem analysis, an effective framework should (a) derive from evidence-based practices; (b) be embedded within the scientific method; (c) emphasize low inference methods of assessment and evaluation; (d) evaluate alterable causal and maintaining variables; and (e) integrate expert approaches of analysis, which embed navigational aids so that the sequence and targets of evaluation are optimized. Effective problem analysis seeks to establish knowledge in assessment and evaluation to optimize contentspecific problem analysis. Identifying and developing explicit hypothesis-testing frameworks will contribute to establish the school psychologist as a problem analyst. There are some early examples that might inform future development, including curriculum-based evaluation and curriculum-based assessment for instructional design and applied behavior analysis. Each provides some systematic approach to assessment and evaluation to declare relevant analytic hypotheses. When a school psychologist lacks expertise, hypothesis-testing frameworks provide scaffolds and a system of support to guide analysis. Likewise, hypothesis-testing frameworks guide the identification of problem solutions. For example, deficits in a critical foundational skill area might disable learning and, thereby, establish and maintain learning problems. Examples of Systematic Hypothesis-Testing Frameworks There are two examples of systematic hypothesis-testing frameworks. The first was presented in Figure 8.1 and it uses oral reading assessments to illustrate a simple and familiar hypothesis-testing framework. This example begins with an oral reading assessment, such as curriculum-based measurement of oral reading (CBM- R). This is a useful starting point because many schools use CBM-R for universal screening, which is useful to identify reading problems. It is at that point that problem analysis begins with the development and consideration of relevant analytic hypotheses related to causal and maintaining variables. Some examples are listed in Figure 8.1. Data subsequently are collected from interviews, observations, tests, and reviews of extant data and evaluated to inform the use of analytic hypotheses. The figure illustrates tests of skills-based oral reading deficits. The bottom left section of that figure is intended to illustrate how the analytic hypotheses and intervention hypotheses/recommendations generally correspond, such that analytics inform interventions. The bottom right section of that figure illustrates the way in which ongoing progress monitoring functions to continually test and update the intervention hypothesis. Although this particular example lends itself to problem analysis of individual students or small groups, problem analysis often will apply to larger groups of students. Data-Based and Collaborative Decision Making, Ch. 8 7

8 Best Practices in School Psychology Figure 8.2. Problem Analysis at the System, Group, and Individual Level Figure 8.2 represents problem analysis for a multitiered system of supports. Neglecting a systems-level or group-level problem can overwhelm both problem solving and the system of supports. Problem analysis is often resource intensive and cannot be sustained if an excessive number of problems are analyzed at the individual level. It is when specific groups and learners have distinct needs relative to the local population that resources are allocated to analyze those problems. If data support the first hypothesis in Figure 8.2, then analysis will progress to test this hypothesis: The magnitude of the problem and need for services are greater for some learners. This hypothesis is designed to provide support for groups and individuals with substantial needs. The needs of the system are established by the discrepancy between the performance of the population and some external criterion such as proficiency on a large-scale assessment in reading. The need for interventions at the group or individual level is established by the discrepancy between a learner s performance and that of the local norms for the school, district, or classroom population. Those students who are discrepant from local standards are at even greater risk for academic failure. This is consistent with the model for multilevel systems of support. Once the scope of the problem within the population is defined at the system, group, and individual level, then the school psychologist, or problem analyst, may begin to develop hypothesis statements. Because poorly defined expectations are often the root of school-based behavior problems, the first hypothesis statement a problem analyst would ask is, Do students know what is expected of them? If the students clearly are aware of expectations, then the problem analyst can progress to evaluate instructional match or ecological contingencies. If expectations were not taught and are not clearly established, then the problem analyst might recommend a remediation strategy. Expectations can be taught through explicit instruction, modeling, practice, or direct feedback. The implicit hypothesis is: If expectations are taught, then the magnitude of the problem will be reduced. This is an example of an if/then hypothesis. An if/then hypothesis establishes a tentative belief that a particular manipulation in the ecology will result in a predicted change in learner behavior. The hypothesistesting framework is directly linked to interventions because the framework terminates with a testable 8 National Association of School Psychologists

9 Best Practices in Problem Analysis hypothesis for intervention, the intervention hypothesis. The intervention itself is a hypothesis because it is a putative and testable solution to the problem. It is best practices to recognize and define recommendations for instruction and intervention as hypothetic statements. The effect of an intervention is unknown until after data are collected and evaluated. Assessment for Problem Analysis Assessment is a basic competency for school psychologists (NASP, 2010). Appropriate assessment data are collected using a multimethod, multidomain, and multisource approach. Figure 8.3 features a matrix that illustrates this approach. Reviews, interviews, observations, and tests serve as different assessment methods, while the instruction, curriculum, environment, and the learner represent the multiple domains (Heartland Area Education Agency 11, 2006). It is necessary to note that problem analysis does not require that school psychologists use an exhaustive set of assessments across all methods, domains, and sources of information. Instead, assessment should generate the necessary information for answering a set of wellspecified questions. The school psychologist should readily be able to respond to the question, How will the assessment procedure answer the assessment question? For instance, if mathematical calculation skills are a concern, then a school psychologist may ask the following two questions: (a) Is this a skill deficit, a generalization problem, or a performance deficit? (b) If this is a skill deficit, then which specific skills are Figure 8.3. Assessment: Multidomain, Multimethod, Multisource Matrix Data-Based and Collaborative Decision Making, Ch. 8 9

10 Best Practices in School Psychology affected? The intensity and thoroughness of assessment will be determined, in part, by the severity and characteristics of the problem. For instance, those influences maintaining minor problems may be relatively easy to identify. Consequently, assessment and evaluation for problem analysis might be brief and relatively few cells in the matrix would be completed. In contrast, the influences maintaining severe problems or problems that are resistant to standard protocol solutions may be difficult to identify. In those cases, assessment for problem analysis would be more extensive and more cells in the matrix would be completed. More severe problems that establish high levels of risk or are resistant to intervention require a more extensive dataset to support problem analysis. SUMMARY School psychologists establish a unique and necessary niche when they engage in effective problem analysis. The field will thrive and school psychologists will become invaluable to multitiered systems of supports, which depend on effective and efficient problem solving and problem analysis. The dynamic process of problem analysis is the essential link between assessment and intervention. In this sense, problem analysts function as applied scientists in that they seek to discover the relationship between independent variables (instruction, curriculum, environment) and dependent variables (skills and behavior of the learner). The goal is to modify the conditions of the independent variables to have an impact on the state of the learner. Problem analysis progresses from (a) a well-identified problem to (b) possible causes to (c) possible solutions to (d) a validated solution. As discussed, successful problem analysis depends on knowledge and appreciation of science as both a method and resulting body of evidence. Problem analysis progresses through steps similar to the scientific method, which includes problem identification, hypothesis development, hypothesis testing, and the generation of tentative conclusions in the form of intervention recommendations. Problem analysis also depends on the body of evidence that emerges from science. Published research and evidence-based practices is relied on to inform analytic hypothesis and intervention hypothesis development. Problem analysis is most effective when low inference methods of assessment and interpretation that focus on the causal and maintaining variables are relied on. There are many theories and perspectives on student development. Those that are most relevant to problem analysis are immediately testable through observation of skills and behavior in the school-based setting. School psychologists and other educators often have expertise in one or more content areas, but rarely have broad and deep expertise in all domains. As discussed, the ability to identify and organize the most relevant information is an important aspect to problem analysis. Together, and for those reasons, we recommend the ongoing development and evaluation of content-specific hypothesistesting frameworks. Although there are a few simple examples presented in this chapter, researchers and content experts must develop, evaluate, and refine frameworks that are content and context specific. These will assist the nonexpert problem analysts as they pursue intervention recommendations through problem analysis. In practice, problem analysis is fundamental to multitiered systems of support. It helps to understand the causal and maintaining features of problems before solutions are proposed. As discussed, it also helps to understand the prevalence of problems. This understanding helps to determine whether the intervention should target the system, core supports, identifiable groups, or individuals. Multitiered systems of support are rarely sustainable if each problem is resolved at the individual student level. Many problems are common to systems and can be either prevented or resolved more efficiently at the group level. REFERENCES Daly, E. J., Lentz, F. E., & Boyer, J. (1996). The instructional hierarchy: A conceptual model of understanding the effective components of reading interventions. School Psychology Quarterly, 11, Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness-to-intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18, Haring, N. G., & Eaton, M. D. (1978). Systematic instructional procedures: An instructional hierarchy. In N. G. Haring, T. C. Lovitt, M. D. Eaton, & C. L. Hansen (Eds.), The fourth R: Research in the classroom. (pp ). Columbus, OH: Merrill. Heartland Area Education Agency 11. (2006). Program manual for special education. Johnston, IA: Author. Retrieved from iowaideainfo.org/vimages/shared/vnews/stories/4a8b fd/ Special%20Education%20Procedures%20Manual%20January% 2015%202013%20final.pdf Hintze, J. M., Christ, T. J., & Methe, S. A. (2006). Curriculum-based assessment. Psychology in the Schools, 43, Hosp, J., Hosp, M., Howell, K., & Allison, R. (2014). The ABCs of curriculum-based evaluation: A practical guide to effective decision making. New York, NY: Guilford Press. 10 National Association of School Psychologists Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:52

11 Best Practices in Problem Analysis National Association of School Psychologists. (2010). Model for Comprehensive and Integrated School Psychological Services. Retrieved from nasponline.org/standards/2010standards/2_practicemodel.pdf National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common Core State Standards for English language arts and literacy in history/social studies, science, and technical subjects. Washington, DC: Author. Scammaca, N., Vaughn, S., Roberts, G., Wanzek, J., & Torgesen, J. K. (2007). Extensive reading interventions in grades K 3: From research to practice. Portsmouth, NH: RMC Research. Data-Based and Collaborative Decision Making, Ch Best Practices in School Psychology B1Ch8_W136_Christ.3d 10/12/13 09:54:52