HOW ACOUSTICS IN CALIFORNIA HIGH PERFORMANCE SCHOOLS RELATE TO STUDENT ACHIEVEMENT

Transcription

1 HOW ACOUSTICS IN CALIFORNIA HIGH PERFORMANCE SCHOOLS RELATE TO STUDENT ACHIEVEMENT Devin Wong Veneklasen Associates, th Street, Santa Monica, CA 90404, United States, Joonhee Lee and Lily M. Wang Durham School of Architectural Engineering and Construction, University of Nebraska-Lincoln, 1110 S. 67th Street, Omaha, NE , United States This project seeks to determine if students at K-12 schools that have received High Performance Incentive (HPI) grants in the state of California in the United States do evidence higher scholastic achievement, based on publicly available results on state-wide standardized tests. Since its inception in 2006, the HPI grant program has provided supplemental funds for school districts to build school buildings with high performance attributes. The attributes are assessed by a High Performance Rating Criteria (HPRC), modeled after criteria set forth by the California Collaborative for High Performance Schools (CHPS). The criteria for indoor environmental quality include assessments of the building s thermal, air quality, lighting and acoustic conditions. In this project, data on these conditions as submitted by schools receiving HPI grants, particularly related to acoustic conditions, will be analyzed against that school s Academic Performance Index (API). The API is based on student achievement results from the California Standardized Testing and Results Program (STAR), which includes both English/Language Arts and Math topic areas. Specifically, this paper will report results of statistical analyses to determine if there are significant differences on API and STAR student achievement results between HPI and non-hpi schools with similar demographics. 1. Introduction The economic and environmental impact of high performance (green) schools has been the subject of numerous studies of the last 20 years. However, research regarding the impact of green schools on the performance of students is not extensive 1. Results from previous studies do indicate better student achievement with better-built environments, but tend to rely on conclusions drawn from subjective surveys and indirect measures 2, or focus on single indoor environmental factors such as acoustics 3. Few studies answer the question of how beneficial green school builds are as compared to schools with traditionally adequate built conditions. The goal of this paper is to determine the comprehensive impact of a high performance indoor environment on student achievement, as measured by standardized test results. A subset of the 200+ schools applying for High Performance Initiative (HPI) Grants in the state of California over the ICSV22, Florence (Italy) July

2 past decade has been correlated with an existing student achievement score database for all public schools in the state. Average student achievement scores and their annual changes are compared between schools that received state-funded HPI grants to build high performance school buildings and those that did not. A second test investigates how schools that received HPI grants performed before and after high performance modifications. A further comparison of the acoustical indoor environmental design conditions, as submitted for all schools receiving HPI grants, against student achievement outcomes is also intended. 2. Previous Research While it is widely accepted that the quality of the built environment has an effect on occupant health and productivity, the complexity of interactions between people and environments means that it is difficult to establish direct cause-and-effect relationships between specific building attributes and human outcomes. Research summaries produced by the McGraw-Hill Research Foundation and National Clearinghouse for Educational Facilities list indoor air quality (IAQ), acoustics, lighting, and thermal comfort, among others, as specific qualities that are of particular importance in schools 4. This study will focus on the cumulative effects of these qualities, and only the acoustics attribute on its own. 2.1 Acoustics Acoustics in classrooms is most often evaluated by two factors: background noise level (BNL) and reverberation time (RT). A high BNL can affect students ability to hear, absorb and retain information. Similarly, a high RT hinders student s ability to understand speech. Poor acoustic conditions in classrooms have been found to result in reduced speech intelligibility 5. Studies focusing on the direct impact of acoustic parameters on student achievement have found that internal and external environmental noise has a direct impact on student achievement 6, while higher BNL s impact student performance on listening and reading comprehension 7, and language areas 3. No significant conclusions have been drawn about reverberation time. 2.2 High Performance Schools and Overall Variable Interactions Very few studies discuss critical interactions that occur between building attributes, for instance, the relationship between increased daylighting and thermal comfort, or the effect of humidity on IAQ. The relative impact of these variables for the purpose of prioritization is not readily available. Additionally, research on the relative overall benefit of high performance school buildings as compared to those with traditionally adequate buildings is lacking. A 2011 study found that when construction projects were undertaken by a school district in Connecticut, test scores across all schools increased noticeably afterward 8. Similarly, a case study of green and energy-retrofitted Toronto schools was found to have lower absenteeism rates and higher student performance when compared with conventional schools. The results were not statistically significant, however, and could not be generalized 9. A study investigating the cost effectiveness of green schools and conventional schools, concluded that green schools offered a slight increase in learning ability, and a decrease in asthma, cold and flu occurrences 10. The cost-benefit analysis of green school construction is of particular importance because of the high initial cost associated with these endeavours. School districts with limited funds may be hesitant to pursue high performance attributes when there is a lack of statistical evidence supporting green building benefits. ICSV22, Florence, Italy, July

3 3. High Performance Incentive Grant: Criteria for High Performance Buildings Definition of what constitutes a green school is important in order to delineate between those schools that qualify as high performing and those that are considered traditionally adequate. There are many programs that certify levels of sustainable design, and qualification in these programs is not mutually exclusive with qualification for a HPI grant. The High Performance Rating Criteria (HPRC) was chosen for this study because it focuses on the construction of school projects in particular, and is publicly accessible by request from the California Division of the State Architect (DSA). 3.1 History In 2007, Proposition 1D, California Assembly Bill 127 was approved, providing $100 million in supplemental incentive grants to promote the use of high performance attributes in new construction and modernization projects for K-12 schools 11. In particular, the HPI Grant program focuses on 5 attributes: site, water, energy, materials, and indoor environmental quality. According to the bill, high performance attributes include using designs and materials that promote energy and water efficiency, utilize recycled materials and those that emit a minimum of toxic substances, and emphasize natural lighting, indoor air quality, and acoustics conducive to the process of teaching and learning 11. To ascertain the level of high performance attributes in each project, the HPRC was created. The HPRC is assessed in the form of a rating scorecard: each of the 5 attributes emphasized by the program is assigned its own section and subsection with more detailed criteria. Each subcategory is designated number of points that may be earned upon completion. To be HPI approved, each new application must meet all the prerequisite requirements in all HPRC categories. Then, other credits may also be pursued. A minimum of 27 HPRC points is required to qualify for a new project grant. Additions and modernization projects must only meet the prerequisites for the categories within the scope of construction; then, other credits are pursued to reach a minimum of 20 HPRC points to qualify for an addition or modernization grant. For the purposes of this study, it is assumed that schools that have not applied and received an HPI grant (referenced as Normal ) do not meet the same construction standard as ones that have received HPI grants (referenced as High Performance ) Acoustics A portion of this study will focus on the Acoustic subcategory of Indoor Environmental Quality. The intention is to concentrate on one of the factors that has been shown to have a direct impact on school occupants in previous research. The HPRC subcategory for acoustics is based upon the American National Standard Acoustical Performance Criteria, or ANSI S In the 2002 version, projects applying for HPI certification must have a maximum (unoccupied) noise level of 45 dba, with a maximum (unoccupied) reverberation time of 0.6 seconds. Extra points are available if the maximum (unoccupied) noise level is reduced to 40 dba (1 point) or 35 dba (2 points). In the 2006 version the number of HPI points awarded for improved acoustical performance was increased from two to three. 4. Academic Performance Index: Criteria for Student Achievement Standardized testing is the principle measure of learning outcomes in the United States today. The California Academic Performance Index (API) is a publicly available measure for evaluating student achievement on a school level, and may be further sorted by socioeconomic factors. The ICSV22, Florence, Italy, July

4 combination of student outcomes with specific, school-related, variables make the API an excellent factor for evaluating student achievement in tandem with the HPRC. The API is a comprehensive accountability system that monitors the achievement of all California public schools that serve students in kindergarten through grade twelve 12. The API is based on an improvement model: the assessment results from one year are compared to assessment results from the prior year to measure improvement. Results are summarized from student achievement scores on the STAR Program and California High School Exit Examination (CAHSEE). STAR administers grade-specific tests in English/Language Arts, Math, History, Science and Writing topic areas. The intent of the API is to compare school achievement results from one year to the next, rather than track individual student progress. Schools that meet state participation and growth criteria may be eligible for awards or financial funding. Schools that do not meet growth targets may be identified for state intervention programs to improve performance. 4.1 Scoring The API score is a single number, ranging from a low of 200 to a high of 1000, calculated by converting students performance on state-wide assessments into points on the API scale. The formula accounts for the number of valid student scores weighted by both performance level and subject. California has set an API score of 800 as the target score for all schools to meet. Schools that do not meet the target score are required to meet annual growth targets until that goal is achieved. The growth target is generally calculated as five percent of the difference between the school s API and the state target of 800. Schools that meet or exceed the target score are expected to annually maintain or improve a score above the target. The annual API reporting cycle includes a Base and a Growth API. The Base API begins the reporting cycle and is calculated using assessment results of the previous year. The Growth API is calculated using the same indicators as the Base API, but uses student achievement scores from the current year. For example, 2011 Base API will be released in spring 2011 using results from spring 2010 testing. The 2011 Growth API will be released in fall 2011, using results from spring The academic improvement, released in the Growth API Report, is the delta between the 2011 Base API and 2011 Growth API. Over the history of the API, different API indicators have been used, as standardized tests are modernized or subject weighting in the API calculation changes. Therefore, only single API scores in the same base and growth year are compared. To compare student achievement across multiple years and different reporting cycles, this study will use the difference in API scores from the same year (Difference API) and average Base and Growth API over time. 4.2 School Demographics and Similar Schools Lists Demographic information pertaining to each school is also included in each API report. In addition to the school type and grade range, the number of students by race, socioeconomic status, English proficiency and disability are also reported. Based on these and other demographic descriptors, California has created a Schools Characteristic Index (SCI). The purpose of the SCI is to group schools that face similar educational opportunities and challenges. Every year, each school s API performance is compared with its List of 100 Similar Schools chosen for their similarity in SCI. 5. Methodology The purpose of this study is to evaluate the comprehensive impact of high performance buildings on student achievement. Using the California HPRC to evaluate building performance and California API scores to evaluate student achievement, three statistical analyses will be conducted. The first analysis (Test 1) will compare API data of those schools that are considered high performance ICSV22, Florence, Italy, July

5 versus those that are not. The second analysis (Test 2) analyses the effect of high performance attributes before and after completion of construction. The final analysis (Test 3) will focus on the specific effect of the Acoustics subcategory in the HPRC on overall API scores. 5.1 Data Collection Sample Set A total of 248 projects were submitted and approved for the HPI grant between 2007 and Only fully completed projects that include new construction, modernization or addition to a classroom building in the scope are used in Test 1 analysis. The total number of schools meeting this required criteria is 78. A corresponding set of 78 schools meeting traditionally adequate conditions was selected using the yearly lists of 100 similar schools. Lists from the years 2005 and 2013 were analysed and ranked based upon the number of student participants in Free or Reduced-Price Lunch programs. The school whose two-year average of participating students most closely matched that of the comparative HPI school was then selected. A check was performed to ensure that the chosen similar school had not applied for an HPI Grant. Finally, the 78 similar non-hpi schools were added to the existing sample for a total of 156 schools. The total sample is also used to define schools in Test 3 analysis. Of the 78 High Performance schools, 14 were characterized as having completed high performance construction (modifications or additions) before Fall These were used in Test 2 analysis. Fall 2010 was selected as the cut-off date to ensure that at least two years of API data would be available for the condition after construction was completed. This study will refer to the time period before high performance construction was completed as Pre-Mod, and the time period after construction was completed as Post-Mod API Data API scores for the years are publicly available for download from the California Department of Education (CDE) website. Available API data for each school in the total sample was obtained for all years the school was active. For each year, the Difference API was calculated. API data was not available for each sample school over every year of existence due to irregularities in testing, as determined by the CDE. Instances of missing API data were excluded from analysis on a case by case basis. For Test 2 analysis, the Base API, Growth API and Difference API for the sample set of 14 schools are divided into Pre-Mod and Post-Mod categories. The Pre-Mod condition is defined as the average of all API scores that include the school years ending before Fall The Post-Mod condition for each school is defined as the average of API scores for all school years beginning after Fall Acoustics Data The acoustics score was obtained from the HPRC scorecard given in each application. For this study, each school was given a score on the scale of 1-3, where a score of 1 indicated the minimum HPRC prerequisite was met with a background noise level of 45 dba and maximum reverberation time of 6 seconds. Schools that indicated a background noise level of 40 dba received a score of 2, while schools that marked a BNL of 35 received a score of Statistical Analysis The relationship between high performance rating and academic achievement was analysed using Microsoft Excel and SPSS. The inclusion of High Performance construction attributes (High Performance or Normal), acoustics rating (1-3), and time (Pre-Mod and Post-Mod) are considered the independent variables. The Difference API is considered the dependent variable. Non- ICSV22, Florence, Italy, July

6 parametric tests are required for some cases; however, only preliminary parametric tests will be presented in this paper using statistical methods for correcting violations of bias. Preliminary analysis of Test 1 was performed using the independent-measures t-test, which compares two experimental conditions where different participants are assigned to each condition 13. Analysis was performed on the Base API, Growth API, and Difference API scores for High Performance and Normal conditions from the school year. Preliminary analysis of Test 2 was performed using a paired-sample t-test, which is used to compare two experimental conditions in which the same participants take part in both conditions. Analysis was performed on the average Base API, Growth API, and Difference API scores for Pre-Mod and Post-Mod conditions. All significant relationships are reported using these two statistical methods. Analysis of Test 3 is ongoing. Both the independent-measure and repeated-measure t-tests report the mean (M), standard error (SE), and degrees of freedom (df) of each condition, as well as the mean difference between the two conditions and its significance. Standard error of the mean (SE) is a standard deviation of the sample means and is used to represent how accurate a sample can be. As the SE increases, the variability of the sample mean increases. 6. Results and Discussion 6.1 Test 1 Results For Test 1, Base, Growth, and Difference API scores for the year were analysed for 156 schools (78 High Performance, 78 Normal). For the Normal category, there were 70, 77, and 69 valid cases for Base, Growth, and Difference API respectively. For the High Performance category, there were 71, 77, and 71 valid cases for the Base, Growth, and Difference API respectively. Normal schools had a higher average Base API (M=742, 95%CI [723, 761], SD = 78.3) than High Performance schools (M=720, 95%CI [698, 742], SD = 92.5). Similarly, Normal schools had a higher average Growth API (M=747, 95%CI [730, 763], SD = 73.1) than High Performance schools (M=719, 95%CI [697, 742], SD = 98.3). However, Normal schools had a lower average Difference API (M=-.073, 95%CI [-4.65, 4.50], SD = 19.0) than High Performance schools (M=3.35, 95%CI [-2.86, 9.56], SD = 26.2). In all cases, though, the t-test indicates that the differences are not statistically significant and represent small effect sizes. Figure 1 plots show the average Base, Growth and Difference API for Normal (x = 0) and High Performance (x = 1) schools. High Performance schools tend to exhibit lower Base and Growth API. This is reasonable, since the API is based upon a growth model (schools are expected to improve annually). Therefore, schools within the same socioeconomic level that have been in operation longer should exhibit higher scores. As part of a relatively recent program, High Performance schools should average lower total years of operation across the sample. Further analysis should be conducted. High Performance schools do exhibit a higher Difference API than Normal schools. While Normal schools did not improve during the school year (M=0.06), High Performance schools showed an improvement of almost 3 API points (M=3.40). That the difference is not significant is also not surprising when accounting for the multitude of factors that may account for overall API scores. The positive relationship between High Performance school conditions and API scores is worth studying further, however. ICSV22, Florence, Italy, July

7 Figure 1. Mean Base, Growth and Difference API for High Performance and Normal Schools. Error Bars represent the standard error of the mean. 6.2 Test 2 Results In Test 2, Base, Growth, and Difference API scores for 14 High Performance schools were analysed over time (Pre-Mod and Post-Mod). All 14 cases were valid over all categories. The average Base API score for Pre-Mod was 679, with scores ranging from 614 to 816. The average Growth API for Pre-Mod was 685, with scores ranging from 622 to 819. The average Difference API for Pre-Mod was 15.5, with scores ranging from 3.00 to The average Base API for Post-Mod was 760, with scores ranging from 698 to 863. The average Growth API for Post-Mod was 758, with scores ranging from 702 to 873. The average Difference API for Post-Mod was -1.00, with scores ranging from to In all cases, the t-test indicates that the differences are statistically significant and represent large effect sizes. Figure 2 plots show the average Base, Growth and Difference API for Pre-Mod (x = 0) and Post-Mod (x = 1) schools. A significant difference between the means is consequently found for Base and Growth API for the same schools across time (Pre-Modification and Post-Modification). Pre-Mod conditions show a lower Base and Growth API than Post-Mod. This may also be explained by the API growth model. Schools are encouraged to seek yearly improvement in their API performance; API for individual schools should tend to increase over time regardless of High Performance construction or not. A significant difference between the means was found for Difference API across time as well. Pre-Mod conditions in the same schools averaged an improvement in Difference API of almost 15 points (M=15.5 compared to M=1.00 for Post-Mod). While significant, this result may be coloured by other factors such as adjustments made to the API calculation. In general, changes to standardized testing result in lower API scores for the following school year. Further analysis should be conducted utilizing different definitions of Pre-Mod and Post-Mod categories to corroborate this result. Figure 2. Mean Base, Growth and Difference API for Pre-Mod and Post-Mod Schools. Error Bars represent the standard error of the mean. ICSV22, Florence, Italy, July

8 7. Conclusions Statistical analyses have been performed on a sample of 156 California Schools. Test 1 examined the effects of High Performance built conditions on API scores for one year. Test 2 examined the effects of High Performance built conditions on API scores across time on a subset of 14 schools, before and after construction of high performance attributes was completed. For Test 1, no significant difference between the means was found for API scores across High Performance built conditions. High Performance schools do tend to exhibit a higher Difference API than Normal conditions. Further research should focus on the performance of High Performance and Normal schools across multiple years. For Test 2 significant differences between the means were found for API scores across time conditions. Schools post-high performance modification exhibited a lower Difference API than schools Pre-Mod. This may be explained by changes to the API calculation method and standardized testing. Further research should focus on a repeated measures analysis across multiple years. A Test 3 is being conducted, focused on the correlation between acoustics scores as rated by the HPRC and High Performance conditions using multivariate statistical methods. REFERENCES 1 National Research council. (2007). Green Schools: Attributes for Health and Learning: National Academies Press [Online.] 2 Baker, L., & Bernstein, H. The Impact of School Buildings on Student and Health and Performance: A Call for Research. [Online.] available 3 Ronsse, L., Investigations of the Relationships between Unoccupied Classroom Acoustical Conditions and Elementary Student Achievement, Doctor of Philosophy Dissertation, Graduate Program in Architectural Engineering, University of Nebraska, (2011). 4 Schneider, M. Do School Facilities Affect Academic Outcomes? Washington D.C.: National Clearinghouse for Educational Facilities. (2002). 5 Nelson, P. B., Soli, S. D., & Seltz, A. (2002). Classroom acoustics II: Acoustical barriers to learning. Melville, NY: Acoustical Society of America. 6 Shield, B.M., Dockrell, J.E. The effects of environmental and classroom noise on the academic attainments of primary school children. The Journal of the Acoustical Society of America, (2008). 7 Klatte, M., Lachmann, T., Meis, M. Effects of noise and reverberation on speech perception and listening comprehension of children and adults in a classroom-like setting, Noise & Health, October-December, , (2010). 8 Nielson, C., Zimmerman, S. The Effect of School Construction on Test Scores, School Enrollment, and Home Prices. Institute for the Study of Labor (IZA) Discussion Paper No November Issa, M.H., Rankin, J., Attalla, M., Christian, A. Absenteeism, Performance and Occupant Satisfaction with the Indoor Environment of Green Toronto Schools, Indoor and Built Environment, 20;5, , (2011). 10 Kats, G. Greening America s Schools: Costs and Benefits. Washington, DC: Capital E. (2006.) [Online.] available 11 Office of Public School Construction., HPI Brochure: High Performance Incentive Grant Promoting the use of high performance attributes in California s schools [Online.] 12 California Department of Education. Executive Summary Explaining the Academic Performance Index. [Online.] available 13 Field, A. Discovering Statistics Using IBM SPSS Statistics 4 th Edition, Sage Publications Ltd., Thousand Oaks, CA, 876 (2013). ICSV22, Florence, Italy, July