Commercial Market Share Tracking and Commercial Saturation Survey Research Plan Draft

Transcription

1 Commercial Market Share Tracking and Commercial Saturation Survey Research Plan Draft Prepared for California Public Utilities Commission Itron, Inc El Camino Real San Diego, California (858) February 7, 2012

2 Table of Contents 1 Introduction Overview Description of the Studies Goals and Objectives Research Objectives Research Plan Evolution over Time Evaluator Contact Information Research End Uses, Measures, and Segments CMST Measures and Segments High Priority Measures CMST Business Segments CSS End Uses, Measures, and Segments CSS End Uses and Measures CSS Business Segments Data Sources Key Input Data Sources Nonresidential Site Development Building Type Classification Program Tracking Data Key Input Data Integration and Output Data Description CMST Supply-Side Data Sample Design Develop the Sampling Frame Stratification Sample Design for the Telephone Survey Sample Design for the CMST On-Site End-User Verification Sample Design for the CSS On-Site Surveys CMST Supply-Side Survey Survey Development and Implementation Telephone Survey Development Telephone Survey Coordination Telephone Survey Content Telephone Survey Implementation On-Site Survey Development On-Site Survey Coordination CMST On-Site Survey Content CSS On-Site Survey Content On-Site Survey Implementation Develop On-Site Protocols Develop On-Site Training Information Itron, Inc. i Table of Contents

3 Conduct Training for Surveyors Research Methodology Telephone Survey Telephone Results On-Site Surveys On-Site Results Supply-Side Actor Surveys Timeline Work Plan and Budget Task 1: Project Management Task 2: Develop and Update the Research Plan Task 3: Data Development: Site Development, Business Categories, and Final Meter Match-Up Site Development Building Type Categories Final Meter Matchup Task 4: Sample Design Task 5: Telephone Survey Development and Implementation Task 5.1 Survey Development Task 5.2 Survey Implementation Task 5.3: Telephone Survey Analysis Task 6: On-Site Survey Development Task 6.1. Development the CMST and CSS Survey Instruments Task 6.2. On-Site Survey Training and Testing Task 6.3. CMST On-Site Survey Implementation Task 6.4. CSS On-Site Survey Implementation Task 6.5. On-Site Survey Expenses Task 7: Develop the Survey Database Task 8: CMST Supply-Side Actor Survey Task 9: Analysis and Reporting Task 10: Web Presentation Tool Budget Itron, Inc. ii Table of Contents

4 1 Introduction This document presents the Research Plan for the Commercial Market Share Tracking Study (CMST) and the Commercial Saturation Survey (CSS). The Research Plan includes a description of the studies, the goals and objectives of the research studies, the data to be collected and how these data will be evaluated, the sources of the information to be used, the approaches for sampling, the methods to be employed to analyze the data collected and how coordination and communication will occur with the other related evaluation and research studies. 1.1 Overview Description of the Studies The CMST and CSS research projects are designed to collect the data necessary to describe current baseline purchases of select high priority equipment and the current baseline saturation of measures in businesses in the commercial population. These data will help describe both the current market activity for select measures and the existing stock of commercial measures and characteristics of commercial buildings. The CMST study will collect information on recent purchases of high priority measures. The high priority measures chosen for evaluation as part of the CMST study include linear fluorescents, televisions, and packaged HVAC. The study will collect information on recent purchases of both standard and high efficiency measures. Collecting information on both standard and high efficiency purchases will lead to the development of an estimate of total market purchases and to the distribution of these purchases between base and high efficiency measures. The study will also use information from the IOU energy efficiency program tracking databases to help determine the share of high efficiency purchases receiving an IOU rebate. The CSS study will collect baseline information that will help program managers and the CPUC better understand the characteristics of current commercial energy consuming measures and commercial buildings. The study will collect an extensive list of information on commercial businesses, including but not limited to commercial firm demographics, the saturation, age, condition, and efficiency level of commercial energy consuming measures, and building usage patterns and square footage. Itron, Inc. 1-1 Introduction

5 Section 2, Research End Uses, Measures, and Segments, provides more details on the end uses, measures and building segments chosen for these research studies. Section 3, Data Sources, will discuss the multiple data sources that will be used to collect the desired information. 1.2 Goals and Objectives In order to extract the greatest value from these two research studies, the information needs from multiple groups have been collected, analyzed and reviewed. The research plan, data collection devices, and test survey findings have been reviewed and commented on by the CPUC, the IOUs, ED consultants, evaluation managers, and the Jeff Hirsch team 1. The careful development of the research and the data collection devices will lead to a more efficient collection of the desired information. Developing survey forms that leverage both the former California Energy Commission Commercial End-Use Survey (CEC CEUS) and Small Commercial Evaluation survey forms will ensure that the data collected represent both ongoing research interests and more recently developed technologies and information needs. A central goal of the CMST study is to describe the recent purchases of high priority measures from both end users and supply-side actors. Collecting information from supply-side actors will provide a broad picture of the current market for linear fluorescents and packaged HVACs. Collecting recent purchase information from end users, and verifying this information with onsite data collection, will provide precise information on the efficiency level of recent purchases and the rebate status of these purchases. These data collection efforts will be coordinated with the CSS and other studies to balance costs with the desire to collect information from multiple types of market actors. The central goal of the CSS study is to document the baseline distribution of energy using measures within commercial businesses. These data will provide the CPUC, IOUs, and other evaluations with information on which measures currently have a high saturation of high efficiency devices and which measures and business types have lower saturations of efficient measures and higher potential for future programs. These data will be compared with data collected from former saturations studies to show the progress achieved in replacing inefficient measures with newer, more efficient technologies. These data will also serves as inputs for future potential studies that will provide the CPUC and the IOUs with a detailed picture of the remaining achievable energy savings potential. 1 Jeff Hirsch and Associates has reviewed the research plan and contributed to the information needs and form development as it relates to the needs of the DEER project. Itron, Inc. 1-2 Introduction

6 1.3 Research Objectives The specific research objectives of the CMST study are listed below. 2 1) Determine the distribution of linear fluorescents by T12 and generation of T8 for lighting purchased by nonresidential business since ) Collect information on the types of linear fluorescent electronic ballast recently purchased. 3) Determine the share of recent purchases of efficient linear fluorescents that are incented by IOU rebates. 4) Collect information on the total number of televisions purchased by nonresidential businesses, determine the base and high efficiency share of recent TV purchases, collect self-report information on TV run times, and determine the share of recent TV purchases that represent TV replacements and those that reflect new load growth. 5) Determine the share of base and high efficiency packaged air conditioning units purchased by nonresidential customers and determine the IOU rebate status of these purchases. The research objectives of the CSS study center around determining the current baseline of equipment in commercial businesses. Some of the specific information to be collected and its research objectives are briefly listed below. The research objectives and coordination with other work orders are described in more depth in Section 2, Research End Uses, Measures, and Segments. 3 1) Collect information on the lighting employed in commercial businesses through a full inventory of commercial lighting. These data will provide a complete picture of the types of lighting currently employed. The data will illustrate the advancements made in linear fluorescents and depict the types of lighting currently used in medium screw sockets. 2) Determine the number and type of office equipment, including computers and printers, currently used in commercial buildings. These data will be compared to previous estimates of the saturation of office equipment to better understand the growth in this end use. 3) Determine the operating schedules, number, and types of televisions currently used in commercial buildings. These data will be combined with information on recent TV 2 The CMST study will include end-user on-site surveys in nearly all non-residential segments, excluding only transportation, communications, and utilities (TCU), agriculture, mining, and street lighting. See Section 2 for a more detailed description. 3 The CSS study will include on-site surveys in many commercial segments. The survey will not undertake surveys in the following segments, TCU, agriculture, mining, street lighting, industrial, hospitals, colleges, and lodging. See Section 2 for a more detailed description. Itron, Inc. 1-3 Introduction

7 purchases to determine the potential for future energy savings and to develop a better understanding of the energy usage associated with this quickly evolving measure. 4) Collect information on the HVAC equipment currently used in commercial buildings. The data collection will focus on packaged units while also collecting information on larger, more diverse systems. The data collected include make and model numbers, age, condition, operating schedules, and maintenance practices. These data will illustrate the current baseline of HVAC equipment and help to describe the potential of HVAC retrofits and quality maintenance services. 5) Determine the current baseline for commercial refrigeration equipment. The on-site survey will focus on both self contained and remote refrigeration. The data collected for self contained will focus on measures currently incented by IOU programs. The form will collect information on key characteristics and make and model number. The data collected for remote refrigeration will include information on cases, walk-ins, compressors and condensers. 6) Collect information on the building square footage that will be combined with the building energy usage to calculate whole-building energy intensities. These data will help to describe the whole-building energy intensities by building type and will provide a baseline for future studies. 7) Determine the current baseline for commercial customers aware of, and those who have participated in, integrated nonresidential audits where they may have learned about multiple forms of DSM including energy efficiency, demand response, and distributed generation. In the future, the baseline awareness collected during this study will be used to gauge progress made toward educating customers on integrated audits, DSM, and the potential to integrate multiple forms of DSM. 1.4 Research Plan Evolution over Time This research plan will guide the CSS and CMST research studies. One of the primary inputs into the development of the plan was the research needs of other evaluation teams and the DEER team, the information needs of the CPUC and IOUs, and future uses of the data for potential studies and to provide baseline data useful for evaluating Market Transformation Indices, Strategic Plan Progress Indicators, and Long Term Program Performance Metrics. The plan will continue to be reviewed and evolve overtime as new data becomes available, as well as when research findings from these or other studies become available. At a minimum, this plan will be updated as results from the telephone survey become available. The results from the telephone survey will be used to determine if some of the high priority measures should be re-evaluated or if additional measures should be incorporated into the analysis. It is not expected, however, that any major deviations from this plan will occur. Itron, Inc. 1-4 Introduction

8 1.5 Evaluator Contact Information Itron will serve as the IOU Prime Contractor managing the CSS and the CMST studies, led by Ms Jean Shelton. The ED Project Manager of the study is Ms Valerie Kao. The following is Ms. Kao s and Ms. Shelton s contact information: IOU Prime Contractor Jean Shelton, Ph.D. ED Project Manager Valerie Kao (858) (415) [email protected] [email protected] Itron, Inc. 1-5 Introduction

9 2 Research End Uses, Measures, and Segments This section summarizes the end uses, measures and segments to be incorporated into the CMST and the CSS studies. Furthermore, the section discusses why and how key decisions were made to include and exclude areas of study. The section also highlights the coordination between these research efforts and other work orders and how this coordination has impacted the studies designs. 2.1 CMST Measures and Segments High Priority Measures To determine the measures to incorporate into the CMST study, Itron reached out to other work order leads, the CPUC, DEER Team, and IOUs, requesting that they help to choose the measures that would be well suited for inclusion in the CMST. The criteria for inclusion included measures with significant program activity or recent code changes, measures whose sales are sufficient to justify a market share study, measures associated with the Strategic Plan or Market Transformation Indices and measures whose sales distribution was important but uncertain. The information needs process helped to determine that the highest priority CMST needs include the following: 1) The distribution of linear fluorescents and information on the types of electronic ballast purchased, 2) The total number of televisions purchased by commercial businesses, the base and high efficiency share, and additional TV options, and 3) The share of base and high efficiency commercial packaged air conditioning units purchased. In addition to the measures listed above, members of the CMST research team have suggested that the research include opportunistic questions with the telephone survey to capture recent purchases of other measures within the nonresidential market. Test telephone surveys conducted in December 2011 incorporated a question querying about other measures purchased since January of The preliminary results from the test telephone surveys found that many businesses had purchased computers during this time frame. Given the high incidence of recent computer purchases and the highly variable and changing nature of this measure, all telephone Itron, Inc. 2-1 Research End Uses, Measures and Segments

10 survey respondents will be asked if they have recently purchased computers. For sites agreeing to a CSS or CMST on-site survey, sites that that indicated in during the telephone survey that they had recently purchased computers will be asked about the number of computers purchased and the make and model number of recent computer purchases. Where possible, the make and model numbers will be verified during the on-site data collection. 4 The computer purchase information will be reviewed and analyzed to determine the number and types of computers purchased. Linear Fluorescent Lighting The Commercial Lighting Evaluation (WO029), the Measure Cost Study (WO017), and the DEER team are interested in gathering information on the purchases of linear fluorescents and ballasts and the types of lighting being sold and installed in high bay applications. With the upcoming implementation of federal lighting codes, most T12 lamps will no longer be manufactured. This code change will also impact IOU per unit claimed savings values. IOU commercial programs, in combination with federal standards, have encouraged California businesses to replace T12 lighting with T8 lighting for several years. High efficiency T8 lighting, and their ballasts, however, can be purchased in several different generations and configurations. The CMST study will help evaluators, the DEER team, and the IOUs better understand the current distribution of sales of T8 generation lighting and the ballasts installed during these retrofits. A better understanding of the share of sales across T8 generations is crucial as federal standards forbidding the manufacturing of T12 lamps come into effect. The end user market tracking information will also be combined with the program tracking information to better describe the types of lighting currently receiving IOU rebates and the types of lighting measures purchased outside the IOU programs. This study will help to describe the current baseline of rebated and non-rebated lighting purchases. These baselines are needed to more accurately calculate the savings from T8 lighting retrofits. Information on recent T8 lighting purchases undertaken with and without IOU rebates will also help the IOUs better develop programs under the new lighting standards. The supply-side actor market tracking information will also provide a broad picture of types of linear fluorescent lighting being purchased in California by commercial businesses. Televisions The CMST study will coordinate with the Business and Consumer Electronics (BCE) evaluation team (WO034) to collect information on recent purchases of televisions within the nonresidential 4 Collecting information on the make and model number of computers requires that the surveyors evaluate the trade-offs associated with collecting the necessary information and disturbing the customers. Where possible, the make and model number information will be collected from site contacts who may be responsible for ordering and/or configuring the new computers. Itron, Inc. 2-2 Research End Uses, Measures and Segments

11 sector. The BCE team is evaluating the energy savings achieved from IOU rebated high efficiency business and consumer electronics, including televisions. Given the upstream nature of IOU television rebates it is difficult to determine if a rebated television measure is intended to be used by residential or nonresidential customers. The hours of use and resultant savings achieved from high efficiency TVs, however, is highly dependent on sector and business type. The CMST study will collect information on recent purchases of TVs by businesses. The study will help to determine the number of TVs purchased by nonresidential customers and the high or standard efficiency of the purchase. Combining the CMST data with the IOU tracking data will help to determine if the share of nonresidential TV purchases reported by IOU tracking data is consistent with nonresidential TV purchases. The questions incorporated into the CMST telephone survey will enable the team to better determine the business types purchasing TVs and may lead to the development of better estimates of the hours of TV use by businesses. Analysis of the telephone survey data will also help to determine the share of TVs purchased as replacements for existing TVs and those representing load growth. The telephone and on-site verification data collected by the CMST study on recent television purchases by the nonresidential sector will provide a baseline of the current TV market purchases that will be used to describe future changes in this quickly evolving market place. The data collected will provide new information on the current share of high efficiency TVs in the nonresidential sector; it will assist with the development of nonresidential savings within the BCE evaluation, and will provide data for the DEER team if they choose to include TVs within the DEER database. Package Air Conditioning The CMST study will coordinate with the DEER team, the Residential and Small Commercial HVAC Evaluation ( WO032), the Measure Cost Study (WO017) and the Residential and Small Commercial HVAC Market Effects Study ( WO054) to track the purchases of base and high efficiency packaged air conditioning units to the nonresidential sector. For many business types the primary source of air conditioning is packaged units. The IOU tracking databases record units rebated through utility programs, but very little is known about the number of base efficiency units purchased or the share of high efficiency units purchased without a utility rebate. The market share tracking study will help to determine the distribution of base and high efficiency sales and will determine the share of high efficiency sales receiving an IOU rebate. This information will help the evaluation and DEER team better understand the current baseline of air conditioning purchases and will help the IOUs better understand the influence of their programs on these purchases. Itron, Inc. 2-3 Research End Uses, Measures and Segments

12 CMST Business Segments The market share tracking effort will collect information from end users and supply-side actors on recent purchases and sales of linear fluorescents and packaged HVAC units. The research will also collect end user information on purchases of televisions. To ensure that sales and purchase data fully represent the nonresidential market for these measures, the study will incorporate business segments that represent both the commercial and industrial sectors. Specifically, the supply-side actor surveys will question contractors and/or distributors on the sales of linear fluorescents and packaged HVAC units to all nonresidential customers. The end users telephone survey will contact end users in the nonresidential frame other than those classified as agricultural, street lighting, mining, and transportation, communications, and utilities ( TCU). 5 The end users on-site verification survey will contact those end users who responded to the telephone survey that they had recently purchased a high priority measure and they were willing to participate in the on-site verification effort. 2.2 CSS End Uses, Measures, and Segments CSS End Uses and Measures To determine the end uses and measures to incorporate into the CSS study, Itron reached out to other work order leads, the CPUC, DEER Team, and IOUs, requesting that they provide input into end uses and measures that would be well suited for inclusion in the CSS. The criteria for inclusion included end uses and measures with significant program activity or recent code changes, end uses, measures, and services associated with the Strategic Plan or Market Transformation Indices, end uses that represent a significant quantity of energy usage, and those whose current saturation was high uncertain or likely to be undergoing rapid change. In addition, the JJH team was asked to provide specific information on the types of information that could be gathered from a commercial population survey that would help to clarify existing uncertainties within the DEER saving calculations. The end uses, measures, and additional information incorporated into the telephone and/or on-site surveys for the CSS are listed below. A more complete description follows. 1) Commercial Lighting 2) Office Equipment 3) Televisions 4) HVAC 5) Refrigeration 5 During the NAICS to business type classification process, the IOUs, CPUC, ED consultants, and Itron decided to eliminate these business types from the survey. The research group also eliminated a limited number of additional business types including non-common area residential buildings and parking lots. Itron, Inc. 2-4 Research End Uses, Measures and Segments

13 6) Program Awareness and Participation 7) Energy Management Systems 8) Distributed Generation 9) Firm Demographics Commercial Lighting The CEC CEUS analysis estimated that lighting accounted for approximately 35% of commercial energy usage. In addition, lighting measures represent technology long targeted by commercial energy efficiency programs and recent technology code updates. For these reasons, the CSS on-site survey effort will include a full inventory of commercial lighting measures. The development of the lighting questions has been coordinated with WO029. This survey provides the energy efficiency community with the opportunity to collect information on lighting for the population of commercial sites, a group of sites whose lighting technology distribution may differ from the energy efficiency lighting participants more often incorporated into verification on-site evaluations. The on-site data collection effort will collect make and model number information on lighting and ballasts wherever possible. The surveyors will collect information on the number of fixtures and lamps, the types of fixtures and lamps, and their special features. The verification effort will ask lighting schedules, age, and condition of lighting measures. The on-site data collection will also collect information on lighting controls, recent delamping and recent purchases. The telephone survey incorporates a series of questions that attempt to determine if the business has recently purchased CFLs through retail outlets. These questions will help to identify sites who may have participated in the upstream CFL rebate programs. These questions have been coordinated with WO028. The extensive quantity of commercial lighting information collected during the CSS on-sites will enable the CPUC, IOUs, and evaluation community to more fully understand the distribution of lighting technologies within commercial business. These data will be analyzed to provide information on saturations and efficiency levels by IOU and business type. Office Equipment According to analysis undertaken as part of the former CEC CEUS, office equipment was the fourth highest electricity consuming end use within commercial businesses. 6 The number of 6 The CEUS reflects the best available information on the distribution of energy use within the California commercial population. The data in the CEUS reflect the commercial energy consumption during the period. Itron, Inc. 2-5 Research End Uses, Measures and Segments

14 different types and configurations of office equipment has grown dramatically since the CEC CEUS and the expansion of commercial businesses employing cubical office designs has likely increased the density of these measures. The likely growth in density and in the different types of office equipment, including computers and printers, points to the need to incorporate these measures into the CSS. The CSS survey will collect information on office equipment observed in commercial businesses. The on-site data collection will indicate if the equipment is for personal or general use, if it uses a smart strip, is an Energy Star product, and the hours of weekly usage. The data collection effort will ask site contacts about recent purchased computers, collecting make and model number information for new computers wherever possible. Televisions The use of televisions within commercial businesses is rapidly expanding, but inadequately documented. In addition, the IOUs are providing upstream rebates for television. These rebates may have contributed to recent reductions in the price of this measure and an expansion in recent purchases. Given the upstream nature of these rebates, it is very difficult to determine if the measure is installed in a residential or nonresidential location. The development of the television questions in the telephone and on-site survey forms has been coordinated with WO034. The CSS telephone and on-site surveys will collect information on the incidence and number of televisions within commercial businesses. The on-site survey will collect information on the televisions make and model number, the television s size, age, and features. The survey will also collect information on the television s usage schedule and any connected devices. With these data, it will be possible to develop a better understanding of the current share of businesses with televisions and the density and usage patterns of these televisions. HVAC HVAC measures account for approximately 29% of the electricity usage within commercial businesses, the second highest end-use share. 7 In addition, HVAC measures have been the target of long standing commercial energy efficiency programs and ongoing code changes. These measures also hold a dominant position in the California Strategic Plan. Therefore, these measures are central to the CSS study. The CSS telephone and on-site surveys collect extensive information on HVAC equipment. The on-site data collection effort includes the collection of make and model number, HVAC schedules, age, and condition. The survey will collect both cooling and heating information. 7 The share of HVAC energy use is derived from the CEC CEUS. Itron, Inc. 2-6 Research End Uses, Measures and Segments

15 The on-site survey will collect information on the cooling type, refrigerant type, and number of compressors. The CSS telephone and on-site surveys will also question businesses about their HVAC maintenance practices. The site contact will be questioned about the frequency of maintenance, who performs the maintenance, and if the site participates in utility-sponsored quality maintenance programs. The maintenance questions for the telephone and on-site surveys will be coordinated with WO032 and WO054. Coordination between the CSS/CMST studies and WO032 and WO054 may lead to some short term (one hour) monitoring of select HVAC units. With these data it will be possible to describe the distribution of HVAC equipment in California businesses and the current share of energy efficient climate appropriate equipment. These data will help the IOUs plan future energy efficiency programs and will be used in future energy efficiency potential studies. These data will help to determine the current baseline from which to measure progress toward Market Transformation and Strategic Plan Goals. Refrigeration The CEC CEUS estimated that refrigeration was the third highest energy using end use at 13% of commercial electricity usage. In addition, refrigeration has been the focus of ongoing utility programs, but it is unclear the extent of program penetration across measure types and market segments. The CSS on-site survey will collect extensive information on remote and self-contained refrigeration equipment. For remote refrigeration, the on-site survey collects information on the type and temperature of the case, the defrost control type and the size of the cases. Similar types of data will be collected for walk-ins and preparation areas. The study will collect information on the lighting used in display cases and walk-ins. Collection of data on compressors, condensers, and system controls is also highlighted on the survey forms. For self-contained refrigeration equipment, the surveyors will collect information on service type, size, condition, and year of manufacture. The refrigeration on-site data collection will be focused on business types with extensive refrigeration requirements, including food and liquor stores, restaurants, and refrigerated warehouses. With the data collected during these surveys, it will be possible to describe the current state of refrigeration measures in these three building segments. These data will help the DEER team better understand the current baseline of refrigeration measures, the IOUs plan future programs, and will be used in future energy efficiency potential studies. Itron, Inc. 2-7 Research End Uses, Measures and Segments

16 Program Awareness and Integration The CSS telephone survey will collect information on customer awareness of IOU programs that encourage energy savings. The program awareness and integration questions have been reviewed by the CPUC, the ED consultants, and the Nonresidential Process Evaluation (WO012). The program awareness and integration questions will ask businesses if they are aware of programs designed to reduce their energy bills. Businesses will be asked about their businesses participation in demand response programs and their use of distributed generation. The program awareness questions will determine the number of customers aware of nonresidential audits and will question those sites that have participated in audits, if the audit recommendations integrated multiple forms of demand-side management. The answers to these questions will provide a current baseline for the share of customers who are aware that they can receive an integrated audit and those who have participated in an integrated audit. The telephone survey will also inquire about the site s adoption of distributed generation measures and demand response programs following the audit. The answers to this series of questions will provide a baseline from which to measure progress toward fulfilling Program Performance Metrics associated with integration and integrated nonresidential audits. Energy Management Systems The CSS on-site survey will collect data on energy management systems found on-site. The data collected will indicate the end uses controlled by the system, and the manufacturer and/or service provider. The survey will collect information on the number of points and the number of controllers. The survey will determine if the system is used for demand response and will determine if the demand response implementation methodology is manual, activated through a signal received from a third party, or based on an alternative algorithm. If the EMS system is used for demand response, the survey will ascertain the loads controlled. The surveyor will also request a print out or computer copy of data on the end use operation or control data. The EMS data collected during the on-site data collection will provide the IOUs and the CPUC with a better understanding of the current baseline of EMS usage. This information will be useful for future program planning for energy efficiency and demand response. The data will also be incorporated into future potential studies. Distributed Generation The distributed generation questions incorporated into the CSS on-site data collection instrument are designed to provide the demand-side management community with a baseline of the current distribution of cogeneration, self-generation, solar cells or photovoltaic systems and emergency generators employed in California businesses. The survey will determine if the distributed Itron, Inc. 2-8 Research End Uses, Measures and Segments

17 generation system is primarily an emergency generator or if the system is more consistently used to generate electricity. The survey will record the type of technology, the fuel, number of unit and year of installation. The survey will collect information on make and model number whenever possible. If the system is a co-generation system, information will be collected on the use of the heat generated. The site contact will also be questioned about the site s use of the system to enable demand response. Importantly, distributed generation sites will be questioned about the system s average daily operating hours, number of operating days per year, and the presence of metering equipment. The distributed generation data collected by the CSS survey is not intended to replace the indepth data collection undertaken by the California Solar Initiative (CSI) or the Self Generation Incentive Plan (SGIP). The data collected through the CSS are intended to provide a baseline description of distributed generation equipment currently operating in California businesses. This information will be useful for program planners and will be incorporated into future distributed generation potential studies. Firm Demographics The CSS survey collects extensive information on a wide variety of measures that could be grouped under the heading of firm demographics. First, the telephone and on-site surveys will provide a better understanding of the current distribution of business types within California. The IOU s nonresidential frame includes NAICs codes designed to describe the site s business activity. Coordinating with WO009, the Data Management Work Order, the NAICs codes were mapped to building types appropriate for the CSS and CMST studies. The NAICs to building type classification led to approximately 10% of the usage and 20% of the sites being allocated to building types that are unclassified or those that do not reflect the business activity at the site. 8 The telephone and on-site surveys will redefine these sites into their appropriate building type, providing a better description of the business activities associated with commercial electricity usage in California. The building type distribution will be discussed in more depth in Section 3, Description of the Primary and Secondary Data Sources. The telephone and on-site surveys will collect information on the year the buildings on the site was built, recent remodels, and the year the business was established in the building. The on-site survey will collect information on the meter numbers associated with the building for both electricity and gas usage. This information will be used to check the site aggregation and usage 8 The 10% of usage and 20% of sites that are unclassified do not include miscellaneous sites. Miscellaneous sites are associated with various business activities that lead the site to be designated miscellaneous. Unclassified sites include sites without a NAICs code or with codes that are unclassified. Sites whose NAICs codes do not reflect the business activity at the site are largely property management sites. The property management classification commonly reflects the business activity of the customer paying the utility bill, but does not usually reflect the business activity at the business location. Itron, Inc. 2-9 Research End Uses, Measures and Segments

18 associated with the site. Both the telephone and the on-site surveys will collect information on the business square footage. This information, in combination with the electricity consumption at the business, will be used to calculate the business whole building energy intensity. The surveyors will create a high-level sketch of the site that will describe how the site and its surroundings are configured. Surveyors will be taking multiple pictures of the exterior, and where possible, of high priority measures. Surveyors will collect information on the building shell construction and the windows. They will collect data on the roof, ceiling, and wall construction and the presence of insulation. Glazing and window frame types will be collected for windows, in addition to an estimate of the window-to-wall percentage for each side of the building. Information on skylights will also be recorded. The survey will also collect data on self-reported business and operating schedules. The surveyors will collect information on daily business hours and standard holidays. The business hour data will incorporate seasonal operation schedules. Self-reported hourly operating schedules will also be collected for lighting, televisions and HVAC equipment. Surveyors will ask the site contact about the percent of lighting equipment and television typically on during the stated lighting and television schedules. For HVAC, site contacts will be questioned about the typical set points for both heating and cooling equipment. The extensive collection of information associated with firm demographics will provide the CPUC, evaluation consultants, and the IOUs with a better description of California commercial businesses. Combining this information with the measure specific information collected during the CSS survey will provide this community with information on the distribution of measures and end uses by specific building types and configurations. The ability to analyze the measure specific information by firm demographics will provide more disaggregated information. Section 6 provides more information on output results possible with these data. CSS Business Segments The commercial saturation survey will collect a vast quantity of information on energy consuming measures and firm demographics within commercial businesses. Given the quantity of data collected during this survey, trade-offs associated with the benefits and costs associated with data collection were required. The trade-offs looked at segments surveyed during previous commercial saturation surveys, the cost to collect information from particular segments, the availability of alternative information on segments, and the IOUs desire for segment specific data. The former CEC CEUS surveyed small offices, large offices, restaurants, retail, food and liquor stores, warehouses and refrigerated warehouses, schools, colleges, health care, hotels and motels, and miscellaneous. The cost of surveying large offices, colleges, hotels and motels, and Itron, Inc Research End Uses, Measures and Segments

19 hospitals was found to be high within the former CEC CEUS due to their size, likelihood of multiple buildings on a site, and diversity and complexity within a site. The CPUC, ED consultants, IOUs and the ED contractors discussed the value and costs associated with incorporating alternative segments in the CSS telephone and on-site surveys. Given the relatively low cost of a telephone survey, combined with the overlap between the CSS and CMST telephone survey, the group decided the CSS telephone survey would contact the same segments as the CMST survey. Therefore, the CSS telephone survey will collect information from the nonresidential frame other than those classified as agricultural, street lighting, mining, and TCU. The CSS and CMST telephone survey is designed as a single survey fielded to a large share of the nonresidential population. The telephone survey development will be further discussed in Section 5: Survey Development. The CSS on-site survey will collect information from a sub-set of sites included in the telephone survey. Given the cost of collecting data from larger, more complicated and diverse sites, the CSS on-site data collection effort will not include industrial sites, colleges, hotels and motels or hospitals. While the cost of collecting data from large offices is high, the group determined that the benefits were higher. The collection of data within large offices, however, may be more limited than in other segments. Itron, Inc Research End Uses, Measures and Segments

20 3 Data Sources This section of the Plan outlines the key data sources incorporated into the research. The data described will include the input data sources, how these data are combined, and the data sources that will be collected during the studies. 3.1 Key Input Data Sources The key input data sources used for the CMST end-users study and the CSS are the IOU nonresidential customer information systems (CIS), the IOU billing data, and the IOU program tracking datasets. Key input data sources for the CMST supply-side study will include lists of lighting and HVAC contractors and distributors. The CMST study will interview both program participant and nonparticipant supply-side actors where possible. The CMST study will coordinate with WO017, WO029, WO032, and WO054 to develop contact lists, surveys and for survey implementation. Nonresidential Site Development The first step in the data development of these research projects was the development of nonresidential sites. The nonresidential site development was largely undertaken as part of Data Management Work Order (WO009). The nonresidential customer information, meter, and account level data were reviewed and nonresidential sites have been developed using electricity billing information. 9 Itron developed algorithms to aggregating account and meter level information into nonresidential sites. These algorithms underwent extensive review and refinement under WO009. The site development that will be used for the CSS and CMST studies is based on data from PG&E, SCE, and SDG&E s electric account information. Gas data will also need to be assembled for study participants. Combining the gas and electric data for SDG&E and PG&E dual fuel customers is relatively straight forward. Aligning the electric and gas accounts for 9 The frame for the study is the IOU electric billing data. Participation in the study will be offered to LADWP and SMUD. The gas usage for SCG customers will be matched to the electrical usage following participation in the on-site survey. Itron, Inc. 3-1 Data Sources

21 customers who do not receive both fuels from a single utility, however, requires considerably more effort. WO024 will assemble the data needed to develop the gas and electric site information of customers receiving gas and electricity from two utilities for on-site participants in the CSS. Building Type Classification Under Task 3 of the CSS/CMST studies, the team reviewed with the DEER team, the evaluation teams, IOUs, and the CEC the desired building type categories to be used to develop the sample design. Itron collected NAICS code mappings from the CEC, IOUs, DEER, and evaluation teams. The NAICS code mappings were reviewed for consistency. The CSS/CMST team reviewed the mappings based on the purpose of these studies. The review was completed using a hierarchy based on the share of the IOU nonresidential frame associated with each code. The codes were sorted from largest to smallest based on the kwh usage. The team reviewed the NAICs to building type mappings for the codes that represent the largest 95% of the classified nonresidential frame. The team produced tables describing where the different mappings agree, where they disagreed and the CSS s final mapping designation. The team reviewed the mappings with the CPUC, CPUC consultants, and the IOUs. NAICS code disagreements were discussed within the study group, with the group deciding on the final mapping and the eligibility of the code for participation in the CSS and CMST surveys. For the remaining 5% of the frame, the Itron team reviewed mappings within two digit NAICS codes and commonly used the CEC s mapping where the CEC has mapped the NAICS. If the CEC has not mapped the NAICS and SDG&E has mapped the code, the team will use SDG&E s mapping. 10 NAICS codes are attached to individual accounts in the IOU nonresidential frames. During the site development, multiple individual accounts may be aggregated into a single commercial site. If the NAICS codes across the individual accounts lead the site to be incorporated into multiple building categories, the building classification algorithm uses the dominant building category based on the account level electricity usage. The nonresidential frame, augmented with defined business types and program participation flags, forms the input data basis for these two nonresidential population surveys. Table 3-1 lists the total number of sites and the total usage by business type as developed under WO009 and Task 3 of the CSS/CMST studies. The site development process led to over 1.2 million sites consuming over 116,000,000 MWh of electricity in The most populous business types are miscellaneous, unclassified, and agricultural sites. The industrial segment uses substantially more electricity than any other business type. The distribution of usage and sites will be further discussed in Section 4, Research Sample Design. 10 The CSS/CMST review of the different NAICs to building type mappings has found that the research team most often agrees with the CEC mapping when the mappings from the different sources were inconsistent. SDG&E is the only IOU to provide the CSS team with NAICs to building type mapping. Itron, Inc. 3-2 Data Sources

22 The distribution of business types incorporated into Table 3-1 includes those business types that are in neither the CMST nor the CSS studies. The business types not included in either study are label as Not in Study in the table. The Not in Study business types are the only business types excluded from the CMST study. The CSS study excludes the Not in Study business types and hospitals, hotels, industrial sites, and colleges. See Section 2 above for a discussion of the business types included in each study. Table 3-1: Nonresidential Population by Business Type CSS/CMST NAICS Business Type Total Number of Sites Total Usage 2010, MWh College 1,563 1,817,479 Food/Liquor 24,824 6,225,206 Health Care - Care 3, ,914 Health Care - Hospital 1,112 2,988,220 Health Care - Med Office 37,256 1,895,539 Hotel 7,354 2,656,040 Industrial 40,730 22,455,634 Misc 150,125 9,331,060 Misc - Assembly 20, ,195 Misc - Res Common 64,165 1,258,139 Office 131,018 9,396,935 Property Managers 66,459 7,860,483 Restaurant 64,858 6,012,116 Retail 77,522 7,731,583 School 12,128 3,380,157 Warehouse 37,936 4,061,992 Unclassified 14, ,800 Undefined 168,075 2,944,952 Not in Study 5,533 1,038,524 Not in Study - Agriculture 130,922 9,019,772 Not in Study - Mining 14,901 3,740,655 Not in Study - Street Light 47, ,831 Not in Study - TCU 99,134 9,924,256 Total 1,222, ,167,482 Eligible for Telephone Survey 923,803 91,876,443 Eligible for CSS 873,044 61,959,071 Program Tracking Data Under WO009 the program tracking data has been integrated with the nonresidential site information. The energy efficiency program tracking data integration incorporates flags indicating that a site has participated in particular programs, the work order associated with the program participation, and flags by end use and measure group. These flags and information Itron, Inc. 3-3 Data Sources

23 associated with the rebated measures are provided to the survey team on the on-site survey form. Incorporating this information insures that the survey team is aware of recent installations that were rebated by the program. The program participation flags also allow the sample selection process to coordinate with the Custom Evaluation (WO033) and the Nonresidential Lighting Evaluation (WO029). Sit es that have participated in these programs are eligible for participation in the CSS and CMST studies. Telephone interviews and on-site data collection at these sites, however, will be coordinated with these work orders to insure that the necessary information in collected with a minimum of customer contact. WO009 and WO024 are also working to integrate program participation information from IOU demand response and distributed generation programs. Program participation information has been provided for IOU demand response programs. A flag indicating demand response program participation will be printed on the on-site survey forms. Program participation information for distributed generation programs is being developed. This information will be printed on the survey forms as it becomes available. 3.2 Key Input Data Integration and Output Data Description As depicted in Figure 3-1, the CMST end-user study and CSS study both begin with the integration of the IOU nonresidential CIS, billing, and program tracking datasets. Integrating these data was the first step in the development of the nonresidential frame. The data management team under WO009 has used these data to develop nonresidential sites with program tracking flags attached. These data were extensively reviewed, with special attention paid to the billing information and the business type classification. The business type classification will be described more fully in Section 4, Research Sample Design. Simultaneous to the development of the nonresidential frame, the research team has been developing the phone and on-site surveys. The phone and on-site survey development has drawn on telephone and on-site surveys previously developed for the Small Commercial Evaluation and the 2006 CEC CEUS on-site survey form. These forms have been combined and modified to fulfill the unique needs of the CMST and CSS surveys. These forms have been tested and reviewed by the CPUC, IOUs, and associated evaluation teams. The telephone and onsite surveys will be described more fully in Section 5, Survey Development. The test implementation of the telephone survey began in November 2011 and a small scale test implementation of the CMST on-site survey began in December These test surveys have provided useful information on survey length, the willingness of respondents to participate in onsite surveys, and the share of the population who were recent purchasers. Itron, Inc. 3-4 Data Sources

24 Figure 3-1 illustrates how the CATI phone survey respondents will be recruited for the CSS and CMST survey efforts. The data collected from these studies will be analyzed. The data and the results will be made available to multiple parties including other work orders, the CPUC, IOUs, and the DEER team. Section 8 describes the timeline for the study and Section 9 lists the individual tasks. Figure 3-1: Flow Chart for the CSS and CMST Studies Non-Res CIS Data Itron SAS Code Master Dataset w/ SiteIDs and Flags for Participation IOU Data Non-Res Billing Data CATI Phone Survey CATI Survey to Include Questions on: Demographic Recent Purchases Dual Recruiting AKA-B Typical Remodels Purchasing Decisions Program Tracking Data Data from All Respondents Used for Mult Purposes Recruits Dual Recruiting CATI Survey Dual Recruiting: All respondents recruit for full on-site survey until 20% over quota is reached. If they decline and they are a Recent Purchaser of high priority equip recruit for quicker verification style on-site. Rapid Feedback EU Specific On-Sites Full CSS On-Site Survey Other WOs AKA-B Other possible items will continue coordinating with other WOs CMST/CSS Self Reported Recent Purchases (by meas group) Incl Where SR retrofit cycles, contractor roles, corp decision Could target specific building types Market Shares (could cover 2-3 year period, by year possible depending on sample) Detailed Data on Retrofits/New Purchases Saturations of Equip Types Saturations of High Efficiency Equip Specific (just ex s): Where are the remaining T12s Multi-plex cases Possible EUIs, etc Results 3.3 CMST Supply-Side Data The CMST supply-side survey will query lighting and HVAC contractors and/or distributors about the quantity of sales, the efficiency levels of recent sales, and the level of IOU program participation. The lighting CMST supply-side actor surveys will focus on lighting contractors while the HVAC survey will interview both contractors and distributors. Both the lighting and the HVAC surveys will attempt to interview supply-side actors who are participating in IOU programs and those who are not participating. Itron, Inc. 3-5 Data Sources

25 The survey design and data collection process for the supply-side actor surveys will be highly integrated with other work orders that will also be contacting the same limited number of supplyside actors. The lighting supply-side actor survey will coordinate with the Nonresidential Lighting Evaluation (WO029) and the Measure Cost Study (WO017). The HVAC supply-side actor survey will coordinate with the Measure Cost Study (WO017), the Residential and Small Commercial HVAC Evaluation (WO032) and the HVAC Market Effect Study (WO054). At this time, the measure cost study is also investigating the possibility of collecting point of sales data for linear fluorescent. If WO017 is able to purchase POS data for linear fluorescent covering a large enough share of distributors within California and at a fine enough degree of disaggregation, the CMST study will coordinate with the Measure cost study purchase and evaluate these data. Itron, Inc. 3-6 Data Sources

26 4 Sample Design As discussed in Section 2, the primary data collection activities planned for the CMST and the CSS studies are: 1) Telephone survey of the nonresidential population 11 2) On-site verification of nonresidential recent purchasers of linear fluorescents, TVs, and HVAC 3) On-site saturation data collection with commercial businesses 12 The goal of the sample design is to develop a representative sample that provides the desired level of statistical precision for estimating a wide range of commercial customer characteristics largely represented by the share of sites purchasing high priority measures and the saturation of electricity consuming end uses and measures. For the CSS saturation estimates, the sample design will attempt to achieve a relative precision of at least 90/10 at the utility level and at least 90/15 at the building type level. 13 To reach this goal, alternative sample-designs have been evaluated. The sample designs put forth in this Research Plan are preliminary in nature. The finalized sample designs will depend on the distribution of business types determined by the telephone surveys. This is discussed in more detail below. Once the study goals were established, the sample design process began with the development of the sample frames for these studies. Following the development of the sampling frames and the stratification approaches, the sample sizes are computed to reach the desired level of precision given the budget constraints. 11 As described in Section 2 and Section 3, agricultural, mining, street lighting and TCU have been eliminated from the telephone survey sample frame. 12 As described in Section 2 and Section 3, the commercial population included in the CSS on-sites is limited to the telephone survey sample frame, with industrial sites, colleges, hospitals, and hotels and motels also eliminated from the survey frame. 13 Given the large number of building types, it will not be possible to achieve 90/15 for all building types by IOU. A 90/10 confidence level and relative error implies that, if the research were undertaken repeated in a similar approach, that the calculated confidence interval would encompass the true population parameter of interest 90% of the time. The 10% relative error statistics refers to the precision of the estimated value. The 10% relative precision indicates that the estimated value is within plus or minus 10% of the true value. Itron, Inc. 4-1 Sample Design

27 4.1 Develop the Sampling Frame As mentioned in Sections 2 and 3, the sampling frame for the telephone surveys is the nonresidential frame following the elimination of agricultural, mining, street lighting and TCU sites. The sites incorporated into the sample frame for the CMST on-site verification will be sites in the sampling frame for the telephone survey that have recently purchased either linear fluorescents, televisions, and/or packaged HVAC. 14 The CSS on-site data collection sample frame will be restricted to commercial sites. The CSS sites will be recruited using the joint telephone survey, where industrial sites, colleges, hotel and motel sites, and hospitals will be designated as not eligible for the CSS on-site data collection effort. Prior to creating the CMST and CSS sample frames, IOU meter and account level data were aggregated into sites under WO009. These data were also classified into business types using NAICS codes under WO024. The process of site aggregation and business type classification were described in Section 3. Table 4-1 lists the CSS/CMST building type distribution by IOU and aggregated to the statewide level by the number of sites. The table shows that miscellaneous, agricultural, office, and unclassified buildings are the most common sites in the nonresidential frame by number of sites. Not all types of building sites are eligible for the CSS/CMST studies, while some sites are eligible for the CMST study but not the CSS study. By number of sites, PG&E has approximately 48% of nonresidential sites, while SCE has approximately 42% and SDG&E has 10%. 14 The telephone survey sample frame include business designated as property management and unclassified and undefined. If the property management or unclassified/undefined sites are found to agricultural, mining, street lighting or TCU sites, they will not be eligible for the CMST on-site verification. Itron, Inc. 4-2 Sample Design

28 Table 4-1: Nonresidential Sites by IOU and Business Type CSS/CMST NAICS Business Types PG&E Sites SCE Sites SDG&E Sites Total College ,563 Food/Liquor 11,490 10,456 2,878 24,824 Health Care - Care 1,789 1, ,168 Health Care Hospital ,112 Health Care - Med Office 16,000 16,815 4,441 37,256 Hotel/Motels 3,796 2, ,354 Industrial 14,013 22,346 4,371 40,730 Misc 64,288 71,555 14, ,125 Misc - Assembly 9,992 8,667 1,977 20,636 Misc - Res Common 9,463 52,685 2,017 64,165 Office 51,310 52,105 27, ,018 Property Managers 17,661 41,046 7,752 66,459 Restaurant 29,163 28,255 7,440 64,858 Retail 30,414 36,317 10,791 77,522 School 6,177 4,727 1,224 12,128 Warehouse 15,540 19,037 3,359 37,936 Unclassified 5,108 9, ,874 Undefined 124,599 39,514 3, ,075 Not in Study 2,516 2, ,533 Not in Study Agriculture 96,513 29,340 5, ,922 Not in Study - Mining 6,285 4,830 3,786 14,901 Not in Study - Street Light 26,315 14,669 6,899 47,883 Not in Study - TCU 46,743 43,041 9,350 99,134 Total 590, , ,501 1,222,176 Eligible for Telephone Survey 411, ,073 93, ,803 Eligible for CSS 392, ,936 88, ,044 The distribution of sites includes property management, unclassified and undefined as business types. These classifications do not clearly define the business activity undertaken at the business location. The property management classification typically describes the business activity of the site s telephone contact, not the business activity of the site. Unclassified and undefined sites are sites without NAICS codes or sites with NAICS codes that were not defined. These sites will be incorporated into the sample frame, the telephone survey sample design, and the telephone survey. During the telephone survey, the activity at these sites will be determined and the site will be allocated to their actual business type The site s business type will be reclassified following the telephone survey but the site s weight will be determined by its original allocation within the sample frame. Itron, Inc. 4-3 Sample Design

29 Table 4-1 aggregates the number of sites by those eligible for the telephone survey and those initially eligible for the CSS on-site survey. Approximately 75% of the nonresidential sites are eligible for inclusion in the CMST/CSS telephone survey. The total eligible for the CSS on-site survey should be considered preliminary, representing those sites eligible based on their CIS business classification. As the business types are updated during the telephone survey, the total number and share of sites eligible for the CSS survey will be updated. Given the large number of sites classified as property management, unclassified, and undefined, even if no other sites change their business type classification (a extremely unlikely hypothesis), the final distribution of business types is likely to be significantly modified following the telephone surveys. These updates will also lead to modifications in the final on-site simple design. Table 4-2 lists the CSS/CMST business type distribution by IOU and aggregated to the statewide level by electricity usage in The table shows that the industrial sector is the largest user of electricity, consuming over 22,000 GWh in Other large energy using business types include miscellaneous, agricultural, TCU, and offices. By usage, SCE nonresidential sites consume approximately 45.7% of nonresidential usage, while PG&E sites consume approximate 43.5% and SDG&E sites consume 10.8%. Table 4-2 also lists the quantity of electricity usage eligible for the telephone survey and those initially eligible for the CSS on-site survey. Approximately 79% of the nonresidential sites by usage are eligible for inclusion in the CMST/CSS telephone survey. The total eligible for the CSS survey should be considered preliminary, representing those sites eligible based on their CIS classifications. As discussed above, updates to site level business classifications will lead to modifications in the quantity of usage eligible for the CSS on-sites. These updates will lead to modifications in the final on-site simple design. Itron, Inc. 4-4 Sample Design

30 Table 4-2: Nonresidential 2010 MWh by IOU and Business Type CSS/CMST NAICS Business Types PG&E MWh SCE MWh SDG&E MWh Total MWh College 760, , ,230 1,817,479 Food/Liquor 2,536,369 2,956, ,882 6,225,206 Health Care - Care 393, , , ,914 Health Care Hospital 1,078,383 1,571, ,085 2,988,220 Health Care - Med Office 772, , ,341 1,895,539 Hotel/Motels 1,073,439 1,054, ,281 2,656,040 Industrial 8,989,207 12,181,657 1,284,770 22,455,634 Misc 4,195,212 4,313, ,502 9,331,060 Misc - Assembly 270, ,530 86, ,195 Misc - Res Common 332, ,855 77,021 1,258,139 Office 4,252,623 3,063,466 2,080,846 9,396,935 Property Managers 3,512,628 3,789, ,277 7,860,483 Restaurant 2,237,780 3,050, ,159 6,012,116 Retail 2,680,578 4,089, ,454 7,731,583 School 1,271,689 1,778, ,296 3,380,157 Warehouse 1,807,844 2,035, ,221 4,061,992 Unclassified 209,346 33, ,800 Undefined 2,513, ,611 35,705 2,944,952 Not in Study 293, ,040 58,077 1,038,524 Not in Study - Agriculture 5,599,874 3,089, ,225 9,019,772 Not in Study - Mining 2,091,284 1,558,242 91,129 3,740,655 Not in Study - Street Light 431,528 77,387 58, ,831 Not in Study - TCU 3,267,264 4,045,763 2,611,229 9,924,256 Total 50,571,753 53,078,906 12,516, ,167,482 Eligible for Telephone Survey 38,888,397 43,620,800 9,367,247 91,876,443 Eligible for CSS 26,986,739 27,975,450 6,996,881 61,959,071 The sample frame eligible for the CMST on-site verification effort is highly dependent on the distribution of sites found purchasing high priority measures. The sample design for the CMST will be largely based on purchase behavior observed from the telephone surveys. 4.2 Stratification Stratification in the sample design is used for three purposes: To reduce the overall variance of estimates of variables included in the study, to optimize costs at a given level of precision, and to ensure adequate precision at the stratum level. The strata can be defined in terms of many different features. For this project, we propose to stratify the on-site survey by IOU, business type, climate region, and total electricity consumption. Stratification on business type and Itron, Inc. 4-5 Sample Design

31 climate region will help to ensure adequate precision in the estimation of various features of the associated segments. Stratification by consumption is useful to ensure that the businesses surveyed represent California businesses across the spectrum of size as well as to define the difference between large and small offices. The final stratification will allocate the sample points allocated to a given business type across IOUs, climate regions, and electricity consumption. The sample design discussed in this Research Plan will focus on the business type allocations. The relative precision estimates are based on the business type allocations of sample points. 4.3 Sample Design for the Telephone Survey The telephone survey will be used to collect information on the nonresidential frame included in the analysis and to recruit for the CMST and the CSS on-sites. Given that the primary purpose of the telephone survey is to recruit for the CMST and CSS on-site, the sample design focuses on insuring that an adequate sample is recruited for the on-site efforts. If the quotas for the on-site recruitment are exceeded prior to reaching the quota for a segment for the telephone survey, the telephone quota for the segment will be modified and dialing in the segment will be discontinued. This approach will save budget while ensuring that the needed number of sites are recruited for the on-site data collection efforts. Table 4-3 lists the business types included in the telephone survey, the business type eligibility for the CMST and CSS on-site surveys, the usage by business type and the proposed telephone survey sample quota. The sample quotas are tentatively set based on the usage for the business segment. Segments with higher levels of usage were given proportionately higher quotas. To insure adequate coverage of all business types, a minimum of 250 telephone survey completes will be attempted for all business types. To insure that no business type dominates the telephone survey, a maximum quota of 800 completes by business type was established. Itron, Inc. 4-6 Sample Design

32 Table 4-3: Telephone Survey Sample Design By Business Type CSS/CMST NAICS Business Type Eligible for CMST/CSS Total MWh Telephone Survey Sample College CMST 1,817, Food/Liquor CMST/CSS 6,225, Health Care Care CMST/CSS 865, Health Care Hospital CMST 2,988, Health Care - Med Office CMST/CSS 1,895, Hotel/Motels CMST 2,656, Industrial CMST 22,455, Misc CMST/CSS 9,331, Misc Assembly CMST/CSS 751, Misc - Res Common CMST/CSS 1,258, Office CMST/CSS 9,396, Property Managers CMST/? 7,860, Restaurant CMST/CSS 6,012, Retail CMST/CSS 7,731, School CMST/CSS 3,380, Unclassified and Undefined CMST/? 3,188, Warehouse CMST/CSS 4,061, Total 91,876,443 8,845 Given that the primary purpose of the telephone survey is to recruit for the CMST and CSS onsite surveys, the research team has reviewed the telephone survey allocation by business type and eligibility for the CSS and CMST surveys. 16 In Table 4-3, business types not eligible for the CSS on-site data collection are highlighted as CMST only sites. Property management and unclassified/undefined sites are listed as eligible for the CMST and questionable for the CSS. Their final eligibility for the CSS survey will depend on the telephone survey business type designation. 17 All business types included in Table 4-3 are tentatively eligible for the CMST onsite verification. The telephone survey sample design limits colleges to only 250 telephone survey completes. But it is likely that 250 telephone completes within the college segment is adequate given that colleges are not in the CSS survey and colleges may have a higher incidence of recent purchases than other business segments given the potentially higher levels of retrofit and new construction in this segment. Hospitals and hotels and motels also have a relatively smaller telephone survey 16 The data collected during the telephone survey will be analyzed and will provide crucial information on recent purchases, program awareness, and the distribution of lighting, HVAC, and refrigeration measures. The analysis of these data, however, is secondary to the collection and analysis of the on-site data. 17 The eligibility of property management and unclassified/undefined sites for the CMST on-site effort is also dependent on their telephone survey business type. If these sites are determined to be agricultural, mining, street lighting, or TCU sites, they will not be eligible for the CMST on-site verification effort. Itron, Inc. 4-7 Sample Design

33 quota and are segments not eligible for the CSS survey. These segments may also have a higher rate of retrofit than other business segments. 18 The telephone sample design allocates 800 sample points to the industrial segment. While this quota appears high for a segment that is only participating in the CMST on-sites, it may be necessary to complete more telephone surveys with the industrial sector to find sites who are both recent purchasers of high priority measures and willing to allow on-site verification of these measures. If, during the implementation of the telephone survey, it is found that the CMST is disproportionately recruiting industrial sites, the telephone survey quota for industrial sites will be reduced. Given that the initial telephone sample design is based on segment level usage, the unclassified and undefined segment was allocated a quota of 402 telephone survey completes while the property management segment was allocated 800. Both of these business segments will be redefined during the telephone survey process. Completes in these segments will be reallocated to the business activity that is undertaken at the site for the CMST and CSS on-site survey effort. It is necessary to keep these two segments separate for the telephone sample design, however, because the types of sites in the two segments may be substantially different. Reviewing the information provided in Table 4-1 and Table 4-2, sites designated as unclassified and undefined use approximately 17 MWh of electricity per site while sites designated as property management use approximately 117 MWh per site in An initial review of the telephone survey quotas for business segments included in CSS on-site data collection effort indicates that some sectors may require more quota than were initially allocated. For example, the CSS health care segment (nursing and skilled care facilities and medical offices), is allocated only 500 completes. If the telephone survey is unable to satisfy the necessary number of on-site recruitments within these 500 telephone survey completes, the number of telephone survey completes will need to be adjusted upward. A similar situation may arise for schools and offices. The office quota of 800 telephone survey completes will need to be allocated between large and small offices for the CSS on-site effort. The process of choosing size cut points for determining large versus small offices has not yet been completed. In addition, it is likely that many of the sites included in the property management and unclassified segments will be reclassified as office, retail and miscellaneous, providing additional sites for the on-site survey effort. 4.4 Sample Design for the CMST On-Site End-User Verification The CMST plan calls for the completion of 800 on-site verifications. The 800 sites will be distributed across sites that have recently purchased linear fluorescents, televisions, and 18 Information on the rate of purchases of linear fluorescents, televisions, and HVAC by building type are not currently available. These data will be available following this survey. Statements concerning potential rates of retrofit are based on professional judgment. Itron, Inc. 4-8 Sample Design

34 packaged HVAC. The objective of these on-sites is to collect information on the number of measures purchased and their efficiency level. The initial allocation of these sites calls for approximately 270 lighting and television sites and 260 HVAC sites. This allocation will enable the analysis to achieve a relative precision level of approximately 90/10 for all three measures if the split between high and base efficiency measures is 50/50 (the worst possible distribution for relative precision). The final allocation of sites, however, may differ from the original plan if the distribution of recent purchasers willing to participate in on-site verifications is highly skewed away from one of the three measures of interest. The sample plan for the CMST on-site verification does not specify the quota by business type. At the on-set of this analysis, it is unclear where recent purchasers of these measures will be found within the nonresidential frame. There is no tracking data to help guide the allocation as the research question involves the nonresidential population, not a participant population. The allocation of on-site verification sites across business types will be reviewed as sites are recruited from the telephone survey. If the recruited sites do not represent the sites reporting recent purchases, adjustments will be made to the on-site verification effort to insure that the verification sites represent the recent purchasers. The 800 on-sites planned for the CMST on-site verification effort will overlap with CSS on-site data collection efforts. Sites that are recruited for both CSS and CMST on-sites will count toward the fulfilling the quota for both the CSS and CMST on-site data collection efforts. Given the test telephone survey recruitment, it is possible that over half of the 800 CMST on-sites will also count as CSS on-sites. The overlap will reduce the total number of on-sites required and reduce the cost of the data collection efforts. 4.5 Sample Design for the CSS On-Site Surveys The initial CSS sample design call for approximately 2,000 surveys, conditional on budget allocations and the expected differential cost of surveys by business type. The allocation of sample points across business segments was designed to achieve relative precision levels of 90/10 to 90/15 on the estimation of measure saturations at the business segment for a measure with a 50% saturation. Table 4-4 lists the preliminary CSS on-site sample design. The sample design was allocated proportional to usage in the CSS segments with the smallest sample allocation restricted to 150 points and the largest allocation restricted to 250 points. The various health care and miscellaneous sub-segments were aggregated for the purposes of on-site sample quota allocation. The final distribution of sites across the sub-segments will be frequently reviewed. If too many sites from a given sub-segment agree to an on-site survey, the quota for the sub-segment will be modified. For example, experience has shown that miscellaneous businesses sub-classified as Itron, Inc. 4-9 Sample Design

35 assembly are more likely to agree to on-site data collection than other miscellaneous sites. If this is found to be true within the CSS on-site recruitment, miscellaneous assembly sites will be independently restricted. Within Table 4-4 the allocation of sites across large and small offices is not specified but the total number of sites allocated to offices has been expanded to 409 to account for the future disaggregation of offices. The estimated relative precision of offices is not listed due to the desire to split these sample points between small and large offices. Table 4-4: CSS On-Site Survey Sample Design By Business Type CSS/CMST NAICS Business Type Eligible for CMST/CSS Total MWh On-Site Survey Sample Estimated Relative Precision Food/Liquor CMST/CSS 6,225, Health Care Care and Med Offices CMST/CSS 2,761, Misc CMST/CSS 11,340, Office* CMST/CSS 9,396, Property Managers CMST/? 7,860,483 Restaurant CMST/CSS 6,012, Retail CMST/CSS 7,731, School CMST/CSS 3,380, Unclassified and Undefined CMST/? 3,188,752 Warehouse CMST/CSS 4,061, Total 91,876,443 1,886 *Offices will be future divided into large and small offices with the sample quota allocated between these two segments. Table 4-4 includes usage data associated with the property management and unclassified/undefined business types. Many of these business types may become eligible for the CSS survey after the telephone survey clarifies the business activity at the site. Given the usage of these two business types, it is possible that the reclassification of these sites may modify the CSS on-site sample design. The sample design will be further clarified as size groupings are determined across the nonresidential frame. The size groups will lead to a size stratification to ensure that the sites visited represent the distribution of building types by usage, IOU, and region. Given the preliminary nature of the current sample design, when the design is finalized, a memorandum will be delivered describing the finalized design. Itron, Inc Sample Design

36 4.6 CMST Supply-Side Survey The CMST supply-side actor survey will query lighting and HVAC contractors and/or distributors about the quantity of sales, the efficiency levels of recent sales, and the level of IOU program participation. These surveys will be high coordinated with other work orders that are also interviewing the same set of contractors and distributors. The initial plan calls for interviewing approximately 100 participant and 100 non-participant lighting and HVAC contractors. The HVAC survey will also interview available distributors, but the number of distributors is relatively small and the analysis does not expect to interview more than five ( 5) distributors. Itron, Inc Sample Design

37 5 Survey Development and Implementation As outlined above, the CSS and CMST will share a joint telephone survey followed by CMST on-site verification and CSS on-site data collection. The content of the on-site surveys for the CMST data verification effort will depend largely on the specific end use recently purchased by the sites. The firm demographic data collected by the CSS and the CMST during the on-site survey, however, will be identical. This methodology will provide the evaluation community with an extensive set of information on firm demographics. This approach will also conserve on survey development expense and simplify the training needed for data collection as the surveyors will only need to learn one on-site form. The CSS on-site form is more extensive, including all surveyed end uses, while the CMST form simply reduces the number of end uses surveyed. 5.1 Telephone Survey Development The joint telephone survey was designed to collect an assortment of information on approximately 8,000 nonresidential sites and to recruit for the CMST and CSS on-site data collection efforts. The survey s development began with the modification of the Small Commercial Evaluation telephone survey. The survey was modified to adjust to the research needs of the CMST/CSS including adding questions on recent purchases of televisions and modifying existing questions on the recent purchases of lighting and HVAC measures. Telephone Survey Coordination The survey development process included multiple reviews by the CPUC, ED consultants, and the IOU. The telephone survey was also sent to other work order leads to ensure that the survey captured population level information to satisfy their needs. The survey was sent to WO029 for general lighting review. WO028, the Advanced and Upstream Lighting Evaluation added a series of questions to identify sites that have recently purchased CFLs from retail outlets. Future coordination with WO028 may include simply providing the site contact information on recent CFL purchasers or coordinating the on-site data collection efforts between WO028 and WO024. The Business and Consumer Electronics Evaluation (WO034) reviewed the series of questions on televisions. The Overarching Nonresidential Process Evaluation (WO012) helped design the program awareness and integration questions. The Small Commercial HVAC Evaluation (WO032) and the Residential and Small Commercial HVAC Market Effects Study (WO054) reviewed the HVAC questions, adding questions on nonresidential maintenance practices. Itron, Inc. 5-1 Survey Development and Implementation

38 Telephone Survey Content Given that approximately 10% of nonresidential usage and 20% of nonresidential sites are classified in the CIS as either property management or unclassified/undefined, a primary objective of the telephone survey is to clarify the site s business type (see Table 4-1 and Table 4-2 in Section 4). The business type information from the telephone survey will be used to update the business type for the evaluation of the telephone survey data. In addition, the updated business type will be used to determine eligibility for recruitment into the CMST and CSS onsite data collection effort. See Section 4 for a full description of the business types eligible for the two on-site surveys. The telephone survey also collects a substantial quantity of information on a business s general characteristics. The survey questions the site contact about the number of buildings and the portion of buildings the firm occupies. In addition, the site contact is asked about the square footage and number of floors within the building that the firm occupies. This information is immediately used to estimate the likely number of hours needed for an on-site data collection effort. 19 This estimate is used during the on-site recruitment process. These data are also used during on-site scheduling to determine the likely number of sites a surveyor can complete during an assigned period. Finally, this information will be used during the analysis process to better describe the current space used by nonresidential customers. The telephone survey also collects information on the building s own versus lease status and who pays the utility bills. If the site pays the utility bills, the site is asked about how often they review their utility bills. Analysis of this information may be useful as utilities move to a system of smart meters and billing formats designed to modify consumers electricity usage. Sites are also asked about the age of the building and the length of time their business has occupied the building. The telephone survey also incorporates questions about increases or decreases in square footage and recent remodels. The answers to this series of questions are used to determine if the building was newly constructed, expanded, or remodeled since January These data are used to flag a site for the series of questions concerning recent purchases of linear fluorescents, HVACs and televisions. The telephone survey incorporates a battery of questions on customer attitudes toward energy consumption, environmental consciousness, and demand-side management. The survey inquires about the site s awareness of programs or resources to help their organization reduce their energy bills, about previous participation in energy efficiency programs that provided rebates or incentives, and about participation in demand response programs and whether the site currently 19 As the full implementation of the survey is undertaken, the relationship between size, configuration, and on-site survey time will be further analyzed and our current estimate of survey time will be updated in the telephone survey recruitment battery. Itron, Inc. 5-2 Survey Development and Implementation

39 has solar, wind, or other energy producing technologies. This series of questions asks whether the site has ever undertaken an integrated audit or if the site contact knows where to get an integrated audit. This series of questions will provide the demand-side management community with an estimate of the current baseline of knowledge about integrated demand-side management options. One of the primary purposes of the telephone survey is to recruit for the CMST on-site verification. To achieve this objective, the telephone survey includes a series of questions to determine if the site has linear fluorescents, televisions, and/or packaged HVAC. Sites are then asked if they have purchased any of these technologies since January Responses to these questions, in combination with the updated building type information and information on new construction and remodels, are used to determine eligibility and recruitment for the CMST onsites. To insure that we are opportunistically collecting information on current nonresidential purchase behavior, the survey also asks the site contact about other types of energy using equipment the firm purchased since January The responses to these questions will be reviewed and may lead to future modifications in the telephone or on-site survey efforts. The telephone survey finishes with a series of questions on business operating hours followed by the recruitment for the CMST and CSS on-site data collection efforts. Information on operating hours is used to schedule on-site data collection and will provide the demand-side management community with a better understanding of the current business hours of nonresidential firms. The recruitment script begins by asking sites that are eligible for the CSS, if they are willing to participate (for a description of sites eligible for the CSS see Section 4). If the site refuses to participate, they are offered a small monetary incentive to participate. If the site continues to refuse, and they are eligible for the CMST, they are asked to participate in the shorter CMST data collection effort. Sites that are eligible for the CMST, but not eligible for the CSS, are also recruited for the CMST effort. Sites that are eligible for the CMST effort, but refuse to initially participate, are also offered a small incentive to increase the participation rate. Sites that are eligible for the CSS and the CMST can choose to participate in only the CMST survey. If a site that is eligible for both surveys chooses to participate in the CSS survey, they will be counted as a site participating in both the CSS and the CMST surveys. 5.2 Telephone Survey Implementation The telephone survey will be implemented using the Itron CATI center. The CATI center has extensive experience interviewing commercial customers in California. The Itron CATI manager has carefully reviewed the telephone survey development with the project and task Itron, Inc. 5-3 Survey Development and Implementation

40 managers, ensuring that the wording of survey questions, the flow of the survey, and the skip patterns correctly represent the desires of the survey development process. The telephone survey has gone through extensive testing to actively review the flow and skip patterns in the survey and to estimate the time needed to complete the survey. The survey center manager and the Itron project managers will work together to develop a protocol associated with call times, methodology to ensure that the correct personnel is responding to the survey, and call back frequency. The protocols will help to limit the potential of non-response bias. The budget calls for approximately 8,000 telephone surveys. Initial testing of the telephone survey implementation began in November The test surveys are providing the team with information on the length of the survey, the share of completes per sample provided, the share of sites identified as recent purchasers or CMST sites, the share of sites willing to participate in a CSS or CMST on-site, and the overlap between the CSS and CMST on-sites. The surveys will also help to determine if sufficient numbers of end users have recently purchased high priority measures to make the end-user verification effort feasible. If the test telephone surveys find that additional surveys are likely needed to reach the desired quota for CSS and/or CMST on-sites, the team will need to determine the budgetary implications of the findings. The telephone surveys will be staged over a two-year period. 5.3 On-Site Survey Development The on-site survey was designed to collect an assortment of information on approximately 2,400 nonresidential sites. The survey was designed to serve the needs of both the nonresidential CMST survey and the commercial CSS survey. The on-site data collection instrument was derived from the former CEC CEUS forms. The forms were modified to incorporate new technologies and fields of interest, and selected end uses and technologies were removed from the survey. The on-site surveys will be site specific when provided to the on-site surveyors. The surveys will be pre-populated with firm demographic and recent purchase data gathered during the telephone survey. The forms will also contain pre-populated information on energy efficiency, demand response, and distributed generation program participation data. 20 Participants in the CMST and the CSS survey will provide information on firm demographics. Sites participating in the CMST will also provide data on recently purchased technologies. Sites participating in the CSS survey will provide a full set of technology information and information 20 The research team currently has the IOU energy efficiency and demand response tracking data. We have limited information on distributed generation program participation and are requesting additional information. Itron, Inc. 5-4 Survey Development and Implementation

41 on available distributed generation systems, EMS systems, and additional information on site characteristics. On-Site Survey Coordination The survey development process included multiple reviews by the CPUC, ED consultants, the DEER team, and the IOUs. The on-site survey was also sent to other work order leads to ensure that the survey captured population level information to satisfy their needs. The survey was sent to WO029 for general lighting review. The Business and Consumer Electronics Evaluation (WO034) reviewed and commented on the television and office equipment survey forms. The Small Commercial HVAC Evaluation (WO032) and the Re sidential and Small Commercial HVAC Market Effects Study (WO054) reviewed the HVAC forms, and provided useful suggestions that were incorporated into the final HVAC form designs. CMST On-Site Survey Content The CMST on-site verification effort is focused on verifying and updating the recent purchase information collected during the telephone survey. The data on a site s recent purchases, which was collected during the telephone survey, are pre-populated on the site s on-site survey form. If the site participated in a utility rebate program, this information is also pre-populated. The surveyor is instructed to review this information and discuss the information with the site contact as the survey is initiated. The measure specific information collected on these forms is described above in Section 2.1. The CMST survey form will appear similar to the CSS survey form, but the surveyor will only be provided the firm demographic forms and forms needed to record information on the technology recently purchased. For example, if the firm recently purchased linear fluorescents, the surveyor will be provided site-specific survey forms, pre-populated with responses from the telephone survey, for firm demographics and the general lighting forms. In this example, the surveyor would not be provided forms for televisions or HVAC. The surveyor will update the firm demographic information with data collected on site. The surveyor will also ask the site contact for the location of the recently purchased technologies and will record the required information on the technology. Collection of information on the make and model number of the technology will allow for the most precise classification of the measure. Surveyors are also required to collect an assortment of technology specific information that will help identify the technology. The site demographic forms include forms on the site s program participation (pre -populated), general site-level information that includes information on business type and square footage (prepopulated), a listing of the site s activity areas, a site-level sketch and a form for sites with multiple buildings. The firm demographics also include forms to collect data on the business and technology level primary and seasonal schedules. For the CMST survey, technology-level Itron, Inc. 5-5 Survey Development and Implementation

42 schedules are only required for the recently purchased technology. For pre-populated forms, the surveyors will review the provided information and update if necessary. CSS On-Site Survey Content The CSS on-site survey effort is designed to collect information on the saturation, quantity, type, and condition of pre-specified energy consuming end uses. The survey will collect information on lighting, HVAC, televisions, office equipment, and refrigeration. These end uses accounted for approximately 84% of the energy consumption in the former CEC CEUS. 21 The survey will also collect information on firm demographics (as describe above for the CMST), the utility account and meter numbers for both electricity and gas, information on shared energy services, data on on-site generation, descriptions of the shell construction, and data on EMS where applicable. The specific information collected on these forms is described above in Section 2.2. The CSS site level demographic forms are the same forms used for the CMST data verification effort. Employing the same forms will provide the energy efficiency community with an extensive dataset on firm level demographics and will simplify the training needed for the on-site surveyors. 5.4 On-Site Survey Implementation The on-site survey implementation process incorporates many steps. The on-site survey task manager will develop protocols and training materials and the field personnel will be trained in the implementation of the surveys. Following the training process, full implementation will gradually begin. During the phase in of the implementation, the data collected by the field personnel will be reviewed immediately following data collection; if issues arise they will be corrected prior to the personnel receiving additional sites. Develop On-Site Protocols This sub-task will entail the development of protocols for the collection of the on-site survey data. The protocols will help to guarantee consistency, accuracy, and efficiency in the collection of the on-site data. The 2006 CEC CEUS and the Small Commercial Evaluation Protocols provide the basis for the new CMST and CSS protocols. The protocols will be developed by the research team and reviewed by the interested parties. 21 The 84% of energy consumption does not include the energy consumption associated with TVs. Televisions would have been included as part of miscellaneous in the former CEUS and this share of energy was not included in the 84%. Therefore, it is possible that the share of energy consumption included in the CSS exceeds 84%. Itron, Inc. 5-6 Survey Development and Implementation

43 Develop On-Site Training Information The research team has extensive experience developing training manuals to provide guidance to the surveyors throughout the fieldwork. The existing training manuals for the CEC CEUS and the Small Commercial Evaluation will be reviewed, integrated, and customized to address the unique features of the CSS and CMST surveys. At a minimum, the manual will be updated to reflect the new CSS and CMST surveys. The manuals will also be adjusted to reflect the new material that the research team has found or developed that help explain the equipment or data collection issues better for these surveys. The training manual provides the foundation for the materials that are to be used in the training workshops. Material will also be prepared for the surveyors that explain the goals of the project and how the data collection efforts have been customized to achieve the project goals. Conduct Training for Surveyors There will be a minimum of two trainings sessions. The first training session was undertaken at Itron s San Diego offices prior to the implementation of the test CMST on-site data collection. The second training session will provide training for the more in-depth CSS on-site data collection. Implementing the data collection for the CMST prior to the CSS will serve as training for the CSS on those parts of the forms (firmographics, activity areas, and square footage) that overlap. CSMT/CSS Test Field Training The initial CMST/CSS training session was undertaken prior to the test implementation of the surveys in December The test implementation of the surveys focused on Itron survey personnel who have previously completed commercial surveys in California. Limiting the first training session to experienced personnel allowed the team to test the protocols, training information, and forms on survey personnel with the background experience needed to provide constructive input while successfully completing the test sites. The forms of the test training sessions followed the format that will be used for all future CSS/CMST trainings. The training incorporated an office based training workshop, field surveys implemented at actual sites, and an office based quality control session to review the collected data. The training began with a one-day training workshop, conducted in Itron s San Diego office for survey staff participating in the field data collection. The training workshop introduced the field staff to the CMST and CSS survey goals and survey forms. The workshop reviewed the training materials. The training instructors included the project manager, a senior engineer, and the field project manager. The project manager provided surveyors with an understanding of why each element of the data to be collected is important while the field manager carefully reviewed the forms and the protocols for filling in the forms. Itron, Inc. 5-7 Survey Development and Implementation

44 Following the in-office training, the surveyors went in groups to sites to implement the data collection procedure using actual sites. The CMST/CSS sites chosen for training were very small sites that incorporated the full spectrum of equipment needing to be verified under the CMST survey. Following data collection, the team spent the following day in Itron s San Diego office reviewing the data collected and working to ensure that all surveyors fully understand the forms, the data required, and the appropriate protocol for entering the data on the forms. After the test training sites, surveyors were given three to eight small sites to complete. These initial sites will be completed, the data quality checked, and any issues reviewed and resolved with the surveyor. Once the issues are reviewed and resolved, the forms will be updated and submitted for data entry. All of the initial sites must be completed to the satisfaction of the field staff manager and the field staff manager must be convinced that the surveyor has a grasp of the data necessary for the survey before the surveyor will be given additional sites. If a surveyor had QC issues with the first set of assigned sites, the extensive review will be repeated with their second set of assigned sites. If quality issues continue, the surveyor will be discontinued from the project. This extensive review and evaluation prior to sending the surveyor to numerous sites will help to ensure that the surveyor understands the data collection needs and how the form is to be completed. CMST/CSS Field Training The CMST/CSS training session will incorporate an introduction to the survey goals and the survey forms, visits to sites within multiple segments to collect field data, and a final day in the office to review and quality check the collected information. The session will start with a one- or two-day training workshop conducted in Itron s San Diego office for all survey staff prior to the commencement of the fieldwork. A similar training workshop may also be conducted out of Itron s Oakland office. The training workshop will be conducted using the training materials previously developed. The training materials will be updated to incorporate lessons learned from the initial test training session. The training instructor will be the senior engineer and the field project manager. The training session will address the overall project purpose and scope and the data collection protocols. The Itron field manager will provide the surveyors with an understanding of why each specific element of data to be collected is important. Understanding the importance of the data fields is critical to insuring that all fields are given sufficient care during the data collection effort. The survey personnel will also be given the procedures handbook and will be told to read the handbook and use it as a field reference. Following the in-office training, the surveyors will go in groups with the Itron field manager and the additional trainers to several sites to review the data collection procedure using actual sites. The sites chosen for training will cover a full spectrum of equipment and segments. Typical sites Itron, Inc. 5-8 Survey Development and Implementation

45 that might be included in the on-site training session include an office or retail building, a restaurant or grocery store, and a school. After the on-site data collection, the surveyors will return to the office for a one-day review of the data collected and to complete the survey forms. The review session will briefly review each surveyed site, discuss issues or problems encountered at the site, and analyze the data entered on the forms, insuring that the data are correctly entered and that data fields are not overlooked. The office review session will help the surveyors to learn from the experiences of others and correct any misconceptions that may have existed. After the test training sites, surveyors were given three to five small sites to complete. These initial sites will be completed, the data quality checked, and any issues reviewed and resolved with the surveyor. Once the issues are reviewed and resolved, the forms will be updated and submitted for data entry. All of the initial sites must be completed to the satisfaction of the field staff manager and the field staff manager must be convinced that the surveyor has a grasp of the data necessary for the survey before the surveyor will be given additional sites. If a surveyor had QC issues with the first set of assigned sites, the extensive review will be repeated with their second set of assigned sites. If quality issues continue, the surveyor will be discontinued from the project. This extensive review and evaluation prior to sending the surveyor to numerous sites will help to ensure that the surveyor understands the data collection needs and how the form is to be completed. Ongoing Quality Control Surveyors will be provided with forms representing four to eight CSS/CMST sites to be completed. The sites allocated to a surveyor will be tracked in the on-site tracking workbook. This workbook identifies which surveyor was assigned the site, if the survey has been completed, which office engineer is QCing the site, and if the site passed the quality checking process and is at data entry or if the site was returned to the surveyor for updating. Using the tracking workbook it will be possible to quickly determine the number of sites completed, the status of various sites, and who is currently responsible for the site s status. It will also be possible to identify surveyors with reoccurring quality control issues. Surveyors with reoccurring quality control issues will be given the training and oversight necessary to improve their performance. If the survey quality does not improve, these surveyors will be removed from the project. Surveyors will not be given additional sites unless their pre-assigned sites are completed and turned into the office engineers for quality control review. The on-site surveyors will be expected to turn in their forms on a semi-weekly to weekly basis. The office-based engineers will be reviewing the forms, either forwarding the forms for data entry or returning the forms to the surveyors with questions. Surveyors will be required to update their forms that need review on a weekly basis. Itron, Inc. 5-9 Survey Development and Implementation

46 The on-site data will be data entered following the quality control process. The data entry personnel will be instructed to report data that does not conform to the field specifications. These forms will be returned to the office engineers and potentially the surveyors for review and updating. The data in the on-site database will be analyzed on an ongoing basis. The analysis will also help with quality control. Queries will be established to calculate whole-building energy intensity, lighting power density, and HVAC tonnage per square foot for each site. Sites with measurements outside established ranges will be reviewed to ensure the site s data was correctly recorded on the form and entered into the on-site database. Itron, Inc Survey Development and Implementation

47 6 Research Methodology This section summarizes the overall research approach and methods for developing shares of purchases, saturations, efficiency information. The section also highlights segmentation variables that will be available to disaggregate results. A web site will be developed to present the survey results. The web site will allow the user to view the data at different levels of aggregation Telephone Survey The telephone survey will collect self-report information from approximately 8,000 nonresidential customers. This data collection effort will provide the demand-side management community with a unique opportunity to learn about the distribution of business, attitudes, awareness, and equipment. The presentation of results from the telephone surveys will be an ongoing process. The preliminary results from the telephone survey will be analyzed shortly following the test implementation of the survey. Banners or tables will be developed to present the distribution of responses for each question. In addition, some questions will be interacted with other information to produce more disaggregated results. The results from the test surveys will be reviewed to insure that the skip patterns have been correctly implemented and to check for question non-response, which could indicate the need to re-word certain questions. Given that the initial telephone test survey did not incorporate all eligible businesses and was not distributed proportional to the final sample design, care must be taken when interpreting the test telephone survey results. As the telephone survey progresses into full implementation, however, it will be possible to use the code developed for the test surveys to analyze the preliminary findings. Preliminary findings from the telephone survey will be presented in an interim memo in July The data presented in the web presentment tool will be sufficiently aggregated to ensure the confidentiality of customer data. The web presentment tool will present information on counts and shares. The tool will not incorporate any information that would allow for the identification of a site. The web presentment tool and the reports will be reviewed by the CPUC and IOUs prior to the public presentation of the tool and the data. Itron, Inc. 6-1 Research Methodology

48 Telephone Results As described in Section 5, the telephone survey will collect information on the activity undertaken at the site (upda ted business type), the square footage of the business, building and business age, and the number of buildings occupied by the business. The survey will also collect information on the own versus lease status of the building and recent increase or decreases in business square footage. These data will be analyzed to provide the demand-side management community with a better understanding of the distribution of businesses and their site characteristics. The variables will be analyzed by IOU and climate region. The analyses of these data will provide an updated understanding of the distribution of businesses, their age, and their building ownership status. A section of the telephone survey results will focus on customer self reported environmental consciousness and their awareness of demand-side management opportunities. The telephone survey will collect information on how important being environmentally conscious is to the business and the use of marketing material to highlight the business s environmental consciousness. The survey questions if the site has a long-term energy plan, if the business has developed a policy for the selection of energy using equipment, and if the business had personnel with responsibility for controlling energy usage and costs. The program awareness section of questions asks the site contact about their knowledge of programs designed to reduce energy bills. This line of questioning quickly evolves into questions concerning awareness of and participation in integrated audits. The questions query the contact about the existence of distributed generation and demand response at the site. If the site has undergone an audit, the survey questions the contact about whether the site received information about demand response and distributed generation during the audit. The environmental consciousness and awareness responses will be analyzed to determine the current baseline consciousness and awareness characteristics of the nonresidential population. Disaggregating these data by business type and IOU may provide IOU outreach/marketing departments with useful information. These data may also be used by the Overarching Nonresidential Process Evaluation (WO012). The telephone survey contains sections on lighting, televisions, and HVAC equipment. These sections were used to determine if a site had recently purchased linear fluorescents, television or packaged HVAC units, making them eligible for the CMST on-site verification survey. The telephone survey also includes questions on recent purchases of refrigeration equipment and a question that provides sites with the opportunity to list recent purchases of other energy using equipment. The responses to the recent purchase questions will also be analyzed to determine the share of the nonresidential population that self-reports purchasing these types of equipment. These data will provide a better understanding of the market for energy using equipments, the share of sites purchasing equipment and the distribution of end uses of equipment purchased. These data will also be disaggregated by business type and IOU. Itron, Inc. 6-2 Research Methodology

49 The lighting, television, and HVAC series of questions also ask customers about the types of equipment currently at the site and the age and condition of this equipment. This information will be analyzed to provide a broad picture of the equipment currently used in the nonresidential population. The telephone survey also questions lighting purchasers about who purchases their lighting and where their lighting is purchased. These questions are asked for general lighting and specifically for CFLs. The responses to the CFL questions will be provided to WO028, the Advanced and Upstream Lighting Evaluation. These questions will be analyzed to better understand the lighting purchasing behavior of nonresidential customers. Program Participation The IOU energy efficiency, demand response, and distributed generation program participation status will be known for all survey participants. The IOU tracking data will be merged with the CIS and customer billing information. Using this information, it will be possible to disaggregate the results of the telephone survey by participant status. For example, it will be possible to calculate the telephone survey respondent program participation share by business type and own versus lease. Of particular interest may be the distribution of responses to the series of questions on environmental consciousness and awareness of demand-side management opportunities by program participation status. Business Type The telephone survey will produce an updated distribution of customer business types. Using these data, it will be possible to describe the customers currently listed as property management and unclassified/undefined. This will provide a better understanding of the current distribution of nonresidential business. As listed above, it will also be possible to use the new business type classifications to disaggregate other telephone survey questions. Customer Size The CIS data used to develop the sample frame for these studies have also been merged with the customer billing information. Using these data, customer size categories will be developed. Using these data, it will be possible to disaggregate telephone survey questions by customer size categories. 6.2 On-Site Surveys The on-site surveys will collect verification data on recent purchases and extensive information on the saturation of energy consuming technologies. Both surveys provide the opportunity to collect information on make and model numbers that will be reviewed using lookup tables to Itron, Inc. 6-3 Research Methodology

50 determine the energy efficiency or nameplate energy characteristics of many technologies. This data collection effort provides the opportunity to update our understanding of the current distribution of purchases and existing equipment in the nonresidential and commercial populations. The presentation of results from the on-site surveys will be an ongoing process. The preliminary results from the initial on-site surveys will be analyzed shortly following the test implementation of the surveys. Tables will be developed to present the data collected for nearly every field. In addition, some questions will be crossed with other information to produce more disaggregated results. The results from the initial test surveys will be reviewed to insure that the surveys are collecting the desired information and that the surveyors are correctly implementing the survey. As the on-site surveys progress into full implementation, it will be possible to use the code developed during the analysis of the test on-site surveys to analyze preliminary findings. If sufficient surveys have been undertaken, preliminary findings from these surveys will be presented in an interim memo in July On-Site Results The extensive information to be gathered in the CSS and CMST surveys was highlighted in Sections 2.1 and 2.2. The results from these data collection efforts will be presented in multiple types of tables. These data will also be available in a web presentment tool that will allow the user to query the data to develop many types of disaggregation. Several different types of results that will be possible with the data are highlighted below. Additional data will also be available from these surveys. Saturations The data collected by the CSS survey will be used to calculate the saturation of particular technologies. For example, it will be possible to determine the share of business with T12 and alternative generations of T8 lighting. These saturations may then be disaggregated by IOU and/or business type. For example, the distribution of saturations of linear fluorescent lighting by business type will be calculated. Many technologies within the CSS are likely candidates for tables describing their saturations. Examples include, but are not limited to, linear fluorescents, medium screw based lighting, televisions and their connected loads, EMS systems, multiple types of HVAC equipment, refrigeration equipment, and office equipment. In addition, data collected on maintenance practices will be analyzed to present information on the saturation or share of sites actively maintaining their equipment. The saturation data will provide the IOUs, CPUC, and larger demand-side management community with a description of the current baseline of equipment present in commercial Itron, Inc. 6-4 Research Methodology

51 businesses. For some technologies, it will also be possible to compare the current saturations of equipment with results from the CEC CEUS. For example, the distribution of linear fluorescent lighting found during the CSS on-sites can be compared to the distribution of linear fluorescents found during the CEC CEUS. The comparison of equipment saturations over time will help to describe the progress made toward improving the efficiency of technologies. Efficiencies During the on-site data collection for the CSS and the CMST, the surveyors will be attempting to collect extensive make and model number information. On an ongoing basis, these data will be looked up to determine the nameplate efficiency of existing equipment. For the CMST verification survey, the efficiency level of the recently purchased equipment will be used to describe the baseline of recent purchases. This information will describe the distribution between base and high efficiency, where it may be possible to disaggregate high efficiency into multiple tiers. Using program participation data, it will also be possible to determine if the recently purchased high efficiency measures were purchased using an IOU rebate. The efficiency distributions found during the on-site verification will be compared to the efficiency distributions found during the supply-side surveys. For the CSS on-site survey, the efficiency level of existing equipment will be used to describe the baseline of many types of energy using equipment. For many older technologies, the nameplate may have eroded, limiting the ability to collect these data. In some of these situations, it is likely that the technologies are older and less efficient. In cases where make and model number are not available, the surveyors will collect information on make, condition, and the approximate age of the technology. This information will be combined with the collected information on make and model number to develop an understanding of the likely underlying efficiency distribution. Measure Vintages During the CSS on-site data collection, surveyors will be collecting information on the approximate age of many technologies. Technology specific age distributions will be presented. This information will provide the IOUs with a better understanding of technologies that are recently turned over and those with more potential for retrofit. These data may also be used to develop estimates of the expected useful life of many technologies. Operating Schedules and Business Hours The CSS and CMST on-site surveys will collect information on business and technology operating schedules and hours. These data will be presented in tables by business type and technology. The data will describe the current hours of operation for nonresidential businesses. Itron, Inc. 6-5 Research Methodology

52 The data will be compared to information collected from the CEC CEUS wherever possible. This comparison will indicate if businesses are operating on reduced schedules, perhaps due to recent macroeconomic events, when compared to the operating hours during the period. Square Footage and Energy Usage The CSS on-site survey participants will be paired with their electricity and gas consumption data. Account and meter level data collected on site will insure that the site aggregation process has aggregated the appropriate meters. Where errors occur, the site aggregation usage will be correct. Using the site level usage data, the research team will produce tables of average usage by business type and IOU. These data may also be further disaggregated by climate region. Using the usage information in combination with the square footage data collected on site, it will be possible to calculate whole building energy intensities for the CSS sample of sites. These data will provide the CPUC and the IOUs with information on the distribution of usage per square foot by business type. These data will be compared to similar data from the CEC CEUS. 6.3 Supply-Side Actor Surveys The CMST supply-side actors survey will interview lighting and HVAC contractors and/or distributors. Given the short nature of the CMST interviews, the team is hopeful that we will be able to incorporate our survey questions into surveys planned by other research and evaluation efforts. The CMST supply-side actor questions will query contractors and distributors on their linear fluorescent and packaged HVAC sales to California businesses. The contractors will be asked to provide information on the quantity of sales and the distribution of sales between base and high efficiency measures. The supply-side actors will also be asked about their participation in IOU rebate programs. The data collected from these surveys will be analyzed to estimate the total quantity of market sales and the distribution of sales between standard and high efficiency. The results of the supply-side actor surveys will be compared to the findings from the end-user verification surveys. Itron, Inc. 6-6 Research Methodology

53 7 Timeline This section presents a schedule for key tasks within the CMST and CSS research efforts. Ongoing activities such as early feedback, administrative progress reporting, and coordination with the CPUC, ED consultants, and the IOUs are cited. Itron, Inc. 7-1 Timeline

54 Table 7-1: CSS/CMST Timeline by Task Activity Start Date End Date Task 1: Research Project Management and Coordination Regular and ad-hoc coordination with ED, consultants, IOUs, and other work orders Mar-11 Ongoing Administrative reporting Mar-11 Ongoing Task 2: Research Plan Coordination for plan development with ED, its consultants, IOUs and other work orders. Mar-11 Feb-12 Draft evaluation plan submitted to ED contract manager Sep-11 Discuss research plan with ED, consultants, IOU, and other work orders Sep-11 Dec-11 Revised evaluation plan submitted to ED contract manager Review and comment on research plan by ED, consultants, IOU, and other work orders Public posting and review of evaluation plan Incorporating revisions into final plan for public release and final ED approval Task 3: Data Development Early January 2012 Dec-11 Late January 2012 Feb-12 Late February 2012 Mar-12 Site Development Jun-11 Oct-11 Business Type Sep-11 Dec-11 Meter matching Jan-12 Ongoing Task 4: Sample Design Develop draft sample design Sep-11 Feb-12 Update sample design due to business reclassification Jan-12 Ongoing Task 5: Telephone Survey Development and Implementation Task 5.1: Telephone Survey Draft Development Sep-11 Oct-11 Draft Survey submitted to ED contract manager Oct-11 ED, consultants, IOU and other work orders review phone survey Oct-11 Nov-11 Revise telephone survey due to comments Nov-11 Revise telephone survey due to test implementation and continued coordination with other WOs Dec-11 Jan-12 Task 5.2: Telephone Survey Implementation Test telephone survey implementation Nov-11 Jan-12 Full scale telephone survey implementation Feb-12 Ongoing Task 5.3: Telephone Survey Analysis Analysis of test survey findings Dec-11 Feb-12 Ongoing review of recruits and post coding Jan-12 Ongoing Preliminary analysis of telephone survey results Feb-12 Jun-12 Analysis of telephone survey findings Jul-12 Ongoing Itron, Inc. 7-2 Timeline

55 Table 7-1 (Cont d): CSS/CMST Timeline by Task Activity Start Date End Date Task 6: On-Site Survey Task 6.1: On-Site Survey Development Draft Survey forms submitted to ED contract manager Oct-11 Jan-12 Review on-site survey with ED, it consultants, IOU, and other work orders Oct-11 Jan-12 Revise on-site survey due to test implementation and continued coordination with other WOs Oct-11 Feb-12 Complete procedure manual for on-sites Task 6.2: On-Site Survey Training and Testing First Training Session Second Training Session Dec-11 Feb-12 Task 6.3: CMST Survey Implementation Dec-11 Ongoing Task 6.4: CSS Survey Implementation Dec-11 Ongoing Task 7: Development of Survey Databases Task 8: Supply-Side Actor CMST Survey Feb-12 Coordinate with other work orders to develop survey and sample frame Nov-11 Mar-12 Implement the supply-side actor survey Mar-12 Jul-12 Analyze the supply-side actor survey results Jul-12 Aug-12 Task 9: Analysis and Reporting CSS/CMST Preliminary memo of telephone survey findings Jul-12 CMST Preliminary analysis and memo on-site survey May-12 Jul-12 CSS Preliminary analysis and memo of on-site survey May-12 Jul-12 CSS/CMST Second Preliminary memo of telephone and on-site survey findings 2013 CSS/CMST Report of telephone and on-site survey findings 2014 Data delivered 2014 Task 10: Web Tool Develop draft web tool Finalize web tool 2014 For the CMST supply-side actor survey, there are many work orders planning on surveying similar actors. For the CMST survey, there are a limited number of questions that we will need to ask the supply-side actors. To improve efficiencies, we will try to incorporate our questions into the surveys implemented by other work orders. We would provide these work orders with financial resources, and analyze our questions within this study. Given the dependence of this task (Task 8) on other work orders, the timing of these tasks should be considered estimates. The schedule of this project is currently planned to incorporate 2013 and It is possible that the implementation of the survey, analysis of the data, and development of the report may not be completed until the end of In this situation, the web tool will be finalized in the beginning of The dates on these final tasks, however, should be considered estimates. Itron, Inc. 7-3 Timeline

56 8 Work Plan and Budget This section summarized the activities and budget for conducting the CSS and CMST research projects. 8.1 Task 1: Project Management The objective of this task is to provide all necessary coordination and communication across the project team, CPUC ED and their consultants, other related work order teams, and the IOUs. Itron will ensure that the CPUC ED, ED consultants, other work order teams, and the IOUs stay fully informed of the project status through of combination of telephone calls, monthly meetings and written project status reports. Monthly progress reports will be linked to the invoices showing percentage of work complete. Cross work order group meeting and meetings with the IOUs and CPUC ED will be scheduled as needed to promote the sharing of information and to leverage data collection and information relevant to multiple contract groups. 8.2 Task 2: Develop and Update the Research Plan This task includes the development and updates of this research plan and future updates to the plan. The development of the research plan has incorporated extensive information gathering on high priority needs from interested parties. These needs were incorporated into the draft research plan. The draft research plan was vetted internally at Itron and suggested changes incorporated. The draft plan was then reviewed by the CPUC. The comments from the CPUC were reviewed and incorporated into the plan s design. The draft research plan was also reviewed by the CEC, IOUs and other interested parties. Comments and suggestions received were reviewed and the plan updated. This process was continued until the finalized research plan was developed. Itron, Inc. 8-1 Work Plan and Budget

57 8.3 Task 3: Data Development: Site Development, Business Categories, and Final Meter Match-Up This task includes the development of data necessary for the research projects. Site Development The nonresidential site development is being largely undertaken as part of Data Management Work Order (WO009). The site development combines data from the IOUs CIS, billing data, and program tracking data associated with energy efficiency (EE), demand response (DR), and distributed generation (DG) programs. The nonresidential meter and account level data have been reviewed and nonresidential sites have been developed using electricity billing information. 23 Itron developed algorithms as part of the commercial evaluations for aggregating account and meter level information into commercial sites. The algorithms have been reviewed, refined, and nonresidential sites have been developed and reviewed under WO009. The original site development under WO009 is based on data from PG&E, SCE, and SDG&E s electric account information. Gas data will also need to be assembled for study participants. Combining the gas and electric data for SDG&E and PG&E dual fuel customers is relatively straight forward. Aligning the electric and gas accounts for customers who do not receive both fuels from a single utility, however, requires considerably more effort. This task will assemble the data needed to develop the gas and electric site information of customers receiving gas and electricity from two utilities for on-site participants in the CSS survey. Building Type Categories Under Task 3 of the CSS/CMST studies, the team reviewed with the DEER team, the evaluation teams, IOUs, and the CEC the desired building type categories to be used to develop the sample design. Itron collected NAICS code mappings from the CEC, IOUs, DEER, and evaluation teams. The NAICS code mappings were reviewed for consistency. The CSS/CMST team reviewed the mappings based on the purpose of these studies with the CPUC, ED consultants, and IOUs. A NAICS code to building type mapping was developed and applied to the CIS data. The CPUC, ED consultants, and IOUs also determined the NAICs code and building types to be included in the two studies. The building type mappings applied to the CIS data are listed in Section 4 of this plan. 23 The frame for the study is the IOU electric frame. Participation in the study will be offered to LADWP and SMUD. The gas usage for SCG customers will be matched to the electrical usage following participation in the study. Itron, Inc. 8-2 Work Plan and Budget

58 Final Meter Matchup During the site development process, accounts and meters were aggregated into nonresidential sites or premises. This process allowed the team to develop sample designs based on our best understanding of the site-level electricity consumption. Invariably, however, the site development process will not get all of the correct meters aggregated for a subset of sites. During the on-site survey process, the field team will collect meter and account information for both gas and electric meters. The meter numbers will be reviewed with the information available for the aggregated sites. This process is likely to lead to the incorporation of additional meters. Prior to eliminating meters from the site a careful review will be undertaken to ensure that the field personnel did not simply miss meters while visiting the site. The final meter matchup is crucial for the development of whole building energy intensities. 8.4 Task 4: Sample Design The purpose of this task is to work with the CPUC, ED consultants, and IOUs, to develop an efficient statistical sample design and to select the customer sites to be included in the sample. The proposed sample designs for the CMST and CSS are documented in Section 4 of this plan. The sample designs will need to be updated as new information on business type is developed from the telephone survey. This task also incorporates the process of pulling sample for the CATI telephone survey. The process of pulling sample is ongoing. During full-scale implementation of the telephone survey, the CATI pulls will be infrequent, but there will be multiple sample pulls that will be needed. The task includes the production of a memorandum describing the sample design. 8.5 Task 5: Telephone Survey Development and Implementation This task is comprised of developing and implementing the telephone survey. The telephone survey development and implementation is discussed in Section 5 of this plan. The telephone survey analysis is discussed in Section 6 of this plan. Task 5.1: Survey Development The development of the telephone survey was an interactive process involving the study team and the CPUC, the DEER team, ED consultants, other evaluation teams, and the IOUs. Minor modifications, if needed, will be approved and documented with the CPUC and ED consultants. Task 5.2: Survey Implementation The telephone survey will be implemented using the Itron CATI center. Itron, Inc. 8-3 Work Plan and Budget

59 Task 5.3: Telephone Survey Analysis Analysis of the telephone survey data will be staged to allow for the testing of the telephone survey form, the development of an early feedback memo, and a final report. 8.6 Task 6: On-Site Survey Development The on-site survey instruments are designed to serve the specific objectives of the data collection effort. The on-site survey form is discussed extensively in Section 2 of this plan. The survey development and implementation is discussed in Section 5 of this plan. Task 6.1: Development the CMST and CSS Survey Instruments In this sub-task, the research team will work with the CPUC, ED consultants, IOUs, the DEER team, and other work order teams to create a set of survey instruments for the CSS and the CMST surveys. The team will develop a single set of forms. For sites choosing to participate in the CSS survey, the on-site will consist of all forms needed to collect end use information pertinent to the survey and available on-site. For sites participating in the CMST verification effort, the forms will be restricted to those needed to collect firm demographic information and to verify the recent purchase. The development of these forms is ongoing, with special attention needed to the refrigeration forms. Minor modifications are possible for the other forms. As updates are made, they will be approved and documented with the CPUC and ED consultants. Task 6.2: On-Site Survey Training and Testing This sub-task will incorporate a minimum of two training sessions. The training will incorporate an office based training workshop, field surveys implemented at actual sites, and an office based quality control session to review the collected data. The training is discussed in Section 5 of this plan. Task 6.3: CMST On-Site Survey Implementation Following the CMST/CSS training and testing, the on-site personnel will begin to implement the survey. This task will cover the time to implement and quality check the CMST survey data. The task will also incorporate the lookups needed to determine nameplate efficiency information. Task 6.4: CSS On-Site Survey Implementation Following the CMST/CSS training and testing, the on-site personnel will begin to implement the survey. This task will cover the time to implement and quality check the CSS survey data. The task will also incorporate the lookups needed to determine nameplate efficiency information. Itron, Inc. 8-4 Work Plan and Budget

60 Task 6.5: On-Site Survey Expenses The implementation of the CMST and CSS surveys will lead to the accumulation of expenses associated with survey implementation. This task will fund these expenses. Expenses covered by this task include travel, lodging, and meals. During the implementation of the CMST and the CSS surveys, the surveyors will be provided sites distributed to reduce the required travel and lodging expenses for these research projects. 8.7 Task 7: Develop the Survey Database The development of the survey database will begin with the database used for the Small Commercial Evaluation. The database would be altered to accommodate the data needs of the CSS and CMST surveys. The CSS/CMST database will hold the site level data collected from the telephone surveys and the site visits. Incorporating the telephone survey data into the database allows the database to be designed to print each site s on-site forms with the telephone survey data pre-populating several fields on the survey form. 8.8 Task 8: CMST Supply-Side Actor Survey This task will involve the development of an interview guide, implementation of the supply-side actor surveys, and an analysis of the results from these surveys. 8.9 Task 9: Analysis and Reporting This task involves the analysis of the data collected during the on-site surveys. This process is discussed in Section 6 of this plan. In general, the analysis will create tables of statistics representing the on-site data collection process. Interim results for both the telephone and on-site surveys will be provided. The interim results will be presented in a brief, high-level report, focusing on the survey results. The final report covering the entire data collection and analysis process will incorporate information on the study design, data development, methods, and results. A draft report will be provided to the CPUC and the ED consultants to review and comment on. Itron will incorporate any necessary changes and finalize the report Task 10: Web Presentation Tool This task involves the development of a tool to present the research output on the web. The tool will be designed to present certain tables in a static mode while other tables will be dynamic. The final design of the tool will depend significantly on the distribution of the data collected. Itron, Inc. 8-5 Work Plan and Budget

61 A draft presentation of the tool will be undertaken with the CPUC, ED consultants, and the IOUs. Comments provided by the research group will be incorporated where needed into the final web tool Budget The cost estimate for the joint CSS/CMST studies is $7.5 million. A task-level breakdown of the project budget is presented below. Table 8-1: Budget by Tasks Task Budget Task 0: Work Order Development $27,500 Task 1: Project Management $88,500 Task 2: Research Plan Development $150,000 Task 3: Data Development $175,000 Task 4: Sample Design $80,000 Task 5: Telephone survey Task 5.1: Telephone Survey Development $50,000 Task 5.2: Telephone Survey Implementation $800,000 Task 5.3: Telephone Survey Analysis $84,000 Task 6: On-Site Survey Task 6.1: On-Site Survey Development $150,000 Task 6.2: On-Site Training $180,000 Task 6.3: CMST Survey Implementation Task 6.4: CSS Survey Implementation Task 6.5: On-Site Expenses CSS/CMST Survey Implementation $4,930,000 Task 7: Develop Survey Databases $80,000 Task 8: CMST Supply-Side Actor Survey $185,000 Task 9: Analysis and Reporting $320,000 Task 10: Web Presentation $150,000 Contingency $50,000 Total $7,500,000 It is noteworthy that 66% of the cost is for on-site data collection. If the cost of telephone survey data collection and the supply-side actor survey are added to the on-site data collection costs, approximately 79% of the costs are in Tasks 5.2, Tasks 6.3 to 6.5, and Task 8. Itron, Inc. 8-6 Work Plan and Budget