PERFORMANCE MEASURES

Transcription

1 RESEARCH AND RECOMMENDATIONS for PERFORMANCE MEASURES For Regulated, Industry-led, End-of-life Electronics Recycling Programs In Canada Commissioned By: Atlantic Canada Electronics Stewardship (ACES) Electronics Stewardship Association of British Columbia (ESABC) Ontario Electronic Stewardship (OES) Saskatchewan Waste Electronic Equipment Program (SWEEP) Prepared By: InterGroup Consultants Ltd Smith Street Winnipeg, MB R3C 1K2

2

3 executive SUMMARY Atlantic Canada Electronics Stewardship (ACES); Electronics Stewardship Association of British Columbia (ESABC), Ontario Electronic Stewardship (OES) and Saskatchewan Waste Electronic Equipment Program (SWEEP) are non-profit industry-led end-of-life electronics stewardship programs. The programs use an environmental handling fee (EHF) to provide program revenue for the collection, transportation, responsible recycling and administration of regulated electronics in their jurisdictions. The four programs jointly commissioned a study to analyze and make recommendations for a core suite of performance indicators. The primary purposes of the core indicators will be to: allow each program to track its own performance over time; facilitate comparisons and benchmarking between jurisdictions; and communicate performance accomplishments and targets to government and other stakeholders. Guiding principles for performance measures were established in order to assist with the selection of a core suite of performance indicators. The guiding principles used for this study are: Representative of performance: The indicator should convey something meaningful about the program s performance. It should be responsive to change and within the program s capacity to influence over time. It should also be reflective of the key policy reasons for implementing the programs. Easily communicated to and understood by stakeholders: The indicator should be intuitive to understand and easily communicated to stakeholders. Data accessibility and reliability: The indicator should be based on data that is feasible for the program to collect, maintain and report with accuracy. Where estimates are used they should be clearly stated. Ideally data could be independently surveyed and verified by a third party. Estimates produced through modelling techniques cannot be independently reviewed and verified and therefore should not be used. Cost effectiveness: The indicator should be cost effective to collect and report. Comparability across programs: The indicator should facilitate comparisons across programs and with other jurisdictions. The Research Team completed a review of performance metrics used by other electronics recycling programs in Canada, the United States and Europe. Based on this review, the Research Team developed a list of potential performance indicators. These were reviewed with the Study Advisory Committee at a workshop in November, 2009 and a suite of core indicators was selected. The Research Team and the Study Advisory Committee also considered data collection protocols related to each of the performance indicators. Table 1 summarizes the 13 recommended performance measures.

4 It is anticipated that nine of these performance metrics can be implemented based on data which is currently being collected. Three indicators (total weight of material recycled as a percentage of material collected, mass balancing of material recovery and trends in processing) are recommended for implementation by 2012, once certain data collection protocols have been established. One other indicator (greenhouse gas emissions) is recommended for further investigation and consideration over a two year period for potential future implementation. While it is understood each program s regulator may require additional performance measure reporting, this suite of indicators has been developed in order to provide a core set of measures that can help evaluate program performance and set future targets for performance. The Research Team recommends revisiting the suite of performance indicators three years from the time of initial implementation, in order to ensure the performance indicators continue to be relevant and meaningful. Table 1 Recommended Performance Indicators Operational Indicators Total WEEE Collected (tonnes) Total WEEE collected per capita (tonnes) Financial Indicators Total program costs per tonne Operational costs per tonne Overhead costs per tonne Accessibility Indicators Per cent of population covered by collection sites Total collection sites Total collection events Awareness Indicators Percentage of population aware of the programs Environmental Impact Indicators Total weight of material recycled as percentage of material collected Greenhouse gas emissions Mass balancing Trends in processing

5 TABLE OF CONTENTS Chapter 1 Introduction Background and Objectives Methods in Brief Cautions and Limitations Document Structure Chapter 2 current programs ACES ESABC OES SWEEP Stewardship Plans and Performance Reporting Chapter 3 Performance Measures Guiding Principles Chapter 4 Case Study Key Findings Programs Examined General Characteristics Indicators Financial Operational Awareness Accessibility Environmental Impact Audience Chapter 5 Evaluation of Options Review of Performance Measure Options Chapter 6 Summary of Findings and Recommendations LIST OF APPENDICES Appendix 1 Case Study Summaries... A1-1 Appendix 2 Key Contacts... A2-1 Appendix 3 Summary of Weee Directive... A3-1 Appendix 4 Bibliography... A4-1 Appendix 5 Glossary... a5-1 LIST OF TABLES Table 2.1 Performance Measurement and Reporting Commitments in Nova Scotia, Ontario, Saskatchewan and British Columbia 2-3 Table 4.1 Programs Examined 4-1 Table 5.1 Summary of Recommended Performance Measures 5-2 Table 5.2 Summary of Data Collection and Reporting Protocols 5-6 Page i

6 chapter 1: Introduction 1.1 Background and Objectives Canada s electronics industries and retailers have spearheaded the development of industry-led and managed end-of-life electronics stewardship programs in response to provincial regulations. Currently, four industry-led nonprofit programs are operational in Canada: Atlantic Canada Electronics Stewardship (ACES) in Nova Scotia, Electronic Stewardship Association of British Columbia (ESABC) in British Columbia, Ontario Electronic Stewardship (OES) in Ontario and Saskatchewan Waste Electronic Equipment Program (SWEEP) in Saskatchewan. The programs use an Environmental Handling Fee (EHF) to provide program revenue for the collection, transportation, responsible recycling and administration of regulated electronics in their jurisdictions. Each program has developed stewardship plans which are reviewed and approved by the provincial Ministry of the Environment. The approved stewardship plans form the commitment of the program to deliver services in compliance with the provisions of the plans and the regulation. Each stewardship plan includes provisions for performance reporting. While the stewardship plans and regulations are different in each jurisdiction, the programs recognize there are advantages to a coordinated and harmonized approach to performance reporting where feasible and appropriate. The objective of this study is to analyze and make recommendations for a core suite of performance indicators for the programs to adopt. The primary purposes of the core indicators will be to: Allow each program to track its own performance over time; Facilitate comparisons and benchmarking between jurisdictions; and Communicate performance accomplishments and targets to government and other stakeholders. The study was carried out by InterGroup Consultants Ltd (the Research Team). Overall leadership and guidance for the study was provided by the Study Advisory Committee which consisted of representatives from the four commissioning programs as follows: Executive Directors for ACES; ESABC; OES and SWEEP; Resource Recovery Fund Board (RRFB) Program manager for ACES; Encorp Pacific (Canada) Program manager for ESABC; StewardEdge Program manager for OES; Product Care Association (PCA) Program manager for SWEEP; SARCAN Recycling Service Provider for SWEEP; and Page 1-1

7 Harmonization office ACES/ESABC/SWEEP. 1.4 Document Structure This Committee was involved in all facets of the study including determination of data requirements and identification of data sources, data gathering, contributing knowledge from experience, and providing feedback on ideas generated by the Research Team. The Research Team interacted extensively with the Study Advisory Committee through conference calls, a workshop held in Winnipeg in November 2009 and review of the draft study report. 1.2 Methods in Brief The study began by establishing a set of guiding principles to use to evaluate different options for performance indicators. The Research Team completed a review of performance metrics used by other electronics recycling programs in Canada, the United States and Europe. Based on this review, the Research Team developed a list of potential performance indicators. These indicators were evaluated against the guiding principles and a suite of core indicators was developed. 1.3 Cautions and Limitations Each of the programs operates in a different legislative and policy environment. The provincial regulator may require additional performance reporting beyond what is recommended as a suite of core indicators. It is also recognized that the programs have been operating for a relatively short time. Data required for some of the recommended measures will need to be collected and reviewed before meaningful reporting can be undertaken. Therefore, the report recommends certain measures that can be implemented immediately, while other measures should be implemented when feasible. The report is organized into the following chapters: Chapter 2 Current Programs: This chapter provides a brief description of the four programs and their stewardship plans. Chapter 3 Performance Measures Guiding Principles: This chapter reviews the guiding principles developed to evaluate potential performance metrics. Chapter 4 Key Findings from Case Studies: This chapter provides an overview of the case studies from other jurisdictions and summarizes key findings from the research. Chapter 5 Evaluation of Options: This chapter reviews potential performance metrics and provides discussion on how each of them compares to the Guiding Principles. Chapter 6 Summary of Findings and Recommendations: This chapter provides a summary of the research findings, makes recommendations for a suite of core performance indicators and identifies research topics that should be investigated in the future. Appendices: Case study summaries are provided as well as a bibliography of sources consulted during the research, a list of key contacts, a summary of the European Union s WEEE Directive and a study glossary. Page 1-2

8 chapter 2: current programs This report addresses performance reporting for four regulated, not-for-profit, industry-led, end-of-life electronics recycling programs. A brief overview of each of the programs is provided below. 2.1 ACES The ACES program is an industry-led non-profit electronics recycling program regulated by the Province of Nova Scotia. ACES provides service to all residents and businesses in Nova Scotia where they can leave their end-of-life electronics without charge and with the assurance that these unwanted items will be responsibly recycled. The Program began February 1, 2008 and has been accepting Phase I items since its launch (including computers (desktop and portable), monitors, printers and televisions). Effective February 1, 2009 the program has also been accepting Phase II items (including personal or portable audio/video systems, home audio/video systems, aftermarket vehicle audio/video systems, home theatre in a box systems and non-cellular telephones). The Resource Recovery Fund Board Inc. was selected to develop, implement and administer this stewardship program. Program materials picked up from a network of 35 drop-off centres are either transported to a temporary warehousing facility where they are stored and sorted as needed and then sent for recycling or transported directly to a recycler. 2.2 esabc Electronics Stewardship Association of British Columbia (ESABC) is a member based, not-for-profit organization responsible to their membership for implementing and operating an environmentally and socially responsible program for handling electronic waste in accordance with ESABC standards and the requirements of the ESABC Stewardship Plan. The organization was established in November 2006, with the current stewardship program for Phase I products operational since August ESABC s electronics recycling program is delivered by Encorp Pacific under the brand Return-It Electronics. As of December 2009 there were 99 electronics collection sites distributed throughout the province. 2.3 OES Ontario Electronic Stewardship (OES) is a non-profit corporation established by manufacturers, retailers, and other stakeholders to develop a waste diversion plan for Waste Electrical and Electronic Equipment (WEEE) in response to the designation of these materials under the Ontario Waste Diversion Act (WDA). The Phase 1 Plan addresses desktop and portable computers, computer peripherals, monitors, printers, fax machines and televisions, and came into effect on April 1, The approved Revised (Phase 1 and 2) Plan addresses additional materials such as phones, cameras and audiovisual equipment, and will commence on April 1, Page 2-1

9 2.4 sweep SWEEP is the non-profit corporation that has been established for the purpose of coordinating the collection and recycling of obsolete electronic equipment in Saskatchewan. SWEEP s mission is to ensure a sustainable electronic recycling program that promotes responsible options for reuse and disposal of end-of-life electronics. The program began operating in February 2007 with the collection of Phase I products. Product Care Association (PCA) is tasked with managing the program. SARCAN coordinates the disassembly and recycling of the equipment and its components. Residents and businesses are encouraged to recycle their equipment by dropping it off at any SARCAN depot. As of December 2009 there were approximately 70 locations across the province. SARCAN provides the currect collection network for SWEEP. 2.5 stewardship plans and performance reporting Electronics recycling in each of the jurisdictions is governed by provincial legislation. Stewardship plans for the respective provincial programs have been developed in response to relevant provincial regulations. The stewardship plans set out each program s commitments for performance measurement and reporting. Table 2.1 summarizes the current performance measurement and reporting commitments in each program s stewardship plan. Page 2-2

10 Table 2.1: Performance Measurement and Reporting Commitments in Nova Scotia, British Columbia, Ontario and Saskatchewan Province Nova Scotia Governing Legislation and Regulation Environment Act, Solid Waste-Resource Management Regulations Performance Measurement and Reporting Commitments Section 8 from the ACES Six-Month report to July 31, 2008 submitted to Nova Scotia Environment committed to providing the following operational key performance indicators (it was noted that some of this information was business sensitive): Total volumes collected (tonnes); Collection costs per tonne; Interim storage, supplies and cartage costs per tonne; Total volumes shipped for recycling (tonnes); and Freight costs (to recycler) per tonne. Commitments were also made to provide communication and public awareness key performance indicators, as well as to regularly access public opinion survey tools to gauge Nova Scotian s knowledge about ACES and its program approach. Section 9.3 from the ACES Proposal for an End-of-Life Electronics Stewardship Program for Nova Scotia as amended to address Phase II Products submitted to Nova Scotia Environment in December 2008 noted that a system of management reporting will be developed within the first year of the program based upon key performance indicators to include, but not be limited to: Total volumes collected (tonnes and units); Total system costs per tonne and per unit; Administrative costs per tonne; Collection costs per tonne; Processing costs per tonne; Mass balance of materials (i.e., equating tonnes processed to number of EOLE product units collected) will be employed as part of mapping the final disposition routes for all obligated materials; Total volumes recycled (tonnes and percentage of total collected); and Total volumes disposed (tonnes and percentage of total collected). Page 2-

11 Table 2.1: Performance Measurement and Reporting Commitments in Nova Scotia, British Columbia, Ontario and Saskatchewan Province British Columbia Governing Legislation and Regulation Environmental Management Act Recycling Regulation Performance Measurement and Reporting Commitments Section 6 from the British Columbia Stewardship Plan for End-of-Life Electronics (October 2006) noted: The regulation calls for an eventual target of 75% recovery rate (or other as set by the Director). The Stewardship Plan outlined the challenges associated with meaningful measure of program performance related to end-of-life electronics; ESABC committed to two principles for performance measurement that are absent in all current North American programs: - Assumed that the existing stockpile of historic waste is sufficient to ensure that the program will show increases in volumes of obligated materials collected in the first few years. - Improvements in this phase will concentrate on reducing the volumes of non-obligated material entering the system thus reducing the overall costs of the program. ESABC also committed to conducting a world-wide study of the metrics used in EOL programs to determine those appropriate for this type of program and those best suited for British Columbia; In addition, the Program committed to measure its collection performance in comparison to other jurisdictions with similar programs such as Alberta and Saskatchewan; The performance of due diligence in ensuring proper handling and processing of collected materials will also be useful as a measurement of success; and The diversion of materials from land fill and the consequent removal of toxic substances from the environment will be measured and reported by weight. Page 2-

12 Table 2.1: Performance Measurement and Reporting Commitments in Nova Scotia, British Columbia, Ontario and Saskatchewan Province Ontario Governing Legislation and Regulation Waste Diversion Act Performance Measurement and Reporting Commitments Section 5.1 OES Final Revised (Phase 1 and 2) Waste Electrical and Electronic Equipment (WEEE) Program Plan - July 10, 2009 committed that at a minimum OES will measure and report the following information to WDO on an annual basis: Total WEEE collected (reported in tonnes and units) through the program including quantities from various types of collection locations (e.g. municipal, steward, retailer, reuse and refurbishment, etc) and by type of collection (e.g. event based, permanent collection site, etc.), and compared to collection targets; Total WEEE collected and processed (reported in tonnes, units, kilograms per capita and percentage of material available for collection) Total WEEE collected and processed via the approved direct shipment option (reported in tonnes, units, kilograms per capita and percentage of material available for collection); Total WEEE managed and processed under the Steward self-management option (reported in tonnes, units, kilograms per capita and percentage of material available for collection); Total collection, transportation, consolidation and processing costs per tonne by material category, and compared to budget. The mass-balance of materials collected and directed to processors will be employed to map the final allocation routes for all obligated materials; The number of OES-Approved collection sites and events, plus information on approved reuse and refurbishment sites, and compared to accessibility targets; Total volume (tonnes and percentage of total collected) of material managed through reuse and refurbishment; The number of whole units that OES-approved reuse and refurbishment organizations reported as being redistributed, and compared to reuse targets; Total volume recycled (tonnes, percentage of total collected and percentage of available for collection), and compared to recycling targets; Total volume disposed (tonnes, percentage of total collected and percentage of available for collection); Designing for Environment initiatives, as outlined in Section and as per the 2009 EPSC Designing for Environment Report located in Appendix 9; If available, new information on product weights, life spans, and portion reused, to populate the Discard Model; and An evaluation of the communication and public awareness tools, and effectiveness at communicating to the public (as outlined in Section 7.0). Page 2-5

13 Table 2.1: Performance Measurement and Reporting Commitments in Nova Scotia, British Columbia, Ontario and Saskatchewan Province Saskatchewan Governing Legislation and Regulation The Environmental Management and Protection Act Waste Electronic Equipment Regulations Performance Measurement and Reporting Commitments Section 8.3 of the SWEEP Stewardship Plan (February 2006) committed as follows: A system of management reporting will be developed within the first year of the program based upon key performance indicators. These key performance indicators will include, but not be limited to: System cost per tonne and per unit; Administrative costs per tonne; Quantity of obligated material collected; and Processing cost per tonne. A mass balance approach will be taken to outlining the final disposition routes for all obligated materials. Page 2-6

14 CHAPTER 3: PERFORMANCE MEASURES GUIDING PRINCIPLES There are a variety of possible performance indicators that could be selected for the programs. In order to develop a preferred set of core indicators, it is necessary to have a series of guiding principles to evaluate different performance indicator options. As an initial step in the research, a brief literature review was undertaken to provide context for developing the guiding principles. Based on the literature review, the Study Team developed a draft set of guiding principles to evaluate the effectiveness of potential performance indicators. The draft principles were reviewed with the Study Advisory Committee and are outlined below: Representative of performance: The indicator should convey something meaningful about the program s performance. It should be responsive to change and within the program s capacity to influence over time. It should also be reflective of the key policy reasons for implementing the programs. Easily communicated to and understood by stakeholders: The indicator should be intuitive to understand and easily communicated to stakeholders. Data accessibility and reliability: The indicator should be based on data that is feasible for the program to collect, maintain and report with accuracy. Where estimates are used they should be clearly stated. Ideally data could be independently surveyed and verified by a third party. Estimates produced through modelling techniques cannot be independently reviewed and verified and therefore should not be used. Cost effectiveness: The indicator should be cost effective to collect and report. Comparability across programs: The indicator should facilitate comparisons across programs and with other jurisdictions. It is understood that not all performance indicators will satisfy all the guiding principles equally. The guiding principles are used in this report to develop a suite of core indicators that each of the four programs can implement. It is recognized that each program may have additional specific performance measures required by their regulators. Page 3-1

15 CHAPTER 4: case study key findings 4.1 Programs Examined Case studies were selected for this research based on several criteria, including: Geographic location It was considered advantageous to review a selection of programs located within Canada, the United States and Europe. Program duration Programs that have been in operation for a longer period of time were assumed to have better established data collection methods and reporting mechanisms. Reputation Programs were selected for being at the leading edge of electronics recycling in their respective jurisdictions. Selection of these programs in particular was undertaken through assessment of relevant literature and communications with key industry personnel. A total of 13 program case studies were completed. Data collection was also undertaken regarding the WEEE Directive. Case study summaries for these programs are included in Appendix 1. These summaries were completed by reviewing available literature (including program stewardship plans, annual reports, business plans, pertinent legislation, academic research and other relevant materials) and telephone calls to key program and industry personnel. A summary of these key contacts is provided in Appendix 2. Table 4.1: Programs Examined CANADA ACES (Nova Scotia) ESABC (British Columbia) OES (Ontario) SWEEP (Saskatchewan) ARMA (Alberta) UNITED STATES California Maine Minnesota Oregon Washington EUROPE El Kretsen (Sweden) Recupel (Belgium) SWICO (Switzerland) WEEE Directive Page 4-1

16 4.2 General Characteristics The programs reviewed during the case studies varied greatly, not only in their selection and use of performance measures but in general program characteristics. It is important to note that different program characteristics can influence the selection and reporting of performance measures Financing Model Where producers/manufacturers bear the direct responsibility for all aspects of the their products recycling, performance measure reporting was typically less substantial compared to those that use a model based on a not-for-profit program financed through a visible environmental handling fee (EHF) or advance recycling fee (ARF) on stewards products. Of the 13 programs considered in this research, seven of them utilized an EHF or ARF (applied to the consumer or product steward). The remaining six programs were administered and paid for by producers and manufacturers. The suite of performance measures used by the latter was relatively limited, particularly the performance measures that are financial in nature Product Categories The programs also differed greatly in the scope of products accepted for recycling. This was largely related to how long programs had been in place with longstanding programs tending to accept the widest range of products. A relatively small number of product categories were observed in the U.S. programs. Canadian programs, particularly those implementing a Phase II set of product categories were much more inclusive than those in the U.S. The European programs, particularly those identified in this research, have been in place for a longer period of time (Switzerland s program has been operational since 1994) and the regulatory context in which they operate has evolved over several years. These programs were more likely to have expansive programs in place that include large appliances, lighting fixtures and other equipment (see Appendix 3). A key reason for this is that the WEEE Directive gave European countries the authority and requirements to initiate programs that were very broad in scope. In North America, authority is at provincial and state levels, allowing programs to start small and expand their product list over time. Many of the European programs are also split along product lines, with separate entities covering different products within the same country. 4.3 Indicators The range of performance measures identified and examined during the course of the research could be divided broadly into five categories, including: Financial indicators Includes the range of indicators that reflect program performance in financial terms, such as costs associated with transportation, collection, processing and communication; often presented in per-weight format. Operational indicators Includes indicators that seek to characterize program performance based on recycling and collection volumes or rates (and other related processes). Awareness indicators Includes measures, often obtained through public surveys to characterize Page 4-2

17 program success in terms of public awareness and engagement. Accessibility indicators Includes measures intended to describe the ease or convenience associated with collection and recycling. Typically presented in a manner that characterizes distance between service population and collection points, number of collection points per specified population, etc. Environmental Impact Measures Indicators intended to characterize or measure actions undertaken to reduce the environmental impacts of electronic products, including design for environment, mass balancing, recycling efficiency and others. 4.4 Financial Summary The degree of reporting on financial indicators is tied closely to the type of program being considered. Financial reporting from programs that are mainly manufacturer/ producer paid for (e.g., U.S.) tends to be relatively minimal. Reporting from the Canadian programs and selected European programs is more substantial Operating and Administrative Costs Several programs (ACES, SWEEP, ESABC, ARMA, SWICO, Recupel, Maine) present costs associated with administration, communication, collection, processing, transportation and other related activities (either in per weight figures or overall costs). Financial indicators for the remaining programs were not in place or not accessible for this research. These types of indicators represented some of the more common financial indicators, although as noted above they may take the form of specific pertonnage expenses or broad details in the annual reports. 4.5 Operational Summary Performance measures related to program operations were found across all jurisdictions in some form. A wide range of measures were identified in this research, not only in what indicators specific programs use but in the way that they are used and calculated. Some of the more common measures include total volume of electronic products collected (weight and units), per capita collection (weight/capita), various forms of capture rates (at times referred to as collection rates, recovery rates or recycling rates). Further details are provided below Volumes Collected Total volume (weights) of electronic products collected or processed represents a very common performance measure. Eleven of the programs utilized this indicator (this type of indicator was not observed for Washington and Oregon) Per Capita Collection (and other per capita operational indicators) Volume (weight) of electronic product collected per capita was also a common indicator used by programs, particularly those in Canada and Europe. Variations were noted in how the population was defined (as of a particular date) as well as timeframe (fiscal year or calendar year). Five of the programs in this study used this performance measure (ACES, ESABC, SWEEP, El Kretsen, Recupel). Page 4-3

18 Variations on this per capita measure were noted, including materials processed or recycled, although some caution must be utilized in interpreting these as the terms are often used interchangeably. Three of the programs in this study utilized an indicator of this kind (ARMA, Minnesota, Maine) Capture Rate 1 The research undertaken for this report indicated capture rates are relatively common performance measures for short life-span non-durable products such as beverage containers. The life-cycle for these products from point of sale to collection for recycling is relatively short. Therefore the calculation of a capture rate can use information on products sold into the market (either units or volume) in a particular period as a reasonable proxy for the amount of material that would be available for collection (the denominator in the capture rate calculation). While there are some simplifying assumptions involved with this method, the approach is generally reasonable and can provide a meaningful indication of program performance. Capture rates and other similar measures were also common among the electronics recycling programs reviewed in this research. These varied significantly from one program to the next, both in the intent of the measurement and in the data that are used to determine them. In the case studies assembled for this research, these were also characterized as collection rates, recycling rates, recycling efficiency rates, market shares or recovery rates. Key examples are noted below. As an interim measure, prior to program commencement, ESABC contemplated reporting a recovery rate calculated by estimating the total number of units sold annually and the total number of units collected. Formula: Total Units Electronics Collected (number of units) Total Units Sold (number of units, in the same fiscal year) The calculation was intended to be based on an approximation of the units collected (based on data collected by the program) and an estimate of the number of units sold (based on fee collection data collected by the program). ARMA calculates a capture rate that compares the total weight of electronics processed for recycling to the total amount available for collection. ARMA uses a model that estimates the total volume of materials available for collection based on products sold, estimated product weights and various product lifespan parameters. 1 The capture rates identified in this study are generally weight-based. The reader is cautioned that different programs define the terms and processes used in these calculations differently (e.g., collection, processing, recycling). Page 4-4

19 Formula: Total Weight of Electronics Processed (tonnes) 2 Estimated Amount of Electronics Available for Collection (tonnes, in the same fiscal year) ARMA also calculates a recovery rate that compares the total weight of electronics processed for recycling to the total weight of product actually sold onto the market in the province. While the numerator in this case is the same, the total sales are calculated through a determination of total units on which an ARF was applied and product weight estimates. Formula: Total Weight of Electronics Processed (tonnes) Total Weight of Electronics Sold (tonnes, within the same fiscal year period; determined using ARF s paid, product weight estimates) The State of Minnesota calculates a recycling rate. It is similar in some respects to those above in that the numerator in the calculation is the total weight of electronics processed for recycling. In this case however, the denominator (total weight of electronics sold) is determined through manufacturer reporting of unit sales in the previous year, with accompanying weight estimates for these products. In the European Union, the WEEE Directive mandates the reporting of a few different performance measures that could be considered capture rates. Their collection rate is determined by comparing the total weight of electronics collected for recycling to the total weight of electronics put on the market. The latter is similar in nature to that put in place in Minnesota. In this case, producers are mandated to report on the total number of units (with weights also provided where feasible) sold into each market. The denominator for this calculation is the average annual weight of products put on the market in the previous two years. Formula: Total Weight of Electronics Collected 3 (tonnes, within specified year) Weight of Electronics Put on the Market (tonnes, average of the previous two years as reported by producers on a country by country basis) In Switzerland, SWICO calculates an indicator they have characterized as a recycling rate. The numerator used is the total weight of electronics collected for recycling while the denominator case is the total weight of electronic products imported into the country eight years previously (i.e., for a 2007 calculation the denominator would be taken from imports in 2000) with their weights estimated. Formula: Total Weight of Electronics Recycled (tonnes, within the given year) Total Weight of Electronics Imported (tonnes, 8 years previously; determined using sales imports data and weight estimates - assumes an average product lifespan of 8 years) 2 Collected refers to the receipt of electronics products for the purpose of recycling. 3 Recycled is used synonymously with collected. Page 4-

20 While the concept of a capture rate is attractive, extending the method to recycling programs that manage durable products such as electronic products raises serious analytical problems. Electronics products are durable goods that have much longer and more variable life-cycles than non-durable products such as beverage containers. In order to develop an estimate for the material available for collection (the denominator in the capture rate calculation), the review from other jurisdictions indicates one of two approaches is generally used: With respect to capture rates based on estimates of the number of units sold in a particular period, the following is noted: 1. Estimates based on EHFs collected will not provide an accurate estimate of units sold, since EHFs may not be applied to every unit sold Electronics sales data purchased from a third-party are not available at the provincial level. 1. In some cases, the calculation uses an estimate based on the actual number of products sold in a particular period compared to the number of units collected. 2. In other cases the calculation uses an estimate based on a model that attempts to predict the amount of material available to be collected based on product. Each of these approaches has analytical limitations. While the programs currently collect robust information related to units to which an EHF is applied and program collections in tonnes of material, estimates of units collected do 3. Finally, regardless of the sales data source, a capture rate based on any estimate of units sold in a particular period does not accurately represent the amount of material available to be collected, due to the long and variable service lives of electronics products. 6 With respect to capture rates based on estimates prepared using a model, the following is noted: 1. These models attempt to predict the amount of material available to be collected based on product weights, life cycle estimates and sales information. not correlate to the sales data for a variety of reasons. 4 4 Electronics products are durable goods that have long service lives. Products may have multiple lives in different uses and may be purchased in one jurisdiction and at some point be moved to another. In addition, collection may be undertaken by another entity outside of the programs. 5 For example, a single EHF submitted on a computer may have been sold with associated cabling, keyboards and mice as part of a package. 6 A television for example, may progress from its first life (primary residence) to a second life (cottage), be placed into an extended period of storage or moved to another jurisdiction. Page 4-

21 2. Where these models have been developed, they are extremely data intensive to produce, expensive to develop and maintain and impossible to independently test and verify. As a result of these limitations, neither approach provides a credible estimate of the material available to be collected in a given period that can be reliably measured and independently confirmed or verified. Given these limitations, a capture rate is not a meaningful performance measure for durable goods such as electronics products Other Indicators A range of other operational performance measures are in use by the programs reviewed in this research, including: Electronics collected but not processed - to identify inventory to be recycled and awaiting shipment to, or stored by, the recycler; Total volumes of substances diverted or recycled (i.e., lead, glass, etc.) - to quantify recovery of specific materials from electronic products; and Total volumes of electronics disposed - to describe residual material that cannot be further processed and therefore must be sent for proper disposition by the recycler. 4.6 Awareness Summary associated with website activity and telephone activity. Comprehensive surveys are also undertaken in some jurisdictions Survey Public opinion surveys are an important measurement tool for the Canadian programs with ACES, ESABC, OES, SWEEP and ARMA all undertaking regular province-wide surveys. Smaller scale surveys have been carried out in other jurisdictions, including Washington and Switzerland (SWICO). Topics usually covered on these include: Program awareness (e.g., public knowledge about program, public support for program, collection site convenience and location, and products included); Other topics (e.g., attempts to characterize the electronics products that are currently out there, motivations of individuals with respect to returning these items for recycling) Website, Hotline Again, the utilization of website and hotline tools in the context of performance measurement is well-developed in Canadian programs (particularly ACES, ESABC, OES and SWEEP). Similar undertakings were not identified in any other jurisdictions Others A range of other activities have been identified by various programs as ways of characterizing their performance. These include: A wide range of measures related to program awareness were identified. The most common indicators were Page 4-7

22 Paid-for advertising coverage (display posters, radio, newspaper/flyer and transit); Speaking engagements and other public communication events; and Earned coverage (through media). 4.7 Accessibility Summary Performance measurement tools for characterizing program accessibility are not as varied or diverse as the other categories of indicators. They range from very simple (e.g., number of collection events) to very complex (e.g., GIS-based coverage analysis) Proximity to Collection - GIS-Based and Similar Three programs (ACES, ESABC and SWEEP) carry out relatively complex GIS-based coverage studies in order to characterize program coverage Collection Sites and Events Seven of the programs in this study include end-oflife collection sites and events in their performance measurement inventory including ACES, ESABC, OES SWEEP and ARMA. El Kretsen and Recupel also compile this information for performance reporting purposes. Other jurisdictions (i.e., United States) did not have substantive reporting mechanisms in place. 4.8 Environmental Impact Summary Although not common across the programs included in this research, some performance measures associated with environmental impact were noted. In many cases these were identified as worth considering or under development (greenhouse gas emissions, mass balancing). The measures presented below are characterized as environmental impact indicators because they are intended to measure the environmental impact associated with recycling. By contrast, operational indicators such as capture rates are intended to measure the ability of the program to divert products from the waste stream to recycling Recycling Efficiency Rate SWICO calculates a recycling efficiency rate. This is done by comparing the total weight of electronics recycled to the total weight of electronics entering the recycling stream. Formula: Total Weight of Electronics Recycled (tonnes, within the given year) Total Weight of Electronics Entering Recycling Others Stream (tonnes, within the given year) Other similar indicators were identified during the course of research including two that are mandated by the WEEE Directive. These indicators were not widely observed across the programs in this study. Page 4-8

23 The WEEE Directive mandates the reporting of a reuse/ recycling rate. Manufacturers are expected to report on the total weight of reusable or recyclable substances (e.g., metals, plastics) found within a specific electronic product as a percentage of the total weight of that electronic product. Formula: Weight of Material Available for Reuse or Recycling (kg. For each of plastic, metal, etc.) Total Weight of Specified Product (kg) reporting. This could be in large part a result of the nature of their different financing models. For example, programs that utilized consumer or steward-paid handling fees were far more likely to report on financial indicators through annual report audit statements or otherwise. This included four of the five Canadian programs (ACES, ESABC, SWEEP and ARMA) and two of the three European jurisdictions (Sweden and Belgium). Maine was the only other program reporting on financial parameters (collection, transportation and processing costs). The Directive also calls for a recovery rate that resembles the above, but with a consideration for energy recovery weight (total weight materials combusted to generate and recover energy from the recycling process). Formula: Weight of Material Available for Reuse or Recycling + Energy Recovery Weight (kg. For each of plastic, metal, etc.) Total Weight of Specified Product (kg) A brief summary of the WEEE Directive is provided in Appendix Target audience The focus of this research was on performance measures that are reported to the general public and regulators. The Research Team did not attempt to characterize the level of performance reporting internally to the organization. The programs included in this research varied widely with respect to performance reporting, both in terms of the indicators used and target audience for performance Page 4-9

24 CHAPTER 5: EVALUATION OF OPTIONS 5.1 Review of performance measure options Following the review of the case study findings, the Research Team drafted an initial list of potential performance indicators. The indicators were reviewed with the Study Advisory Team at a workshop in November 2009 with the goal of selecting a set of core indicators that could be compiled for each program. Potential indicators were assessed using the guiding principles outlined in Chapter 3. Indicators were organized into five categories in order to ensure that all aspects of program performance could be captured: Operational; Financial; Awareness; Accessibility; and Environmental Impact. Table 5.1 summarizes the core indicators selected by the Research Team and the Study Advisory Committee. The indicators were selected with the objective of providing a suite of core indicators that aligns with the guiding principles set out in Chapter 3. The table provides discussion on how the indicators align with the guiding principles. Most of the indicators can be implemented based on data that is presently collected by the programs. However, implementing some indicators will require additional time and data. In order to ensure that the performance measures allow for comparisons across jurisdictions, the Research Team and the Study Advisory Committee identified certain data collection and reporting protocols. These protocols are summarized in Table 5.2. Of the 13 recommended indicators, it is anticipated that nine could be implemented based on data that are available at present. Three indicators are recommended for implementation by 2012, once certain data collection protocols have been established. The final indicator would not be fully implemented for at least two years (2012 or beyond). Page 5-1

25 Table 5.1 Summary of Recommended Performance Measures A: Operational Indicators Indicator Discussion Recommendation A1 Total WEEE collected (tonnes) A2 Total WEEE collected per capita (tonnes) Representative of performance, although increases in population or addition of new product categories may influence results. Easy to communicate clearly. Programs well positioned to collect data. Representative of performance, although addition of new product categories may influence results. Easy to communicate and understand. Programs can collect WEEE data, but will need to agree on a data protocol for population statistics. Include in initial suite of core indicators Include in initial suite of core indicators Page 5-2

26 Table 5.1 Summary of Recommended Performance Measures B: Financial Indicators Indicator Discussion Recommendation B1 Total program costs per tonne B2 Operational costs (collection, consolidation, transportation and processing) per tonne B3 Overhead costs (administration, communication, outreach etc) per tonne Representative of performance in terms of the programs ability to manage costs per unit. Data is already collected by the programs and reported as part of audited financial statements. Representative of performance in terms of the programs ability to manage operating costs. Data are already collected by the programs. May be some concerns with respect to consistency of defining what constitutes operational costs that would need to be addressed. Representative of performance in terms of the programs ability to manage overhead costs. Data are already collected by the programs. May be some concerns with respect to consistency of defining what constitutes overhead costs that would need to be addressed. Include in initial suite of core indicators Include in initial suite of core indicators Include in initial suite of core indicators Page 5-