How To Measure The Effectiveness Of A Transit Security Awareness Campaign

Transcription

1 Exploring the Effectiveness of Transit Security Awareness Campaigns in the San Francisco Bay Area MTI Report 09-19

2 MINETA TRANSPORTATION INSTITUTE The Norman Y. Mineta International Institute for Surface Transportation Policy Studies (MTI) was established by Congress as part of the Intermodal Surface Transportation Efficiency Act of Reauthorized in 1998, MTI was selected by the U.S. Department of Transportation through a competitive process in 2002 as a national Center of Excellence. The Institute is funded by Congress through the United States Department of Transportation s Research and Innovative Technology Administration, the California Legislature through the Department of Transportation (Caltrans), and by private grants and donations. The Institute receives oversight from an internationally respected Board of Trustees whose members represent all major surface transportation modes. MTI s focus on policy and management resulted from a Board assessment of the industry s unmet needs and led directly to the choice of the San José State University College of Business as the Institute s home. The Board provides policy direction, assists with needs assessment, and connects the Institute and its programs with the international transportation community. MTI s transportation policy work is centered on three primary responsibilities: Research MTI works to provide policy-oriented research for all levels of government and the private sector to foster the development of optimum surface transportation systems. Research areas include: transportation security; planning and policy development; interrelationships among transportation, land use, and the environment; transportation finance; and collaborative labormanagement relations. Certified Research Associates conduct the research. Certification requires an advanced degree, generally a Ph.D., a record of academic publications, and professional references. Research projects culminate in a peer-reviewed publication, available both in hardcopy and on TransWeb, the MTI website ( Education The educational goal of the Institute is to provide graduate-level education to students seeking a career in the development and operation of surface transportation programs. MTI, through San José State University, offers an AACSB-accredited Master of Science in Transportation Management and a graduate Certificate in Transportation Management that serve to prepare the nation s transportation managers for the 21st century. The master s degree is the highest conferred by the California State University system. With the active assistance of the California Department of Transportation, MTI delivers its classes over a state-ofthe-art videoconference network throughout the state of California and via webcasting beyond, allowing working transportation professionals to pursue an advanced degree regardless of their location. To meet the needs of employers seeking a diverse workforce, MTI s education program promotes enrollment to under-represented groups. Information and Technology Transfer MTI promotes the availability of completed research to professional organizations and journals and works to integrate the research findings into the graduate education program. In addition to publishing the studies, the Institute also sponsors symposia to disseminate research results to transportation professionals and encourages Research Associates to present their findings at conferences. The World in Motion, MTI s quarterly newsletter, covers innovation in the Institute s research and education programs. MTI s extensive collection of transportation-related publications is integrated into San José State University s world-class Martin Luther King, Jr. Library. DISCLAIMER The contents of this report reflect the views of the authors, who are responsible for the facts and accuracy of the information presented herein. This document is disseminated under the sponsorship of the U.S. Department of Transportation, University Transportation Centers Program and the California Department of Transportation, in the interest of information exchange. This report does not necessarily reflect the official views or policies of the U.S. government, State of California, or the, who assume no liability for the contents or use thereof. This report does not constitute a standard specification, design standard, or regulation.

3 MTI Report EXPLORING THE EFFECTIVENESS OF TRANSIT SECURITY AWARENESS CAMPAIGNS IN THE SAN FRANCISCO BAY AREA June 2010 Nina Rohlich Peter J. Haas, Ph.D. Frances L. Edwards, Ph.D. a publication of the College of Business San José State University San José, CA Created by Congress in 1991

4 Technical Report Documentation Page Report No. CA-MTI Government Accession No. 3. Recipients Catalog No. Title and Subtitle Exploring the Effectiveness of Transit Security Awareness Campaigns in the San Francisco Bay Area Report Date June 2010 Performing Organization Code Authors Nina Rohlich Peter J. Haas, Ph.D. Frances L. Edwards, Ph.D. Performing Organization Name and Address College of Business San José State University San Jose, CA Performing Organization Report No. MTI Report Work Unit No. 11. Contract or Grant No. DTRT 07-G Sponsoring Agency Name and Address 13. Type of Report and Period Covered Final Report 14. Sponsoring Agency Code California Department of Transportation Sacramento, CA U.S. Department of Transportation Office of Research MS42 Research & Special Programs Administration P.O. Box th Street, SW Washington DC Supplementary Notes 16. Abstract Public involvement in alerting officials of suspicious and potentially harmful activity is critical to the overall security of a transit system. As part of an effort to get passengers and the public involved, many transit agencies have security awareness campaigns. The objective of this research is to learn how transit agencies seek to make security awareness campaigns effective and explore how they measure the effectiveness of such campaigns, if at all. This research project includes data from case studies of five major agencies that provide transit service in the San Francisco Bay Area region. The case study data are comprised of descriptions of the types of security awareness campaigns the agencies have implemented, the goals of the campaigns, and how they seek to make their campaigns effective, as well as whether and how these agencies measure and determine the effectiveness of their campaigns. A positive finding of this research is the consistency with which Bay Area transit organizations address the need for passenger awareness as part of their overall security program. However, none of the five agencies analyzed for this study measures the effectiveness of their campaigns. Whereas they all have a similar goal to increase passenger awareness about security issues little evidence exists confirming to what extent they are achieving this goal. The paper concludes with suggestions for using outcome measurements to provide a reasonable indication of a campaign s effectiveness by capturing the public s response to a campaign. 17. Key Words Public Transportation; Security; Terrorism; Marketing; Public Information Programs 18. Distribution Statement No restrictions. This document is available to the public through The National Technical Information Service, Springfield, VA Security Classif. (of this report) Unclassified 20. Security Classifi. (of this page) Unclassified 21. No. of Pages Price $15.00 Form DOT F (8-72)

5 Copyright 2010 by All rights reserved Library of Congress Catalog Card Number: To order this publication, please contact the following: College of Business San José State University San José, CA Tel (408) Fax (408)

6 Acknowledgments Many thanks to the following people, whose input and support made this project possible. Thank you to Christine Dunn, public information officer, San Mateo County Transit District; Frances Kernodle, president, Frances Kernodle Associates, Inc.; Kellee J. Hopper, marketing & communications director, Golden Gate Bridge Highway & Transportation District ; Matt Raymond, chief communications officer, Los Angeles County Metropolitan Transportation Authority; Priscilla T. Kalugdan, marketing manager, Capitol Corridor Joint Powers Authority. Thank you to Bay Area Rapid Transit (BART) staff, including Frederick A. Edwards, manager, Operations Control Center; Melissa A. Miller, principal marketing representative, Marketing and Research; Pat Seto, supervisor, Transit Information Center. Federal Transit Administration (FTA) staff who provided their assistance are Bridget Zamperini, Transit Safety & Security specialist, Office of Transit Safety and Security; and Mike Flanigon, director, Office of Transit Safety and Security. Thank you to Santa Clara Valley Transportation Authority staff, including Bernice Alaniz, deputy director, Marketing & Communications; Brandi Childress, community outreach supervisor, Media Relations and Community Outreach Department; and Catherine A. Hendrix, senior management analyst, Protective Services Thank you to the anonymous peer reviewers for their input. (MTI) staff instrumental in assisting in the creation of this report include Executive Director Rod Diridon, Sr.; Director of Communications & Special Projects Donna Maurillo; Director of Research Karen Philbrick, Ph.D.; Research Support Manager Meg Fitts; Student Publications Assistant Sahil Rahimi; Student Graphic Artists JP Flores and Vince Alindogan; Webmaster Frances Cherman; and Editorial Associates Robyn Whitlock and Catherine Frazier. Author s Note: Similarities exist between this publication and a version to be published by the Transportation Research Board in December 2010.

7 i Contents Executive Summary 1 Background 3 Research Methods 9 Data SUMMARY 11 Bay Area Rapid Transit District 11 San Mateo County Transit District 14 Capitol Corridor 16 Santa Clara Valley Transportation Authority 18 Golden Gate Bridge, Highway and Transportation District 21 Findings 23 Making Campaigns Effective 23 Measuring Effectiveness 26 Recommendations For ACTION 31 CONCLUSION 33 Appendix A: Transit Watch Program Templates 35 Appendix B: Transit Agency Security Awareness Campaign Materials BART 37 SamTrans 39 Capitol Corridor 40 VTA 41 Golden Gate Transit 43 Appendix C:Complete List Of Transit Agency Interview Questions 45 Endnotes 47 Abbreviations and acronyms 53 bibliography 55 About the Authors 57 Peer Review 59

8 ii Contents

9 iii List of Tables 1. Summary Data for Transit Agency Security Campaigns Outcome Indicators and Possible Measurements for Security Campaigns 28

10 iv List of Tables

11 1 Executive Summary Public transit has been and will likely continue to be a target of terrorist attacks. The terrorist attacks of September 11, 2001 against the Pentagon and the World Trade Center using airplanes highlighted the need for increased security in the United States transportation sector overall, and prompted additional security efforts for many public transit agencies across the U.S. The March 11, 2004 Madrid commuter train bombings, the July 7, 2005 London transit system bombings, and the March 29, 2010 Moscow metro attacks are more recent reminders of the need for vigilance. Due to its openness and accessibility, public transit is considerably more vulnerable than airports, seaports, and other transportation modes organized around limited access points that can institute widespread security screening measures. In addition to relatively open access points, transit systems often have large numbers of passengers during commute hours, accessible schedules and timetables, are in close proximity to other potential targets, and are critical pieces of infrastructure for urban areas. Public involvement in alerting officials to suspicious and potentially harmful activity is critical to the overall security of a transit system. As part of an effort to get passengers and the public involved, many transit agencies have created security awareness campaigns. The objective of this research project was to (1) determine how transit agencies seek to make security awareness campaign effective, and (2) explore ways in which to measure the effectiveness of such campaigns. This project focused on understanding the types of security awareness campaigns that a select group of transit agencies in the San Francisco, California Bay Area region have implemented, the goals of the campaigns, and whether the agencies are achieving those goals. Furthermore, have campaigns increased security awareness and passenger engagement? If so, how do agencies evaluate the effectiveness of the campaigns? The case study data are comprised of descriptions of the types of security awareness campaigns the agencies have implemented, the goals of the campaigns, and how they attempt to make their campaigns effective, as well as whether and how these agencies measure and determine the effectiveness of their campaigns. A positive finding of this research is the consistency with which Bay Area transit organizations address the need for passenger awareness as part of their overall security program. However, none of the five agencies analyzed for this study measures the effectiveness of their campaigns. Whereas they all have a similar goal to increase passenger awareness about security issues little evidence therefore exists confirming whether they are achieving this goal. In order to capture the public s response to the campaign and to understand whether they are achieving their campaign goals of increasing awareness, providing tools for action, and encouraging passenger involvement, agencies should implement a combination of output and outcome measurements. At a minimum, agencies should track the level of marketing activities and strive to capture at least one set of meaningful data that captures passenger behavior and comprehension by using internal tracking mechanisms or surveys.

12 2 Executive Summary

13 3 Background There are a number of protective measures that transit agencies can implement to provide additional security for passengers. Such measures could include having security patrols on the transit system or having staff well-trained in terrorism detection and prevention. 1 Among the essential elements of an effective security program for a transit agency is implementing a public awareness and preparedness campaign. 2 Security Awareness Campaigns as Part of Transit System Security Transit security awareness is a key component of overall security initiatives for a transit system. In 2002, the Federal Transit Administration (FTA) began a technical assistance program for the largest transit agencies, an initiative developed to analyze security gaps and provide products to assist agencies. 3 Similar to transit system safety efforts, the primary goal of this program is to ensure that security is an integral component of all system operations. 4 The TSA/FTA publication Security and Emergency Management Action Items for Transit Agencies emphasizes implementation and reinforcement of a public security and emergency awareness program. Key components highlighted in an updated version of this action item include: prominent display of the security awareness information throughout the system, incorporating the messages into public announcements in stations and on board vehicles, and posting information on the agency website. In addition, the components include ways to make the campaigns more effective by ensuring that information provided to the public emphasizes the importance of vigilance and gives clear direction on reporting suspicious activity with the suggestion to vary the content and appearance of messages to retain public interest. 5 The list also states that the frequency of security awareness activities should increase as the Homeland Security Advisory System (HSAS) threat level is raised. 6 Of note is that the recent Homeland Security Advisory Council task force report findings recommend that the new HSAS baseline remain as guarded and that future threats be more narrowly targeted. As such, current transit agency protocol may be in flux. 7 Furthermore, transit agency public awareness and preparedness campaigns were identified as one of the Department of Homeland Security s (DHS) top selection criteria for the Fiscal Year 2007 Infrastructure Protection Program Transit Security Grant Program. The grant literature states that effective awareness programs enlist the public in becoming an informal part of an agency security plan. They should explain specific actions the public can take to contribute to the strengthening of system security. 8 While the importance of transit security public awareness programs is clear, there are potential disadvantages to encouraging public involvement in security. One report states that repeated public warnings may frighten passengers, lead to false alarms, and even inspire hoaxes. It noted that in places where the terrorist threat is remote, the adverse consequences of repeated campaigns and warnings may outweigh the benefits of public awareness. 9

14 4 Background Transit Watch Program After the events of 9/11, the FTA prioritized transit passenger and employee participation in transit security measures and recognized the need to provide agencies with tools to administer transit security awareness campaigns. In a partnership with the DHS and the TSA, the FTA launched Transit Watch in 2003, a nationwide public awareness outreach campaign that encourages the active participation of transit passengers and employees in maintaining a safe transit environment. 10 Its goal was to create a useful toolkit that enabled transit operators to customize campaign materials so as to maximize community interest and involvement. The second Transit Watch initiative was conducted in by the FTA, DHS/TSA and the then-dhs Office of Grants and Training. One of the purposes of the second initiative was to gauge the implementation of Transit Watch and its effectiveness. 11 Information gathered by the FTA/DHS Transit Watch Program Effectiveness Form includes how effective Transit Watch is in terms of usage, the degree to which certain transit agencies have applied the various programs included in the toolkit, as well as other security awareness measures. 12 Many agencies have used and continue to use the programs provided by Transit Watch such as the Unattended Items Be Alert! or Is this yours? campaigns to engage the public in security measures. 13 Some agencies have created their own campaigns with additional messages, targeting specific audiences, and customizing the outreach to the passengers on their system. The Transit Watch program effectiveness questionnaire was very extensive and included a number of sections related to transit agencies announcement systems and evacuation messages. The questionnaire asked transit agency respondents whether they noticed an increase in reports from passengers when the campaign was implemented. 14 However, individual or aggregate agency responses from the questionnaire are not available to the public. The FTA has provided analysis of certain responses from the questionnaire pertaining to Emergency Announcements, Unattended Baggage, and Transit Evacuation. 15 In the Emergency Announcements document, the analysis recommends that guidelines for correlating the frequency of emergency announcements with the DHS threat code advisory system be created. As previously mentioned, this guideline may not be applicable due to the recommended changes to the HSAS threat level baseline. 16 The Unattended Baggage report contained supplemental detail about the use of cell phones near potentially explosive devices and provided alternative messages for transit agency use. 17 Examples include Is this yours? If not, don t touch but tell us. or Whose is that? Don t touch. Tell us. and Be Alert! Don t touch unattended items. 18 The Transit Watch program provides a means for transit agencies to educate their passengers on how to be observant and how to identify suspicious or unusual activity. In addition, the messages instruct passengers on how to communicate with transit officials in such events. The ready-to-use templates allow transit agencies to customize the outreach for their own system, while maintaining consistent messaging with the nationwide effort.

15 Background 5 Comparative Examples Distinctions ought to be drawn between transit agencies located in areas that have been targets of terrorist attacks in the past from those agencies where the threat of attack is remote. Transit agencies on the East Coast such as the Washington Metropolitan Area Transit Authority (WMATA) and New York Metropolitan Transportation Authority (NYMTA) are more sensitive when it comes to terrorism security concerns than those on the West Coast. Whether a security campaign is effective and the way in which it is measured may be completely different when comparing New York City and Washington DC with Bay Area transit agencies. International efforts also vary considerably from local counterparts, with the degree of what is considered effective stretching even further across the globe. In London, for example, the system goes far beyond campaigns to encourage public involvement by ensuring a support infrastructure that makes reporting suspicious activity easy. 19 Their system requires a major investment and includes call boxes for public use throughout stations that are monitored by multiple cameras, resulting in high quality surveillance. 20 Since characteristics of these transit agencies are of a different nature from those in the Bay Area, they are not in the scope of this research. Public information on measuring the effectiveness of security campaigns is scarce, though one article illustrates the impact of a campaign in New York. In 2003, New York s Metro North Railroad ran an awareness campaign If You See Something, Say Something, that encouraged commuters to report suspicious packages and people. Shortly after the launch of the campaign, the NYMTA reported an increase in the number of calls it received about suspicious behavior. 21 MTA spokesperson Tom Kelly said that the campaign was considered to be successful due to the increased interaction between customers and the system, and that this has given people a reason to be more communicative because this is something that we as a railroad and overall transit system are encouraging them to do. 22 Program Effectiveness Public awareness campaigns, which are defined as social marketing, and commercial advertising campaigns, are quite different. The goal of public awareness campaigns is to generate an action by the public to support a social good, rather than to purchase a product or patronize a service. Social marketing, according to Kotler and Lee, is the use of marketing principles and techniques to influence a target audience to voluntarily accept, reject, modify, or abandon a behavior for the benefit of individuals, groups, or society as a whole. Its intent is to improve the quality of life. 23 While the intent or the desired call to action of public awareness campaigns is different from commercial ad campaigns, the basic principles to motivate people to take action may be very similar. 24 The authors provide principles to make these types of marketing efforts more successful. The first principle is to take advantage of prior and existing successful campaigns. 25 One of the benefits of the public sector is that organizations can borrow campaigns and use ideas that other agencies have spent time and money to develop. A second principle is to promote a single, simple, doable behavior, and to promote it one at a time. There is a wealth of advertising clutter targeting the audience, so it is important to provide a simple,

16 6 Background clear, and action-oriented message to the target market. 26 A third principle is to identify and remove barriers to behavior change. This principle states the importance of ensuring that nothing is perceived to be in the way of considering the desired behavior. 27 Simple and clear messages generally lead to a successful communication campaign, and transit security awareness campaigns are no exception. However, because the outcome is a social good that benefits a common community, effective transit security awareness campaigns also need to be educational. 28 This is because they are encouraging passengers to take some sort of action, whether it is being more alert on the transit system or contacting transit police when they see something suspicious. The campaign message must clearly explain exactly how to contact the appropriate employee or agency when the passenger sees something. Each message that goes out to the passengers, employees, and general public must be consistent. 29 Branding the message with the same logo or slogan can also help the audience recognize the main point of the message as well as the sender of the communication. 30 The use of the same logo or slogan throughout various communication tools helps link all system communications and thus improves their effectiveness. 31 A recent Transit Cooperative Research Program (TCRP) report suggests a good campaign may also differentiate the brand or message being communicated and invokes an emotional or physical response from the audience. As a result, the branding messages must use phrases that direct the audience to take action. 32 For example, the New York Metropolitan Transportation Authority s wording is simple and directs action, If You See Something, Say Something. 33 The report suggests that agencies should follow examples from other public advocacy groups or corporations to come up with original ideas for public transit security campaigns. 34 However, communication techniques that work for one transportation system may not be effective or even applicable to other systems. Images and slogans should be reflective of the individual agency because that is what the individual agency s passengers are most familiar with. 35 Measuring Effectiveness Measuring the performance of an awareness campaign is complex and challenging. Existing efforts often lack hard data to support conclusions, providing anecdotal evidence of success at best. Yet reporting the results of a campaign is often expected by public officials. 36 Output measures, the easiest data to obtain, quantify the volume or level of marketing activities. Examples include the number of materials distributed, the reach and frequency of the campaigns, and mentions in the news media. The number of total impressions, or number of people that see the ad, is commonly used as a marketing measurement. 37 The easiest way to measure outputs is by using internal records; however, output measures tell us little about program impact or success. A second category of outputs reflects how citizens responded (outcomes) to what [agencies] did. 38 These are called outcome measures. Possible values include whether people noticed the effort, whether it changed their level of knowledge, or whether it influenced their action or behavior. Internal records and tracking mechanisms can be

17 Background 7 used to measure outcomes by analyzing before and after effects. 39 Surveys can also be reliable ways to measure outcomes from campaign efforts. If goals and objectives of the campaign are clearly defined, then outcome measures may be made operational by means of survey items. The third and perhaps most challenging category of measurement is impact measures. This measurement captures the actual effect that citizen actions had on social, economic, and/or environmental conditions. 40 For transit security awareness campaigns, an example of an impact measure is when terrorist activity is actually prevented by a passenger who knew to take action and alert transit personnel based on the messages included in the campaign. Obviously, these types of examples are rare and/or involve scientific or technical surveys. As such, this project will primarily focus on outcome measures.

18 8 Background

19 9 Research Methods This research focuses on five agencies that provide rail or a combination of light rail, bus, and ferry transit service in the San Francisco Bay Area region. The five agencies are Bay Area Rapid Transit District (BART), San Mateo County Transit District (SamTrans), Capitol Corridor Joint Powers Authority (Capitol Corridor), Santa Clara Valley Transportation Authority (VTA), and Golden Gate Bridge, Highway and Transportation District (Golden Gate Transit or the District). These agencies were selected for comparison due to the differences in the type of transit services they provide to passengers in the Bay Area, as well as their geographic range. Specific case study information was primarily gathered through interviews with Bay Area transit agency representatives in charge of passenger communications, such as the Public Information Officer or marketing and communications manager. Interviews were conducted in person when possible and over the phone if an in-person meeting was not practical. Interviews were conducted using a set of pre-determined questions, yet remained flexible to accommodate a conversational dialogue and facilitate a useful information exchange. The interview questions focused on gathering information on the types of campaigns the transit agency conducts, with a series of questions that probed how they evaluate and measure the effectiveness of the campaign. Thus, questions were categorized in such a way as to understand output measures, such as types of media used and where they were placed, and outcome measures such as before and after surveys or call volume increases. Agencies were asked whether they have data that demonstrate the effectiveness of their security campaign to allow for further analysis. Data for this project were also gathered from various official documents, including publicly available documents from the Federal Transit Administration (FTA), Transportation Security Administration (TSA), (MTI), transit agency websites and news articles. Security awareness campaigns of other transit agencies outside the Bay Area were examined to gain perspective. These sources provided the historical context of transit security awareness campaigns and their role in overall security, as well as recent best practices. Supplemental interviews were conducted with additional staff members at BART to add perspective from an operations standpoint and provide input on possible measurements for security awareness campaigns. In addition, phone interviews were conducted with industry experts in the field of transit marketing for perspective on the Transit Watch program and transit awareness campaigns in general.

20 10 Research Methods