Using Evaluation Theory to Analyze the Homeland Security Exercise and Evaluation Program (HSEEP)

Size: px
Start display at page:

Download "Using Evaluation Theory to Analyze the Homeland Security Exercise and Evaluation Program (HSEEP)"

Transcription

1 1 Using Evaluation Theory to Analyze the Homeland Security Exercise and Evaluation Program (HSEEP) Ralph Renger, PhD, MEP, Jillian Bradshaw, MBA, MEP, Anneke Jansen, MPH, Erin Peacock, MPH, Adriana Cimetta, MPH University of Arizona Arizona Center for Public Health Preparedness

2 Abstract HSEEP Evaluation 2 Exercises play a crucial role in emergency preparedness, providing opportunities for emergency responders and officials to practice and assess their collective capabilities. Conducting and completing a thorough evaluation of those exercises is critical to ensuring the safety of lives and property in an actual response. The Homeland Security Exercise and Evaluation Program (HSEEP) serves two primary purposes. First, it provides common exercise policy and program guidance that constitutes a national standard for exercises. The exercise program uses a capabilities-based framework to guide the rationale for progression from discussion-based to operations-based exercises. As the name implies, the second purpose of the HSEEP is to provide guidance for evaluating the exercise program. However, the HSEEP lacks an evaluation framework. As a result, the evaluation guidance lacks structure and is not as thorough as it could be. The integrated theory of evaluation was selected as a means for analyzing potential ways to improve the HSEEP because of its emphasis on providing data to assist decision-making, which in turn leads to improvements in programs and policies. This philosophy is congruent with that of the HSEEP. Several recommendations for improving HSEEP based on the application of the integrated theory of evaluation are forwarded.

3 3 Using Evaluation Theory to Analyze the Homeland Security Exercise Evaluation Plan (HSEEP) It is clear that the events of September 11, 2001 resulted in a significant shift in federal funding priorities toward improving emergency response. In the past five years, approximately $250 billion have been spent on funding initiatives to better prepare the workforce to respond in an emergency (Redlener, 2006). The overall purpose of these preparedness initiatives is to prevent, protect against, respond to, and recover from major events to minimize the impact on lives, property and the economy (Federal Emergency Management Agency, 2005). Recognizing the importance of correctly designed and evaluated exercises, the United States Department of Homeland Security (2007a,b,c) (DHS) developed the Homeland Security Exercise Evaluation Program (HSEEP) which is a capabilities and performance-based exercise program that provides a standardized policy, methodology, and language for designing, developing, conducting, and evaluating all exercises. To better understand exercise design and evaluation, the first and second authors enrolled in and completed the Emergency Management Master Exercise Practitioner Program presented by the Federal Emergency Management Agency (FEMA). The Master Exercise Practitioner Program is a three-week resident training over a sixmonth period at the National Emergency Training Center (NETC) Emergency Management Institute (EMI), which culminates in the Master Exercise Practitioner (MEP) certification. MEP graduates have demonstrated their proficiency in exercise design and evaluation. The first author was the only attendee whose profession was evaluation; all other attendees were from emergency operations and logistics backgrounds (e.g., FDNY, CDC-SNS, Massachusetts Corrections, numerous state level emergency departments, etc). This offered a unique perspective into the analysis of HSEEP evaluation methodology. Providing the highest quality evaluation in the emergency preparedness context is particularly important given the real world consequences of not being prepared. After completing the MEP there appeared to be many ways in which the evaluation portion of the HSEEP could be improved. What was needed was a systematic way of analyzing the HSEEP to confirm this initial observation and to provide a basis for making recommendations for improvement. Although HSEEP design and implementation, or planning and conduct, (United States

4 4 Department of Homeland Security, 2007b) rest on a solid program theory, there was no evident theory guiding the evaluation component (United States Department of Homeland Security, 2007c). Therefore, the authors turned to the integrated theory of evaluation (Mark, Henry, and Julnes, 2000) to assist in this analysis. The justification for selecting this theory over other theories of evaluation is made later in this paper. It is hoped that the analysis will assist other evaluators in the emergency preparedness environment to better place the HSEEP guidance in context and understand how the guidance can be augmented to produce higher quality plans of evaluation. The manuscript begins by explaining the Multi-Year Training and Exercise Plan (MYTEP), which is the foundation of HSEEP. The MYTEP consists of seven exercises divided into two categories: discussion-based and operations-based. The overview of discussion- and operationsbased exercises is followed by a summary of the HSEEP guidelines for evaluation. The integrated theory of evaluation is then presented. Finally, on the basis of applying the theory, several recommendations for improving the quality of the evaluation component of the HSEEP guidance are provided. The Multi-Year Training and Exercise Plan The Multi-year Training and Exercise Plan (MYTEP), developed by FEMA, uses a capabilitiesand performance-based framework as its guiding structure. FEMA and the DHS developed this framework by first identifying 15 scenarios for which the nation is at particular risk. The scenarios include natural hazards (e.g., earthquake, hurricane) and bioterrorist attacks (e.g., Radiological Dispersal device). The FEMA and DHS then defined 36 target capabilities and the associated critical tasks necessary to effectively mitigate, prepare for, respond to, and recover from these 15 scenarios. For example, the emergency management phase of mitigation is defined by five target capabilities: (i) information collection and threat recognition, (ii) intelligence fusion and analysis, (iii) information sharing and collaboration, (iv) terrorism investigation and apprehension, and (v) chemical, biological, radiological, nuclear, and explosive materials (CBRNE) detection.

5 5 Each of the 36 capabilities is further defined by one or more critical tasks. For example, the target capability of information collection and threat recognition is defined by six critical tasks including: (i) develop and maintain surveillance detection systems, (ii) disseminate timely and accurate national strategic and threat intelligence consistent with security clearances as appropriate, (iii) develop policies and processes to enhance sharing of intelligence and surveillance information within and between regions and states and with federal and local agencies, (iv) collect strategic information, (v) support federal intelligence and surveillance information collection, and, (vi) conduct surveillance and information collection and produce intelligence. The target capabilities and critical tasks are especially important for evaluators because they provide a basis for forming objectives that can be evaluated by observing behaviors. That is, complex functions have been operationalized as target capabilities and critical tasks, allowing for objective assessments. Once objectives (i.e., capabilities and tasks) have been defined, an appropriate exercise or series of exercises must be designed to evaluate them. The HSEEP describes seven exercise types: seminars, workshops, tabletops, games, drills, functional exercises and full-scale exercises designed to test the capabilities of agencies to effectively mitigate, prepare for, respond to and recover from an emergency. Each type of exercise provides a different level of intensity, slowly broadening the scope of those involved and what is exercised. The exercises are grouped into two broad categories: discussion-based and operations-based. This distinction is important from an evaluation perspective because the HSEEP provides evaluation guidance based on this grouping. A brief description of each exercise is presented below. This is followed by a summary of the HSEEP recommendations for evaluating the MYTEP.

6 Discussion-Based Exercises HSEEP Evaluation 6 MYTEP includes four discussion-based exercises: the seminar, workshop, tabletop, and game (United States Department of Homeland Security, 2007b). The purpose of discussion-based exercises is to familiarize individuals and agencies with current policies, agreements, and procedures, and to identify potential areas of improvements. Discussion-based exercises are not typically played out in real time and therefore tend to be low stress. They can be conducted with single or multiple agencies and range from two to eight hours. (i) Seminar The first building block of the MYTEP is a seminar. A seminar is an informal discussion, designed to inform attendees about new or revised plans and/or procedures. (ii) Workshop The second tier of the MYTEP is the workshop. A workshop is intended to produce a deliverable, such as an Emergency Operating Plan (EOP) or new Memorandums of Understanding (MOUs). Thus, workshops tend to be more focused on a specific issue as compared to the lecture-based seminar. Workshops can also be used to assist an exercise design team to define objectives, write scenarios, and develop evaluation criteria. (iii) Tabletop Exercise Emergency mitigation, response and recovery activities often require integration of multiple agencies. Although the EOP or Standard Operating Procedures (SOP) for a single agency typically attempts to include elements to promote effective interaction with other agencies it is impossible to anticipate all contingencies. For this reason, Tabletop Exercises (TTX) are utilized. Tabletop exercises involve multiple agencies and are designed to better integrate EOPs/SOPs between agencies. In an effort to make continual improvements to the EOP/SOP an agency can be involved in multiple tabletop exercises. The number and type of agencies involved in a tabletop exercise will vary as a result of the scenario or goals and objectives (e.g., a bioterrorist attack versus a wildfire).

7 (iv) Games HSEEP Evaluation 7 A game is a simulation of operations that often involves two or more teams (multiple agencies), used to explore decision-making processes and examine consequences of those decisions. Advances in computer and internet technology are allowing for games to become increasing real and serve as a useful tool in simulating how an EOP/SOP might play out. Usually a game is developed as a competition. For example, teams are given objectives/missions and asked to work through the steps necessary to complete them in a given time. The team that completes the mission first or correctly wins. Operations-Based Exercises MYTEP includes three operations-based exercises: drill, functional and full scale. The purpose of operations-based exercises is to confirm and validate the plans, policies and procedures solidified in discussion-based exercises; to clarify roles and responsibilities; and to identify gaps by reacting to situations (United States Department of Homeland Security, 2007b). Operationsbased exercises are played out in real time and tend to include a time element and to be higher stress than discussion-based exercises. There is no set time for how long operations-based exercises can range. Some drills may last 30 minutes while FSE may last 3-4 days. (i) Drills Drills are conducted to test a specific function or operation of a single agency. Drills are performed in a real environment, but in isolation of other functions. For example, a drill could be conducted to test whether epidemiological response teams follow proper procedure for conducting outbreak analysis. This is a very specific and isolated task within the broader context of emergency response. Drills begin to assess the capability of individuals as part of a system responding to specific aspects of an emergency under increased stress. (ii) Functional Exercise A functional exercise (FE) is a simulation designed to test a specific aspect of emergency response, namely the Emergency Operations Center (EOC). The EOC is the nerve center where key decisions regarding the planning, operations, logistics and finance of an emergency are coordinated. The EOC is primarily a support mechanism (resource) for the field operations;

8 8 in a FE the deployment of assets in the field is simulated. A number of different agencies will be represented in the EOC; however, all are governed by a common set of rules and conduct in the form of the National Incident Management System (NIMS) (Federal Emergency Management Agency, 2004) with which the Incident Command System (ICS) is a component (Federal Emergency Management Agency, 2005). (iii) Full Scale Exercise The final level of the exercise program is the full-scale exercise (FSE). The EOC is operational and multiple agencies are involved in the response. One key difference between the FE and the FSE is that resources are actually deployed. The HSEEP Evaluation Guidelines The focus of the HSEEP evaluation is on identifying gaps and improvement planning. This process consists of defining exercise objectives, testing the objectives through one of the aforementioned exercises, collecting data, analyzing results and making corrective actions. Writing exercise objectives is fairly straightforward. The target capabilities form the basis of writing objectives. Drawing on the example presented earlier, an exercise objective might be to improve information collection and threat recognition (i.e., a target capability associated with mitigation). The HSEEP recommends that all objectives be specific, measurable, achievable, realistic, and task-oriented, or SMART (United States Department of Homeland Security, 2007d). Objectives that are based on the critical tasks associated with a target capability are more likely to meet the SMART criteria. From the earlier example, developing and maintaining surveillance detection systems and disseminating timely and accurate national strategic and threat intelligence consistent with security clearances as appropriate are two critical tasks that make the objective SMART. Once objectives are defined, an appropriate evaluation tool is developed to test these objectives. The HSEEP recommends using Exercise Evaluation Guides (EEGs), which are templates to assist evaluators in gathering data regarding the extent to which critical tasks were achieved. The EEG lists the critical tasks being evaluated and asks evaluators to rate the extent to which

9 9 expected actions were completed on a three-point scale (i.e., not met, partially met, completely met). Despite scoring each critical task on a three-point scale, the ratings from critical tasks within a target capability are not to be combined. Instead, the HSEEP recommends that evaluators engage in a root cause analysis to understand why a particular critical task was not met (United States Department of Homeland Security, 2007c). Used in this way, the rating of a critical task simply serves as a flag for the evaluator as to where to conduct a more thorough qualitative inquiry to understand why the critical task was not met. Once the root cause is identified, a recommendation for corrective action and/or improvement is made and the improvement planning cycle is closed. The improvement planning cycle is not complete until the action is to correct the issue is taken. Then the process begins again with planning to exercise the corrections. The EEGs are ideally suited for operations-based exercises. Discussion-based exercises present a different challenge because critical tasks cannot be observed and reference to such tasks must be captured during the flow of a discussion. The HSEEP suggests creating customized evaluation forms to assist evaluators in capturing key discussion points related to the stated objectives. Further, the HSEEP suggests that consideration of the EEGs may be helpful in designing the discussion-based evaluation forms, but the authors were unable to find any further guidance on this issue (United States Department of Homeland Security, 2007c). Immediately following both discussion- and operations-based exercises, HSEEP recommends conducting a hot wash, or debriefing. The primary purpose of the hot wash is to solicit feedback from evaluators, facilitators, members of the planning team and players about the strengths and weaknesses of the exercise design, including the realism of the scenario, whether appropriate agencies were included, and so forth. The hot wash is also an opportunity for evaluators to ask for clarification and gather additional information to ensure a thorough assessment. Analysis of the evaluator s observations (e.g., EEG s or notes) and the hot wash are summarized in an After Action Report (AAR), which describes the extent to which the exercise objectives were met. The AAR also includes an Improvement Plan (IP), which is intended to provide participating agencies with specific steps to make modifications to their respective SOPs, usually

10 in the form of recommended changes to SOPs and or training for agency staff to better understand their respective SOPs. HSEEP Evaluation 10 The Integrated Theory of Evaluation The capabilities approach used by the HSEEP can be considered a program theory. The theory was essential in creating the building block approach of the MYTEP and assisting with exercise design. However, there is no reference in the HSEEP to a theoretical basis that guides the exercise evaluation. One robust evaluation theory that can be applied to the emergency management context is the integrated theory of evaluation by Mark, Henry, and Julnes (2000). A central tenet of the integrated theory of evaluation is social betterment. The theory posits that results of evaluation assist in making decisions that lead to higher quality programs and policies. Better programs and policies, in turn, result in a better society. The premise of the integrated theory is philosophically aligned with the HSEEP in that evaluation data is used to assist exercise developers in making decisions regarding appropriate corrective actions so as to improve programs and policies. Better exercises and resulting policies, will help minimize the loss of life and property. It is for this reason that the integrated theory of evaluation was chosen as a frame of reference for analyzing the evaluation component of the HSEEP. The theory describes four basic purposes of evaluation that can assist decision-making. Three of these: oversight and compliance, program monitoring, and merit and worth are applicable to the HSEEP. Oversight and Compliance The first purpose of the integrated theory of evaluation relevant to the HSEEP is oversight and compliance. An evaluation of oversight and compliance provides a basic measure of accountability. In practical terms, it includes the data necessary for describing the characteristics of the program and people served by the program. Federal funding agencies will often define the type of data required for evaluating oversight and compliance. Typical information includes, but is not limited to, the number, length, type, and cost of training programs created as well as the number and characteristics of people to whom the program was delivered. An evaluation of

11 11 oversight and compliance can provide a gross indication of the level of activity, but often is limited in its ability to provide insight into the quality of the program. Oversight and compliance was deemed relevant to the HSEEP because many exercise programs are funded by taxpayer dollars and mandated by federal agencies (United States Department of Homeland Security (2007a). As such, exercise programs are subject to the Government Performance Results Act (GPRA, 1993), which requires basic reporting of the level of activity. Program Monitoring The second purpose of the integrated theory of evaluation relevant to the HSEEP is program monitoring. As the name suggests, the purpose of this evaluation is to provide feedback regarding the extent to which a program is being delivered with fidelity. To develop a thorough program monitoring plan, it is necessary for an evaluator to have an understanding of how the program was to be delivered. Thus, written documentation, such as the original grant application, supplemental grant guidance, site visit reports, implementation protocols, facilitator manuals, participant manuals, and so forth are important for the evaluator to gain this understanding. Using this documentation, the evaluator is in a position to create the necessary data collection tools and strategies. Examples of tools used to ascertain whether the program plan was followed and if not, why, include checklists, interviews, client feedback forms and focus groups. The benefits of program monitoring are twofold. First, by providing ongoing monitoring, potential problems that could affect whether a program meets its objectives are detected, enabling mid course corrections and resulting in significant cost savings. Second, ongoing monitoring is a form of quality control providing information about how to improve the content and delivery of the program. Because the foundation of the HSEEP rests on overall improvement, program monitoring is a crucial component. Merit and Worth The third purpose of the integrated theory of evaluation applicable to the HSEEP is the evaluation of merit and worth. Simply put, an evaluation of merit and worth attempts to answer the question: What difference does this make? Central to establishing the merit and worth of a program is an understanding of what the program is trying to change. This is often stated in the form of goals and objectives. In terms of the HSEEP an evaluation of merit and worth is

12 12 important to assess whether individuals and systems are capable of effectively responding to an emergency. Application of the Integrated Theory of Evaluation to the HSEEP This section describes the application of the integrated theory of evaluation to analyze the thoroughness of the HSEEP. Each purpose of evaluation and its impact on improving the quality of the evaluation of discussion- and operations-based exercises is discussed. Oversight and Compliance No formal and standardized reporting of oversight and compliance is required by HSEEP. Since there are no standardized reporting requirements across funding agencies and jurisdictions. This makes it difficult to quantify the overall level of activity, characteristics of those being trained, and the types of exercise programs being instituted across the nation. Although not formally required, there are several sources that an evaluator might exploit to gather data needed for evaluating oversight and compliance. For example, in real events, NIMS requires forms such as ICS 211 and ICS 214 (Federal Emergency Management Agency, 2007) to track participants for reimbursement purposes. Such forms could be adapted for all exercises to introduce consistency to reporting. In a functional exercise or full scale exercise evaluators could capitalize on the fact the Planning and Finance Sections of the EOC already gather this information, thus saving data collection resources. A second source might be the AAR, as most evaluators, despite a lack of formal guidance, will typically report on the characteristics of the players, agencies and the exercise itself. A third source could be the grant guidance, which will stipulate the number and types of exercises that should be completed in a fiscal year. In terms of standardizing data collection, thought should be given to creating formal categories for type of agency and professional background of participants, on which data should be gathered during all exercises. Experience has shown that gathering such data is more cost effective, efficient, and accurate if coordinated by a central agency, providing a standardized method for gathering and entering data (Renger, 2006).

13 13 Program Monitoring The foundation of the HSEEP rests on program improvement. Hence, program monitoring is of central importance. To the extent that data regarding critical tasks is gathered and analyzed for the purpose of identifying corrective actions, the HSEEP evaluation plan meets this purpose. However, with an understanding program monitoring is the primary purpose of HSEEP evaluation several limitations can be noted. Those limitations are presented here with a discussion of recommendations for improvement of each. 1. Although the HSEEP recognizes the importance of supporting documentation for the MYTEP, it overlooks the utility of these documents for evaluation, and specifically, for developing a more thorough program monitoring plan. Supporting documentation that evaluators could employ in developing a program monitoring plan include the Situation Manual, Exercise Plan, Controller/Evaluator Handbook, the Master Scenario Events Lists (MSEL), and the Player Handbook. Using these documents can help develop more structured evaluation data collection instruments to identify specific aspects of an exercise that were not implemented according to plan. For example, the MSEL is the timeline for how the exercise is to be delivered. Evaluators should review the MSEL with the controller to understand the extent to which the planned injects were in fact delivered and if not, why. This information is critical in making refinements to the exercise and interpreting why the exercise objectives may not have been met. Such improvements are important because exercises are often repeated, albeit in different regions, allowing subsequent exercises to be of increasing quality. Similarly, the Player Handbook outlines the expectations of the players. This document could be used to design specific questions for players to be used during the debriefing, creating more structure in an otherwise open-ended process. At a more fundamental level of exercise program monitoring, the 8-step design process document, from which all other supporting documentation is derived, (Federal Emergency Management Agency, 1995) is another important document. Information regarding the extent to which an exercise was developed following prescribed guidelines may help explain why certain aspects of the exercise were not delivered as intended. If an exercise was not

14 14 delivered as intended, understanding the extent to which the design process was implemented as intended can help gain a better understanding of the root causes. 2. HSEEP does not include an evaluation of supporting documentation s utility, leaving no opportunity for the design team to get direct and structured feedback. In general, inquiry should be directed into the utility of the supporting documentation itself. During the hot wash, questions should be included to ask about the clarity of the briefing materials, such as the Player Handbook, as well as the player briefing itself. Questions should be posed to controllers regarding the clarity of the exercise plan and the controller briefing. Also, questions should be posed of the evaluators regarding clarity of the evaluation and evaluator briefing. All these are critical elements to monitor to ensure that subsequent iterations of the exercise continue to improve. 3. HSEEP does not include a clear evaluation of facilitator or controller effectiveness. HSEEP recommends having a debriefing after the player hot wash for the controllers and evaluators to discuss the exercise. However, there is no structure to this process. For example, the HSEEP notes in several places the importance of a trained and skilled facilitator or controller. The skill level of this facilitator or controller could have a major impact on the fidelity of exercise implementation. However, the only way in which feedback about the facilitator would be forthcoming is if a participant noted this during the hot wash. It is suggested that post-exercise questions be developed that specifically focus on the skill of the facilitator at guiding discussion-based exercises or the controller in leading operations-based exercises. Further, participants should be asked specific questions about logistics, such as the layout of the room for discussion-based exercises, which is essential to effective communication between multiple partners. Evaluators should also be asked about the layout of the room as this can affect the ability to capture important discussions. 4. Finally, a major point of emphasis in exercise design is to keep scenarios realistic so that participants will buy-in to the value of the exercise and not fight the scenario. Thus, within each exercise the scenario is important. For discussion-based exercises, questions about the usefulness of the supporting audio-visual aids in creating a realistic scenario should be asked.

15 15 For operations-based exercises, questions should be formulated to ascertain the realism of injects or in the case of the Functional Exercise, the effectiveness of simulators in creating realism. Merit and Worth Clearly, a major emphasis of the HSEEP evaluation guidance is writing and assessing objectives that measure capability. By definition, assessing whether there are improvements in capability is tantamount to evaluating the merit and worth of the MYTEP. It is reasonable to posit that assessing improvements in capabilities is in fact more related to merit and worth than improvement planning, which the HSEEP argues is the primary reason for assessing objectives. Understanding that one purpose of the HSEEP is to evaluate merit and worth the following limitations and recommendations are noted: 1. The HSEEP design is not consistent with tracking change. As noted above, the premise of engaging in the MYTEP is to document improvement in capabilities. This is congruent with the premise underlying the evaluation of merit and worth. However, documenting improvement within a single exercise is likely not possible, as at least two points of observation are required to document change. Thus, the same exercise objectives and associated critical tasks have to be tested over time by repeating the exercise. Doing so, however, is resource intensive and cost prohibitive. Thus, even though documenting improvement is emphasized, often exercises can only assess the extent to whether a standard of acceptability (i.e., some critical task) was met. Areas that require improvement are noted in the AAR and specifically within the IP. However, recommended corrective actions are often expressed as changes to the EOP/SOP. If, within the EOP/SOP, required tasks are rewritten, then there may not be an opportunity to assess directly whether a deficient behavior actually changed over time because the actual task being evaluated has changed over time. 2. HSEEP does not clearly define how to gather the data necessary to assess merit and worth. One of the greatest challenges in operations-based exercises is being able to observe all the critical tasks associated with the exercise objective(s). Several, often distant geographical

16 16 locations, multiple players, the need to remain a neutral observer, and the fact that the exercise is being played in real time make it difficult to a) staff sufficient evaluators and b) make accurate observations. Evaluators might consider examining other data that is routinely collected during the exercise to assist in evaluating objectives and critical tasks (e.g., unit logs from participants). However, in the chaos of operations-based exercises this is an unreliable source of information for evaluating exercise objectives: participants often do not have time to make notes and keep accurate records of communications while trying to respond to an incident. One solution might be for evaluators to consider the feasibility of taping (i.e., audio and video) the exercise and examining the tapes later for expected actions and points of review. 3. The HSEEP lacks guidance n evaluating the merit and worth of seminars and workshops, both discussion-based exercises. However, if the purpose is to assess the merit and worth, then several recommendations can be forwarded. If one purpose of a seminar is to raise awareness about the need to write or update EOP/SOPs, then assessing whether this occurred could be used to evaluate the merit and worth. This could be done directly by prospectively asking participants or indirectly by using secondary data sources to follow up on (a) whether agencies without EOPs/SOPs have started to draft one, (b) whether agencies with EOPs/SOPs have taken steps to ensure they are updated, or (3) whether agencies have commenced training for staff to become more competent with their EOPs/SOPs. Second, although there is a lack of guidance, HSEEP is very clear in stating that what differentiates a seminar from a workshop is that the latter results in a product. Therefore, evaluators should know what the intended product is and determine whether that was delivered. The workshops might be designed to provide training to participants to better prepare them for a Tabletop Exercise, Functional Exercise or Full Scale Exercise. Therefore, the evaluation of merit and worth might include whether the knowledge or skills targeted by the workshop improved. Even in the absence of direction from the HSEEP for evaluating seminars and workshops, evaluators can derive and evaluate objectives for both. Summary

17 17 Conducting and completing a thorough evaluation of emergency preparedness exercises is critical to ensuring the safety of lives and property in an actual response. The basic premise of the HSEEP is to provide data upon which exercise designers can make decisions regarding how to improve exercise delivery and the extent to which exercise objectives were achieved. A theoretical foundation is essential to ensuring that an evaluation plan is thorough, of the highest quality and meets the needs of the client. However, the HSEEP lacks such a foundation for evaluation. The integrated theory of evaluation was selected as a means for analyzing the thoroughness of the HSEEP because of its emphasis on providing data to assist decision making, which in turn leads to improvements in programs and policies. This philosophy is congruent with that of the HSEEP. The analysis revealed that the HSEEP does not provide guidance for evaluating oversight and compliance. Although this is not a stated purpose of the HSEEP, while applying the integrated theory of evaluation the usefulness of standardizing data collection quickly becomes evident. This data can describe the characteristics of the participants and training exercises, provide legislators with a general indication of the reach of emergency preparedness training, and assist them in determining how best to allocate dwindling resources. Improvement planning is a central theme of the HSEEP. This purpose aligns with that of program monitoring. The AAR and IP are consistent with this purpose. The HSEEP also recommends monitoring exercises through the use of a hot wash and player debriefings. These are valuable processes that do serve their intended purpose. However, understanding that one purpose of evaluating the MYTEP is to monitor implementation can direct evaluators to other valuable sources of information, such as required supporting documentation, to develop a more comprehensive and objective evaluation of exercise delivery. Further, understanding that program monitoring is a major purpose of exercise evaluation can provide direction as to how to add structure to gathering critical pieces of data that would otherwise be left to chance to surface during the open ended hot wash. The focus on exercise objectives grounded in target capabilities is central to evaluating the merit and worth. The HSEEP is clear in the need to assess the critical tasks associated with each target

18 18 capability. Critical tasks are indeed a significant strength of the HSEEP evaluation by providing an objective and unobtrusive performance measure. However, similar direction to evaluate exercise effectiveness is lacking for discussion-based exercises. By understanding the purpose of merit and worth, several recommendations of evaluating discussion-based exercises were forwarded. The work here demonstrates how quickly evaluators can recognize other sources of information that may be of utility in completing an evaluation if the purpose of the evaluation is clear. In the current context, the integrated theory of evaluation was applied. However, there are certainly other theories of evaluation that could serve the same function. The point is that in the absence of evaluation theory, the evaluation plan (if there is one) may be incomplete, and therefore evaluators should seek a framework that can assist in providing structure to create a more thorough evaluation plan. The FEMA and the DHS should be commended for developing the HSEEP and recognizing the importance of evaluation in assisting decision making and improving the nation s emergency preparedness. However, by applying evaluation theory, better evaluation tools can be developed. These, in turn, will add meaning and direction to the HSEEP and therein provide exercise developers information that will assist in generating better data upon which to base decisions regarding the scope, delivery, and effectiveness of exercises.

19 References HSEEP Evaluation 19 Federal Emergency Management Agency Emergency (1995). An orientation to community disaster exercises. Emergency Management Institute, Emmitsburg, MD. Federal Emergency Management Agency (2004). ICS-700: National incident management system (NIMS), an Introduction self-study guide. Emergency Management Institute, Emmitsburg, MD. Federal Emergency Management Agency (2005). ICS-100: Introduction to ICS student manual. Emergency Management Institute, Emmitsburg, MD. Federal Emergency Management Agency (2007). Emergency Management Institute, Emmitsburg, MD. Government Performance Results Acts (GPRA) (1993). Mark, M. M., Henry, G. T., & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving policies and programs. San Francisco: Jossey Bass. Redlener, I. (2006). Americans at Risk: Why We Are Not Prepared for Megadisasters and What We Can Do. New York, New York: Knopf Publishing Group. Renger, R. (2006). Consequences to Federal Programs when the logic modeling process is not followed with fidelity. American Journal of Evaluation, 27 (4), United States Department of Homeland Security (2007a). Homeland Security Exercise and Evaluation Program: Volume I HSEEP Overview and Exercise program Management. United States Department of Homeland Security (2007b). Homeland Security Exercise and Evaluation Program: Volume II Exercise Planning and Conduct. United States Department of Homeland Security (2007c). Homeland Security Exercise and Evaluation Program: Volume III Exercise Evaluation and Improvement Planning. United States Department of Homeland Security (2007d).

The Homeland Security Exercise and Evaluation Program (HSEEP)

The Homeland Security Exercise and Evaluation Program (HSEEP) 1 What is the Homeland Security Exercise and Evaluation Program (HSEEP)? The Homeland Security Exercise and Evaluation Program (HSEEP) A capabilities and performance-based exercise program Provides a standardized

More information

Homeland Security Exercise and Evaluation Program Terminology, Methodology, and Compliance Guidelines

Homeland Security Exercise and Evaluation Program Terminology, Methodology, and Compliance Guidelines Homeland Security Exercise and Evaluation Program Terminology, Methodology, and Compliance Guidelines HOMELAND SECURITY EXERCISE AND EVALUATION PROGRAM (HSEEP) The Homeland Security Exercise and Evaluation

More information

SUPPORT ANNEX 16 TRAINING AND EXERCISES

SUPPORT ANNEX 16 TRAINING AND EXERCISES I. PURPOSE SUPPORT ANNEX 16 TRAINING AND EXERCISES Training is provided to prepare local and State emergency response personnel and partners to accomplish their emergency or disaster assignments. It is

More information

Overview Of Emergency Management Exercises

Overview Of Emergency Management Exercises U.S. Department of Education Office of Safe and Healthy Students Overview Of Emergency Management Exercises Readiness and Emergency Management for Schools (REMS) Technical Assistance (TA) Center www.rems.ed.gov

More information

1) Introduction. Why do Organizations Conduct Exercises? Exercises are used by organizations to:

1) Introduction. Why do Organizations Conduct Exercises? Exercises are used by organizations to: Homeland Security Exercise and Evaluation Program (HSEEP): Quick Reference Guide Michael Petrie, EMT-P, MBA, MA, EMSci Program Director, mgpetrie@berkeley.edu 1) Introduction What are Exercises? Exercises

More information

Texas Department of Public Safety Texas Division of Emergency Management. Preparedness Standards for Emergency Management in Texas TDEM-100

Texas Department of Public Safety Texas Division of Emergency Management. Preparedness Standards for Emergency Management in Texas TDEM-100 Texas Department of Public Safety Texas Division of Emergency Management Preparedness Standards for Emergency Management in Texas June 2000 FOR ADDITIONAL INFORMATION Requests for additional copies of

More information

Homeland Security Exercise Evaluation Program (HSEEP) INTRODUCTION & HSEEP FUNDAMENTALS

Homeland Security Exercise Evaluation Program (HSEEP) INTRODUCTION & HSEEP FUNDAMENTALS Homeland Security Exercise Evaluation Program (HSEEP) INTRODUCTION & HSEEP FUNDAMENTALS HSEEP Training Course Agenda Instructor Introduction Participant Introductions please respond with: Name preference

More information

Homeland Security Exercise and Evaluation Program (HSEEP) SELF-HELP GUIDE AGENCY LOGO

Homeland Security Exercise and Evaluation Program (HSEEP) SELF-HELP GUIDE AGENCY LOGO Homeland Security Exercise and Evaluation Program (HSEEP) SELF-HELP GUIDE AGENCY LOGO Introduction You need help let s face it. Homeland Security Exercise and Evaluation Program or HSEEP for short. A federal

More information

After Action Report/Improvement Plan

After Action Report/Improvement Plan [Full Exercise Name] [Exercise Name Continued] [Note for Template: Text found in this document that is highlighted and bracketed is included to provide instruction or to indicate a location to input text.

More information

Emergency Preparedness at Nuclear Power Plants

Emergency Preparedness at Nuclear Power Plants A White Paper Addressing Compliance with NRC Proposed Rule making Emergency Preparedness at Nuclear Power Plants Ensuring Readiness and Compliance with New NRC Regulation of Emergency Preparedness Programs

More information

Homeland Security Exercise and Evaluation Program. Volume IV: Sample Exercise Documents and Formats U.S. DEPARTMENT OF HOMELAND SECURITY

Homeland Security Exercise and Evaluation Program. Volume IV: Sample Exercise Documents and Formats U.S. DEPARTMENT OF HOMELAND SECURITY U.S. DEPARTMENT OF HOMELAND SECURITY OFFICE FOR DOMESTIC PREPAREDNESS Homeland Security Exercise and Evaluation Program Volume IV: Sample Exercise Documents and Formats Table of Contents I) Introduction

More information

Tampa Bay Catastrophic Plan ANNEX L: HURRICANE PHOENIX EXERCISE

Tampa Bay Catastrophic Plan ANNEX L: HURRICANE PHOENIX EXERCISE Tampa Bay Catastrophic Plan ANNEX L: HURRICANE PHOENIX EXERCISE This page intentionally left blank Tampa Bay Catastrophic Plan Hurricane Phoenix A Storm Recovery Tabletop Exercise August 5, 2010 EXERCISE

More information

Texas Exercise Frequently Asked Questions 2013

Texas Exercise Frequently Asked Questions 2013 What documents or resources are available for EMPG exercise requirements? Each fiscal year s Local EMPG Guide and associated Information Bulletins are available at http://www.txdps.state.tx.us/dem/councilscommittees/empg/index.htm

More information

How To Write A National Exercise And Evaluation Program

How To Write A National Exercise And Evaluation Program State of Nevada Exercise Program Guidance July 12, 2012 Page left intentionally blank ii Table of Contents Table of Contents iii Record of Changes.iv Record of Distribution.v I. Introduction..1 II. Homeland

More information

Federal Emergency Management Agency

Federal Emergency Management Agency Federal Emergency Management Agency Site Activation Call-down Drill Exercise Plan [MASS CASUALTY DRILL] Exercise Date: 12/14/12 Publishing Date: 10/08/12 FINAL INTENTIONALLY LEFT BLANK PREFACE National

More information

HSEEP Exercise Documentation

HSEEP Exercise Documentation This brief summary provides the basic rationale for the HSEEP resources and documents. 1 It is not meant to be a full summary of the HSEEP program. For more information on HSEEP compliance and the exercise

More information

Template introduction:

Template introduction: Template introduction: The attached template is designed to assist you in creating an Orientation Seminar, Drill, Tabletop, Functional, or Full Scale exercise. The format allows for the creation of the

More information

Maryland Fire and Rescue Institute University of Maryland College Park

Maryland Fire and Rescue Institute University of Maryland College Park Homeland Security Exercise and Evaluation Program HSEEP By: Richard Armstrong, BS, CFPS, Fire Officer IV Regional Coordinator MFRI, North East Regional Office Maryland Fire and Rescue Institute University

More information

Homeland Security Exercise and Evaluation Program. Volume 1: HSEEP Overview and Exercise Program Management. Revised March 2006

Homeland Security Exercise and Evaluation Program. Volume 1: HSEEP Overview and Exercise Program Management. Revised March 2006 Homeland Security Exercise and Evaluation Program Volume 1: HSEEP Overview and Exercise Program Management Revised March 2006 U.S. Department of Homeland Security Preface Homeland Security Exercise and

More information

Bay Area Regional Catastrophic Planning Team Urban Shield 2013 Functional and Full Scale Exercise

Bay Area Regional Catastrophic Planning Team Urban Shield 2013 Functional and Full Scale Exercise Bay Area Regional Catastrophic Planning Team Urban Shield 2013 Functional and Full Scale Exercise Read-Ahead Package For April 25, 2013 Regional Catastrophic Plans BAY AREA UASI FUNCTIONAL EXERCISE OVERVIEW

More information

TEXAS HOMELAND SECURITY STRATEGIC PLAN 2015-2020: PRIORITY ACTIONS

TEXAS HOMELAND SECURITY STRATEGIC PLAN 2015-2020: PRIORITY ACTIONS TEXAS HOMELAND SECURITY STRATEGIC PLAN 2015-2020: PRIORITY ACTIONS INTRODUCTION The purpose of this document is to list the aligned with each in the Texas Homeland Security Strategic Plan 2015-2020 (THSSP).

More information

Flooding Emergency Response Exercise

Flooding Emergency Response Exercise Flooding Emergency Response Exercise James Woodward, Senior Exercise Planner California Emergency Management Agency 3650 Schriever Ave. Mather, CA 95655 Cell: (916) 439-3546 Email: james.woodward@calema.ca.gov

More information

For Official Use Only. Springfield-Greene County, Missouri Multi-Year Training and Exercise Plan 2016-2018 (TEP) July 27, 2015. For Official Use Only

For Official Use Only. Springfield-Greene County, Missouri Multi-Year Training and Exercise Plan 2016-2018 (TEP) July 27, 2015. For Official Use Only For Official Use Only Springfield-Greene County, Missouri Multi-Year Training and Exercise Plan 2016-2018 (TEP) July 27, 2015 For Official Use Only SPRINGFIELD-GREENE COUNTY Point of Contact Erin Pope

More information

DON T MUSSEL WITH IDAHO TABLE TOP EXERCISE PLAYERS HANDBOOK

DON T MUSSEL WITH IDAHO TABLE TOP EXERCISE PLAYERS HANDBOOK DON T MUSSEL WITH IDAHO TABLE TOP EXERCISE PLAYERS HANDBOOK PURPOSE The purpose of this handbook is to provide the exercise players with the information required for them to participate effectively in

More information

U.S. Department of Homeland Security Office for Domestic Preparedness 810 Seventh Street, NW. Washington, DC 20531

U.S. Department of Homeland Security Office for Domestic Preparedness 810 Seventh Street, NW. Washington, DC 20531 U.S. Department of Homeland Security Office for Domestic Preparedness 810 Seventh Street, NW. Washington, DC 20531 Tom Ridge Secretary Office for Domestic Preparedness World Wide Web Homepage: www.ojp.usdoj.gov/odp

More information

GUIDE TO DEVELOPING AND CONDUCTING BUSINESS CONTINUITY EXERCISES

GUIDE TO DEVELOPING AND CONDUCTING BUSINESS CONTINUITY EXERCISES GUIDE TO DEVELOPING AND CONDUCTING BUSINESS CONTINUITY EXERCISES ATLANTA, GEORGIA FEBRUARY 12, 2011 Table of Contents FOREWORD... ii 1.0 Introduction... 1 1.1. Purpose... 1 1.2 Organization... 1 2.0 Rehearsal,

More information

Public Health Coalition Exercise Request for Proposal (RFP)

Public Health Coalition Exercise Request for Proposal (RFP) December 3, 2014 Public Health Coalition Exercise Request for Proposal (RFP) SCOPE OF WORK The Metropolitan Chicago Healthcare Council is soliciting proposals from prospective consultants to develop Homeland

More information

STATE HOMELAND SECURITY GRANT PROGRAM

STATE HOMELAND SECURITY GRANT PROGRAM FY2015 State Homeland Security Investment Justifications STATE HOMELAND SECURITY GRANT PROGRAM OREGON OFFICE OF EMERGENCY MANAGEMENT www.oregon.gov/omd/oem Mailing address: P.O. Box 14370 Salem, OR 97309-5062

More information

Homeland Security Exercise and Evaluation Program

Homeland Security Exercise and Evaluation Program U.S. DEPARTMENT OF HOMELAND SECURITY Homeland Security Exercise and Evaluation Program Volume IV: Sample Exercise Documents and Formats Introduction Homeland Security Exercise and Evaluation Program (HSEEP)

More information

June 2015 Communications Full-Scale Exercise

June 2015 Communications Full-Scale Exercise June 2015 Communications Full-Scale Exercise Exercise Plan June 22-26, 2015 The Exercise Plan gives elected and appointed officials, observers, media personnel, and players from participating organizations

More information

Operation: Healthy Shelters

Operation: Healthy Shelters Operation: Healthy Shelters Exercise Plan October 29, 2014 The Exercise Plan gives elected and appointed officials, observers, media personnel, and players from participating organizations information

More information

Training and Exercise Plan Workshop User s Handbook. [Month Year]

Training and Exercise Plan Workshop User s Handbook. [Month Year] Training and Exercise Plan Workshop User s Handbook [Month Year] PREFACE The U.S. Department of Homeland Security (DHS) Fiscal Year (FY) 2006 Grant Guidance requires that any State or Urban Area (designated

More information

ST. JOHNS COUNTY COMPREHENSIVE EMERGENCY MANAGEMENT PLAN APRIL 2012. Appendix E. Training Program

ST. JOHNS COUNTY COMPREHENSIVE EMERGENCY MANAGEMENT PLAN APRIL 2012. Appendix E. Training Program ST. JOHNS COUNTY COMPREHENSIVE EMERGENCY MANAGEMENT PLAN APRIL 2012 Appendix E Training Program Appendix E Training - 1 I. PURPOSE St. Johns County Training Appendix To outline a training program that

More information

New Mexico Homeland Security and Emergency Management REQUEST TO USE FEDERAL GRANT FUNDS For Training, Conferences or Exercise Activities

New Mexico Homeland Security and Emergency Management REQUEST TO USE FEDERAL GRANT FUNDS For Training, Conferences or Exercise Activities New Mexico Homeland Security and Emergency Management REQUEST TO USE FEDERAL GRANT FUNDS For Training, Conferences or Exercise Activities NMDHSEM, State Administrative Agency (SAA) PO Box, 27111, Santa

More information

Minnesota Department of Health Homeland Security Exercise and Evaluation Program (HSEEP) Training Cards

Minnesota Department of Health Homeland Security Exercise and Evaluation Program (HSEEP) Training Cards Minnesota Department of Health Homeland Security Exercise and Evaluation Program (HSEEP) Training Cards Minnesota Department of Health Homeland Security Exercise and Evaluation Program (HSEEP) Training

More information

NEBRASKA STATE HOMELAND SECURITY STRATEGY

NEBRASKA STATE HOMELAND SECURITY STRATEGY NEBRASKA STATE HOMELAND SECURITY STRATEGY 2014-2016 Nebraska Homeland Security Policy Group/Senior Advisory Council This document provides an overall framework for what the State of Nebraska hopes to achieve

More information

August 10, 2009 August 9, 2010

August 10, 2009 August 9, 2010 August 10, 2009 August 9, 2010 Public Health Emergency Preparedness Cooperative Agreement Budget Period 10 (BP10) Performance Measures Guidance December 2009 Contact Information Key contacts for information

More information

PHILADELPHIA GAS WORKS Business Continuity Plan and Consulting RFP # 28576 Questions & Answers Emailed April 21, 2014

PHILADELPHIA GAS WORKS Business Continuity Plan and Consulting RFP # 28576 Questions & Answers Emailed April 21, 2014 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 QUESTIONS Under Task 1: It appears there are two tabletop exercises to be provided? Is there an estimate of how many personnel will take part in each exercise? Will PGW provide

More information

v. 03/03/2015 Page ii

v. 03/03/2015 Page ii The Trident University International (Trident) catalog consists of two parts: Policy Handbook and Academic Programs, which reflect current academic policies, procedures, program and degree offerings, course

More information

SAMPLE COURSE SYLLABUS*

SAMPLE COURSE SYLLABUS* DSS CENTER FOR DEVELOPMENT OF SECURITY EXCELLENCE (CDSE) EDUCATION DIVISION Managing a DoD Installation Security Program ED 507 SAMPLE COURSE SYLLABUS* A. DESCRIPTION The 2007 Defense Installations Plan

More information

HSEEP Mechanics Self-Paced Training Written Transcript

HSEEP Mechanics Self-Paced Training Written Transcript HSEEP Mechanics Self-Paced Training Written Transcript Module 1 Homeland Security Exercise and Evaluation Program HSEEP Mechanics Training (Image man turning gears, HSEEP: Plan, Train, Exercise & Evaluate

More information

UNLV Programs in Crisis and Emergency Management FALL, 2016 Admissions Information

UNLV Programs in Crisis and Emergency Management FALL, 2016 Admissions Information UNLV Programs in Crisis and Emergency Management FALL, 2016 Admissions Information Overview As a result of recent national, state, and local experiences such as September 11, 2001, and Hurricanes Katrina

More information

Security and Emergency Services Community of Interest 0089 Emergency Management Career Road Map

Security and Emergency Services Community of Interest 0089 Emergency Management Career Road Map Security and Emergency Services Community of Interest 0089 Emergency Management Career Road Map Prepared by: Booz Allen Hamilton Career Progression within the 0089 Occupational Series is Marked by Support

More information

Homeland Security Exercise and Evaluation Program. Volume II: Exercise Planning and Conduct

Homeland Security Exercise and Evaluation Program. Volume II: Exercise Planning and Conduct Homeland Security Exercise and Evaluation Program Volume II: Exercise Planning and Conduct Revised February 2007 This page is intentionally blank. Preface Homeland Security Exercise and Evaluation Program

More information

Plan Development and Review Guidance for local Emergency Operations Plans

Plan Development and Review Guidance for local Emergency Operations Plans Nancy J. Dragani, Executive Director Ohio Emergency Management Agency 2855 West Dublin-Granville Road Columbus, Ohio 43235-2206 www.ema.ohio.gov Plan Development and Review Guidance for local Emergency

More information

Hospital & Health Facility Emergency Exercise Guide

Hospital & Health Facility Emergency Exercise Guide Standardized Policy Method0logy Terminology Compliance Guidelines Hospital & Health Facility Emergency Exercise Guide Part 1 - The Table Top Exercise HSEEP Compliance Principles for Exercise Development,

More information

FY 2013 EMERGENCY MANAGEMENT PERFORMANCE GRANTS (EMPG) PROGRAM

FY 2013 EMERGENCY MANAGEMENT PERFORMANCE GRANTS (EMPG) PROGRAM OMB Control Number: 1660-0126 Public reporting burden for this form is estimated to average 3 hours per response. The burden estimate includes the time for reviewing instructions, searching existing data

More information

ShakeOut Exercise Manual For Hospitals

ShakeOut Exercise Manual For Hospitals Are You Ready to ShakeOut? Major earthquakes can cause unprecedented catastrophes. With earthquakes as an inevitable part of our future, hospitals should make plans and take actions to ensure that disasters

More information

Cornell University PREPAREDNESS PLAN

Cornell University PREPAREDNESS PLAN Cornell University PREPAREDNESS PLAN Table of Contents Table of Contents Section 1 EMERGENCY MANAGEMENT PROGRAM TRAINING, TESTING, AND EXERCISES...2 1.1 National Incident Management System Training...2

More information

EMERGENCY MANAGEMENT BRITISH COLUMBIA A STRATEGY TO ADVANCE SUPPORT FOR LOCAL AUTHORITY EMERGENCY MANAGEMENT PROGRAMS OCTOBER 14, 2015

EMERGENCY MANAGEMENT BRITISH COLUMBIA A STRATEGY TO ADVANCE SUPPORT FOR LOCAL AUTHORITY EMERGENCY MANAGEMENT PROGRAMS OCTOBER 14, 2015 EMERGENCY MANAGEMENT BRITISH COLUMBIA A STRATEGY TO ADVANCE SUPPORT FOR LOCAL AUTHORITY EMERGENCY MANAGEMENT PROGRAMS OCTOBER 14, 2015 1 MESSAGE FROM THE ASSISTANT DEPUTY MINISTER I am pleased to introduce

More information

HOMELAND SECURITY AND EMERGENCY MANAGEMENT AGENCY HSEMA (BN)

HOMELAND SECURITY AND EMERGENCY MANAGEMENT AGENCY HSEMA (BN) HOMELAND SECURITY AND EMERGENCY AGENCY HSEMA (BN) MISSION The mission of the District of Columbia s (HSEMA) is to manage the District s emergency operations to prevent, respond to and recover from natural

More information

Georgia Emergency Operations Plan. Emergency Support Function # 5 Annex Emergency Management

Georgia Emergency Operations Plan. Emergency Support Function # 5 Annex Emergency Management Emergency Support Function # 5 Annex Emergency Management 2015 Emergency Support Function #5 E S F C o o r d i nator and Support Ag e n c i e s ESF C oordi na t or Georgia Emergency Management Agency/Homeland

More information

A Framework to Support Healthcare Continuity of Operations in an Information Technology Failure:

A Framework to Support Healthcare Continuity of Operations in an Information Technology Failure: A Framework to Support Healthcare Continuity of Operations in an Information Technology Failure: Lessons learned from a novel exercise series Jendy Dunlop, MPH, CHEP Paul Biddinger, MD, FACEP http://001yourtranslationservice.com/computer-tips/protecting-your-computer.htm

More information

FY 2012 PERFORMANCE PLAN. Homeland Security and Emergency Management Agency

FY 2012 PERFORMANCE PLAN. Homeland Security and Emergency Management Agency FY 2012 PERFORMANCE PLAN District of Columbia MISSION The Mission of the District of Columbia is to support and coordinate homeland security and emergency management efforts, ensuring that the District

More information

DRAFT AFTER ACTION REPORT/IMPROVEMENT PLAN

DRAFT AFTER ACTION REPORT/IMPROVEMENT PLAN f[note for Template: Text found in this document that is highlighted and bracketed is included to provide instruction or to indicate a location to input text. All text that is not highlighted is to be

More information

V. TRAINING IN SCHOOL SECURITY

V. TRAINING IN SCHOOL SECURITY V. TRAINING IN SCHOOL SECURITY Security technology is a supplement to, but not a substitute for, the human factor. And if I had to pick between a well-trained, highly alert staff and student body versus

More information

MIKE EMERGENCY, CEM, MEP 860 Peachtree St NE #2206 Atlanta, GA 30308 Home: 404-380-1779 Work: 888-278-8149 E-mail: info@robinresumes.

MIKE EMERGENCY, CEM, MEP 860 Peachtree St NE #2206 Atlanta, GA 30308 Home: 404-380-1779 Work: 888-278-8149 E-mail: info@robinresumes. MIKE EMERGENCY, CEM, MEP 860 Peachtree St NE #2206 Atlanta, GA 30308 Home: 404-380-1779 Work: 888-278-8149 E-mail: info@robinresumes.com VACANCY ANNOUNCEMENT NUMBER: 123456 POSITION TITLE: Disaster Recovery

More information

HOMELAND SECURITY EXERCISE AND EVALUATION PROGRAM (HSEEP) CLASSROOM COURSE DELIVERY POLICY JUNE 26, 2014

HOMELAND SECURITY EXERCISE AND EVALUATION PROGRAM (HSEEP) CLASSROOM COURSE DELIVERY POLICY JUNE 26, 2014 HOMELAND SECURITY EXERCISE AND EVALUATION PROGRAM (HSEEP) CLASSROOM COURSE DELIVERY POLICY JUNE 26, 2014 PURPOSE The purpose of this policy is to provide guidance to local jurisdictions and state agencies

More information

HE MANTRA SAYS IT: Practice makes perfect.

HE MANTRA SAYS IT: Practice makes perfect. T HE MANTRA SAYS IT: Practice makes perfect. Perhaps nowhere is this more important than emergency preparation, and that s absolutely critical at airports. So we at Salt Lake City International Airport

More information

The Joint Commission Approach to Evaluation of Emergency Management New Standards

The Joint Commission Approach to Evaluation of Emergency Management New Standards The Joint Commission Approach to Evaluation of Emergency Management New Standards (Effective January 1, 2008) EC. 4.11 through EC. 4.18 Revised EC. 4.20 Emergency Management Drill Standard Lewis Soloff

More information

GRANT APPLICATION DETAILS and INSTRUCTIONS

GRANT APPLICATION DETAILS and INSTRUCTIONS STATE OF ARIZONA DEPARTMENT OF EDUCATION PILOT PROGRAM ON SCHOOL EMERGENCY READINESS FY 2016 COMPETITIVE GRANT ANNOUNCEMENT VIA SENATE BILL 1476 GRANT APPLICATION DETAILS and INSTRUCTIONS ELIGIBILITY Arizona

More information

Active Threat Scenario (ATS) RFP 1. INTRODUCTION

Active Threat Scenario (ATS) RFP 1. INTRODUCTION 1. INTRODUCTION The Cook County Department of Homeland Security and Emergency Management (DHSEM) plays a lead role in the homeland security and emergency management mission within the Chicago Urban Area;

More information

National Continuity Programs Directorate Integrated Public Alert and Warning (IPAWS) Fiscal Year 2013

National Continuity Programs Directorate Integrated Public Alert and Warning (IPAWS) Fiscal Year 2013 National Continuity Programs Directorate Integrated Public Alert and Warning (IPAWS) Fiscal Year 2013 Supplemental Guidance on Public Alert and Warning A Message to Stakeholders On behalf of the IPAWS

More information

TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION Disaster Recovery Testing Is Being Adequately Performed, but Problem Reporting and Tracking Can Be Improved May 3, 2012 Reference Number: 2012-20-041 This

More information

NIMS Study Guide. Lesson One: What Is the National Incident Management System (NIMS)? What is NIMS?

NIMS Study Guide. Lesson One: What Is the National Incident Management System (NIMS)? What is NIMS? NIMS Study Guide Lesson One: What Is the National Incident Management System (NIMS)? What is NIMS? NIMS is a comprehensive, national approach to incident management that is applicable at all jurisdictional

More information

CAPABILITY 3: Emergency Operations Coordination

CAPABILITY 3: Emergency Operations Coordination Emergency operations coordination is the ability to direct and support an event 38 or incident 39 with public health or medical implications by establishing a standardized, scalable system of oversight,

More information

DIVISION OF PUBLIC HEALTH DPH CONTRACT 24140 AMENDMENT 4

DIVISION OF PUBLIC HEALTH DPH CONTRACT 24140 AMENDMENT 4 DIVISION OF PUBLIC HEALTH DPH CONTRACT 24140 AMENDMENT 4 The Department of Health Services, on behalf of the Division of Public Health and Brown County Health Department agree to amend their original agreement

More information

Project Management Plan

Project Management Plan Project Management Plan FY 2009 UASI Project Information Project Title Subgrantee Subgrant GIS Data Exchange District of Columbia [to be generated by SAA] Subgrant Award $600,000.00 Jurisdiction (if applicable)

More information

Exercise Plan PROJECT ESCAPE U.S. DEPARTMENT OF HOMELAND SECURITY. Exercise Date: 04/26/12 Publishing Date: 04/19/12

Exercise Plan PROJECT ESCAPE U.S. DEPARTMENT OF HOMELAND SECURITY. Exercise Date: 04/26/12 Publishing Date: 04/19/12 PROJECT ESCAPE U.S. DEPARTMENT OF HOMELAND SECURITY Exercise Date: 04/26/12 Publishing Date: 04/19/12 PREFACE The Project Escape is jointly sponsored by Robert Wood University Hospital Hamilton, Hamilton

More information

Business Continuity Exercise Program (BCEP) Information Packet

Business Continuity Exercise Program (BCEP) Information Packet BUSINESS CONTINUITY PLANNING Business Continuity Exercise Program (BCEP) Information Packet This exercise program was developed by the Texas Engineering Extension Service (TEEX) A Member of the Texas A&M

More information

NORTH CAROLINA EMERGENCY MANAGEMENT CERTIFICATION PROGRAM

NORTH CAROLINA EMERGENCY MANAGEMENT CERTIFICATION PROGRAM NORTH CAROLINA EMERGENCY MANAGEMENT CERTIFICATION PROGRAM October 15 th, 2013 Table of Contents Program Overview and Administration:...4 Certification process:...4 Responsibilities:...4 North Carolina

More information

Business Continuity Management Program Development Guide

Business Continuity Management Program Development Guide Business Continuity Management Program Development Guide Prepared by The NS Emergency Management Office, Winter 2012 Version 1.1 Page 2 of 24 Document Revision History Date Author Revision Notes Fall 2011

More information

National Exercise Program (NEP) and the Homeland Security Exercise and Evaluation Program (HSEEP)

National Exercise Program (NEP) and the Homeland Security Exercise and Evaluation Program (HSEEP) National Program (NEP) and the Homeland Security and Evaluation Program (HSEEP) NEP and HSEEP: Interagency Strategy (NRT/RRT Co-Chairs Meeting) James Kish, Senior Director, and Evaluation Division National

More information

How To Write A Book On Risk Management

How To Write A Book On Risk Management National Center for Risk and Economic Analysis of Terrorism Events CREATE FY2015 (Year 11) Call for White Papers CREATE, the DHS-sponsored Center of Excellence at the University of Southern California,

More information

The University of Southern Mississippi National Center for Spectator Sports Safety and Security. Sport Event Security Aware Designation

The University of Southern Mississippi National Center for Spectator Sports Safety and Security. Sport Event Security Aware Designation The University of Southern Mississippi National Center for Spectator Sports Safety and Security Sport Event Security Aware Designation Frequently Asked Questions September 2015 What is Sport Event Security

More information

What you will learn..

What you will learn.. What you will learn.. The Exercise Program! Copyright November 2007 Updated January 2009 Acknowledgements: This guide was made possible by the collective and collaborative efforts of the National Emergency

More information

Business Continuity Position Description

Business Continuity Position Description Position Description February 9, 2015 Position Description February 9, 2015 Page i Table of Contents General Characteristics... 2 Career Path... 3 Explanation of Proficiency Level Definitions... 8 Summary

More information

Guide for Developing High- Quality Emergency Operations Plans for Houses of Worship

Guide for Developing High- Quality Emergency Operations Plans for Houses of Worship Guide for Developing High- Quality Emergency Operations Plans for Houses of Worship June 2013 Table of Contents Introduction and Purpose... 1 Planning Principles... 3 The Planning Process... 4 Step 1:

More information

Flu at the Zoo II AAR/IP

Flu at the Zoo II AAR/IP AAR/IP Improvement Plan 10/15/ The After-Action Report/Improvement Plan (AAR/IP) aligns exercise objectives with preparedness doctrine to include the National Preparedness Goal and related frameworks and

More information

COUNTY OF COOK. Characteristics of the Position

COUNTY OF COOK. Characteristics of the Position COUNTY OF COOK Bureau of Human Resources 118 N. Clark Street, Room 840 Chicago IL 60602 Job Code: 0112 Job Title: Director of Financial Control III Salary Grade: 23 Bureau: Office of the President Department:

More information

Emergency Management

Emergency Management 23 Classes Offered Earthquake: An Introduction to Emergency Disaster Planning Disaster Mitigation Disaster Recovery Disaster Medical Managment Managing Sustained Operations Developing a Local Multi-Hazard

More information

Emergency Support Function (ESF) #5. Emergency Management

Emergency Support Function (ESF) #5. Emergency Management Emergency Support Function (ESF) #5 Emergency Management Lead Agency: Support Agencies: Cleburne County Emergency Management Agency Cleburne County Commission I. Introduction A. Purpose B. Scope ESF #5

More information

COUNTY OF COOK. Characteristics of the Position

COUNTY OF COOK. Characteristics of the Position COUNTY OF COOK Bureau of Human Resources 118 N. Clark Street, Room 840 Chicago IL 60602 Job Code: 0112 Job Title: Director of Financial Control III Salary Grade: 23 Bureau: Office of the President Department:

More information

Program Outline & Accreditation Application

Program Outline & Accreditation Application Program Outline & Accreditation Application The field of emergency management is emerging into higher visibility in communities throughout the nation as they are victimized by disasters that are increasing

More information

U.S. Department of Homeland Security Office for Domestic Preparedness 810 Seventh Street, NW. Washington, DC 20531

U.S. Department of Homeland Security Office for Domestic Preparedness 810 Seventh Street, NW. Washington, DC 20531 U.S. Department of Homeland Security Office for Domestic Preparedness 810 Seventh Street, NW. Washington, DC 20531 Tom Ridge Secretary Office for Domestic Preparedness World Wide Web Homepage: www.ojp.usdoj.gov/odp

More information

Emergency Preparedness Training and Exercise Guide for Nursing Homes

Emergency Preparedness Training and Exercise Guide for Nursing Homes Emergency Preparedness Training and Exercise Guide for Nursing Homes July 2008 ACKNOWLEDGEMENTS The federal Department of Homeland Security Exercise and Evaluation Program (HSEEP) provides specific standards

More information

This page intentionally left blank.

This page intentionally left blank. This page intentionally left blank. This page intentionally left blank. CONTENTS List of Tables...vii List of Figures...vii What Is the National Incident Management System?...1 PREFACE... 3 INTRODUCTION

More information

Ohio Homeland Security Strategic Plan 2013-2016

Ohio Homeland Security Strategic Plan 2013-2016 GOAL 1 Strengthen Ohio s intelligence and information sharing system for the detection and prevention of threats to public safety. Objective 1.1 Support continued development of the information sharing

More information

Following are detailed competencies which are addressed to various extents in coursework, field training and the integrative project.

Following are detailed competencies which are addressed to various extents in coursework, field training and the integrative project. MPH Epidemiology Following are detailed competencies which are addressed to various extents in coursework, field training and the integrative project. Biostatistics Describe the roles biostatistics serves

More information

Overview of Homeland Security Funding 1999 to Present National Incident Management System Mandates and Training Requirements

Overview of Homeland Security Funding 1999 to Present National Incident Management System Mandates and Training Requirements Overview of Homeland Security Funding 1999 to Present National Incident Management System Mandates and Training Requirements Jim Weldin Delaware League of Local Governments 1 Homeland Security Grant Funding

More information

SECURITY-BASED TABLETOP EXERCISE

SECURITY-BASED TABLETOP EXERCISE SECURITY-BASED TABLETOP EXERCISE USER GUIDE 6/23/2015 Page 1 of 6 INTRODUCTION Emergency incidents are not common occurrences, but the timely and appropriate response to an incident is critical to the

More information

GAO HOMELAND DEFENSE. U.S. Northern Command Has a Strong Exercise Program, but Involvement of Interagency Partners and States Can Be Improved

GAO HOMELAND DEFENSE. U.S. Northern Command Has a Strong Exercise Program, but Involvement of Interagency Partners and States Can Be Improved GAO United States Government Accountability Office Report to Congressional Requesters September 2009 HOMELAND DEFENSE U.S. Northern Command Has a Strong Exercise Program, but Involvement of Interagency

More information

OCCUPATIONAL GROUP: Public Safety. CLASS FAMILY: Emergency Management CLASS FAMILY DESCRIPTION:

OCCUPATIONAL GROUP: Public Safety. CLASS FAMILY: Emergency Management CLASS FAMILY DESCRIPTION: OCCUPATIONAL GROUP: Public Safety CLASS FAMILY: Emergency Management CLASS FAMILY DESCRIPTION: This family of positions includes those whose purpose is to plan, coordinate, and implement emergency preparedness

More information

Recent Changes to Emergency Preparedness Mandates and Funding

Recent Changes to Emergency Preparedness Mandates and Funding National Association of Public Hospitals and Health Systems MAY 2008 1301 Pennsylvania Avenue, NW, Suite 950 Washington, DC 20004 202 585 0100 tel / 202 585 0101 fax www.naph.org Larry S. Gage / president

More information

Essential Components of Emergency Management Plans at Community Health Centers Crosswalk of Plan Elements

Essential Components of Emergency Management Plans at Community Health Centers Crosswalk of Plan Elements Plan Components Health centers will have an emergency management plan Plan and organization are NIMS compliant Bureau of Primary Health Care Policy Information Notice 2007-15 Plans and procedures for emergency

More information

Performance Audit Report

Performance Audit Report Performance Audit Report The Small Business Administration Did Not Effectively Assess Disaster Assistance Staffing Requirements, Availability, and Readiness January 25, 2013 Report Number 13-10 U.S. Small

More information

Academic Program Review Handbook

Academic Program Review Handbook Handbook Continuously Improving Programs and Student Learning Revised July 2014 Original Issue: December 6, 2010 Approved: Derry Connolly, President Current Issue: July 3, 2014 Effective: July 3, 2014

More information

Lesson 1: What Is the National Incident Management System (NIMS)? Summary of Lesson Content

Lesson 1: What Is the National Incident Management System (NIMS)? Summary of Lesson Content Lesson 1: What Is the National Incident Management System (NIMS)? Lesson Overview On February 28, 2003, President Bush issued Homeland Security Presidential Directive 5. HSPD 5 directed the Secretary of

More information

Cyber Intrusions: More than an IT Challenge

Cyber Intrusions: More than an IT Challenge Cyber Intrusions: More than an IT Challenge ICSJWG Fall Meeting October 16, 2012 Presented by: Bridgette Walsh DHS Cyber Exercise Program (CEP) State of the World: Cyber Realities Increasing frequency,

More information