Analysis of Cybersecurity Content in the Air Traffic Collegiate Training Initiative (AT CTI) Program

Transcription

1 Analysis of Cybersecurity Content in the Air Traffic Collegiate Training Initiative (AT CTI) Program Juan Lopez Jr. MS & Deanne W. Otto, PhD 1 Abstract The aim of the study was to assess the existence and integration level of cybersecurity content in educational programs from institutions participating in the Air Traffic Collegiate Training Initiative (AT CTI). The curricula of 36 post secondary academic institutions were reviewed to ascertain the inclusion of cybersecurity content. Program catalogs, course syllabi, and descriptive course narratives were subjected to content analysis for manifest variables. Results indicate that cybersecurity content is almost non existent in the curriculum of institutions participating in the program. The massive overhaul of the Federal Aviation Administration s (FAA) National Airspace System (NAS) to the Next Generation Air Transportation System (NextGen) warrants consideration of integrating cyber oriented content in AT CTI curriculum. Recommended strategies to provide relevant cyber oriented content tailored to the Air traffic Control community from participating institutions designated as Centers of Academic Excellence in Information Assurance are discussed. Introduction: Managing the cybersecurity of information and communications systems for critical infrastructure and key resources (CIKR) has become an increasingly complex endeavor. Protecting and ensuring the continuity of the CIKR of the United States is essential to the Nation s security, public health and safety, economic vitality, and our way of life (Chertoff, 2009). National critical infrastructure systems such as the power grid and transportation systems are also increasingly becoming interconnected to enterprise 1 Juan Lopez, Jr. Holds an MS in Information and Telecommunications Systems Management from Capitol College and a MS in Information Resource Management focused on Information Assurance and Computer Networks from the Air Force Institute of Technology. He is currently pursuing a PhD in Computer Science from the Air Force Institute of Technology where he is also a Cyberspace research engineer contractor. Deanne W. Otto holds a PhD in Teaching & Learning with emphases in Research Methodologies & Statistics, Education Foundations, and Educational Testing & Measurement from the University of North Dakota. Dr. Otto currently serves as a research engineer contractor supporting the Air Force Institute of Technology s Center for Cyberspace Research. 1

2 networks to achieve efficiencies, improve system performance, improve productivity, and increase safety through automation. While the benefits have been enormous, this widespread interconnectivity also poses significant risks to our nation's computer systems and, more importantly, to the critical operations and infrastructures they support (GAO, 2005). Federal and industry experts believe that critical infrastructure control systems are more vulnerable to cyber threats today than in the past (Powner & Rhodes, 2007; Baker, Waterman & Ivanov, 2010; Baker, Filipiak & Timlin, 2011). The transportation systems sector, and more specifically the aviation subsector, is undergoing tremendous change in systems automation with the implementation of the Next Generation Air Transportation System (NextGen). Critical Infrastructure system owners and stakeholders in the aviation subsector are finding it increasingly difficult to make informed cyber security decisions as the drive to increase automation in aviation systems continues. The reason for the increased difficulty in decision making is the aviation subsector s expectation that NextGen will safely and efficiently accommodate the large growth of air traffic expected in the near future. NextGen should be sufficiently scalable to contend with a two fold or more increase in demand expected by the year 2020 (Ezberger, 2004). According to the FAA NextGen implementation plan (2012), NextGen will enhance safety, reduce delays, save fuel and reduce aviation s adverse environmental impact (Huerta, 2012). The transformation of NextGen consists of six core programs, (1) Automatic Dependent Surveillance Broadcast (ADS B), (2) Data Communications (Data Comm), (3) System Wide Information Management (SWIM), (4) Common Support Service Weather (CSS Wx), (5) National Airspace System (NAS) Voice System (NVS), and (6) Collaborative Air Traffic Management Technologies (CATMT). The introduction of technology, particularly the use of computer technology, has changed the way that workers perform their tasks and the types of training and skills that workers need in order to complete these tasks. Research has shown that the use of computers has eliminated the need for humans to perform tasks that involve solving routine problems or communicating straightforward information (Autor, Levy & Murnane, 2003; Levy & Murnane, 2004). Automation is expected to relieve air traffic controllers of routine monitoring and control tasks so they can devote more time to solving strategic control problems, managing traffic flows during changing weather conditions and handling other unusual events (Ezberger, 2004). Planned NextGen automation system improvements will provide air traffic controllers with greater decision making tools, while digital information sharing is helping aircraft operators, controllers and traffic managers to work together to maximize efficiency in the air and on the ground (Huerta, 2012). Nevertheless, automation has solved old problems but ultimately caused new and different types of accidents (Chialastri, n.d.). According to Robert Scott (cited in Flightglobal, 2012): 2

3 The intellectual and physical skills once required of the pilot have largely been replaced by an emphasis on 'soft skills' and automation management. The pilot who once cynically challenged sources of information now readily accepts information from a variety of sources, many computergenerated, without question. We should expect that Air Traffic Controllers could potentially experience a similar deterioration of skills due to increased automation and information richness as a consequence of the transition to NextGen. Various studies on aviation safety continue to focus on identifying inconsistencies to improve factors that contribute to aviation mishaps. A key finding by the Government Accounting Office (2002) with a potential to be amplified as a consequence of NextGen was the communication processes between the FAA and the military services. Most notable were existing formal networks of communication that have not always been sufficient to ensure the comprehensive exchange of all critical aviation safety information. Specifically, they found (1) a lack of a common definition of what constitutes an aviation safety hazard of mutual concern, and (2) gaps in the formal communication processes which caused delays in bringing critical safety information to the attention of key officials. Most alarming is the degree of institutionalized informal information exchange that is currently sustained largely by personnel rather than formal agency/service relationships, which the GAO concluded is therefore vulnerable to the retirement of key aviation safety personnel and senior leaders. The informal networks can be expected to experience considerably more disruption from adoption of NextGen information exchanges. It appears that a significant portion of the information exchange paths were personality driven and based on tacit knowledge. The rapid replacement rate expected over the next 5 10 years of an aging workforce coupled with a more cyber oriented and virtually astute employees could skew the information path patterns in unexpected ways with undesirable consequences if not proactively and deliberately managed. There also exist the potential to realize unexpected operational benefits from information rich environments. At a minimum, senior leaders and key stakeholders should plan for and develop strategies to mitigate inconsistencies during the transition that could otherwise be masked from the dilution of human derived information paths caused by the overlay of virtual information paths created by NextGen. The National Research Council (1998) continues to emphasize that efforts to modernize and further automate the air traffic controller system should not compromise safety by marginalizing the human controller s ability to effectively monitor the process, intervene as spot failures occur in the software or environmental disturbances require, or assume manual control if the automation becomes untrustworthy. However, the threat from 3

4 cyber induced events (malicious or inadvertent) can potentially further complicate the decision making process particularly when a cyber threat may mimic normal airport operations (McCallie, Butts, & Mills, 2011). Cybersecurity has become an interdisciplinary issue for many organizations. Any cybersecurity initiative must account for technology, operations, and people. If, in fact, any of those three measures are not accounted for, systems become immediately vulnerable (Maconachy, Schou, Ragsdale, & Welch, 2001). An essential element of cybersecurity has always been the people portion of the triangle. Many organizations have learned that the human in the loop, the proverbial human firewall of a system, must be adequately trained and educated to serve as an effective tool against cyber induced events (von Solms & Warren, 2011). However, a message that has continued to propagate with increasing frequency is that there appears to be an acute shortage of Internet security experts in the government with no large pool of applicants waiting in the wings to join the fight (Beidel & Magnuson, 2011). Critical infrastructure sectors are currently experiencing substantial cybersecurity challenges. A variety of tangential and fundamental issues make implementing robust cybersecurity measures in critical infrastructure a challenge. Some of the most prominent issues are information sharing (GAO, 2012), improving the management of security surveys and vulnerability assessments of CIKR asset owners and operators (GAO, 2012), and public private partnerships (Cavelty, & Suter, 2009). In 2007, the Government Accounting Office recognized that while DHS has initiatives under way to fulfill its many cybersecurity responsibilities, major tasks remain to be done. These include assessing and reducing cyber threats and vulnerabilities and coordinating incident response and recovery planning efforts. However, the human capital talent required to meet these demands are still not being produced in sufficient quantity and quality. There is a recognized and persistent shortage of cybersecurity experts (current and projected) in both the public and private sectors (Finkle & Randewich, 2012; Beidel & Magnuson, 2011). In 2009, the Department of Homeland Security (DHS) publically announced they were seeking to hire 1,000 cybersecurity experts during the time period when the U.S. Cyber Command was being established (Krebs, 2009).DHS has fallen very short of that number and readjusted their hiring target to less than half the initial estimate. In 2012, DHS established a cybersecurity recruitment and development program to institutionalize cybersecurity and recruit the appropriate talent. Secretary Napolitano s intent is to expand the national pipeline of men and women with advanced cybersecurity skills, and also enable DHS to become a preferred employer for the talent produced by that pipeline. The three programs included are: 4

5 Cybersecurity Internship Program is a 10 week program designed for current undergraduate students during the summer between their junior and senior years in college Secretary's Honors Program for Cybersecurity Professionals is a two year program for recent college graduates designed to develop technically skilled cyber professionals across the U.S. Department of Homeland Security Emerging Leaders in Cybersecurity Fellowship is a two year program for recent graduates with a master's degree and is designed to develop the next generation of cyber leaders within the U.S. Department of Homeland Security The critical shortage in cybersecurity expertise has prompted several new business models to emerge in the private sector as well. For example, the National Security Agency (NSA) partnered with academic institutions with a primary goal of exposing students to the scientific and intellectual foundation of cyber operations, giving them a glimpse of how such knowledge can be applied in innovative cyber careers with the government (NSA, 2012). Graduates of the program are expected to pursue careers in cyber operations with the NSA and other similar agencies. Similarly, Northrop Grumman partnered with the University of Maryland in hopes of providing them and other technical firms with an annual crop of graduates in cybersecurity (Overly, 2012) Although these initiatives are designed to develop the cybersecurity talent needed in several critical infrastructure sectors, they are clearly not focused on cybersecurity skills needed by the NextGen air traffic control community. The FAA recognizes that their sociotechnical environment is in the midst of a rapid transformation. According to Huerta (2011), Our information systems are taking a quantum leap from the days of closed legacy networks that were created before the Internet became a household word. We are evolving towards entire networks of networks that use internet protocols. And we are moving our entire national air space system into the Next Generation. We are going from closed computer systems to open systems. We have to do this in order to implement NextGen and the advantages of GPS based navigation such as better terrain awareness, streamlined routes and better weather data.with that evolution from closed systems to open systems the cyber security risks will increase. We know this. The FAA has made substantial progress in raising the cybersecurity literacy of their workforce that interacts with traditional computer information systems (e.g. web browser, electronic mail, and business transaction systems). More than 96 percent of 5

6 FAA employees completed their cyber security [sic] awareness training last year. Nearly 10,000 FAA contract support personnel also completed their annual security and privacy awareness training (Huerta, 2011). The progress achieved across the FAA is commendable. However, the training only addresses the cybersecurity literacy for nonair traffic control related systems. Not addressed is the cybersecurity literacy needs of the air traffic controller workforce operating, maintaining, and interacting with NextGen systems that are highly reliant on increased automation (Huerta, 2011). Although increased reliance on automation is expected to achieve efficiency gains and improve air safety of the NAS, over reliance on automation for error detection by humans can degrade below detection thresholds for non automation dependent operators (Sanchez, Krumbholz, & Silva, 2009; Skitka, Mosier, & Burdick, 1999). The Automation Paradox is well documented in the literature and is out of scope for discussion in this study. However, what warrants concern in the context presented herein is the degree to which air traffic control system reliability can be diminished by cyber means (i.e. cyber attack). Research and development opportunities to investigate this phenomenon should be deliberately planned into the research life cycle of the FAA. In order to promote research synergy, the government accounting office has recommended that NASA and the FAA combine research and development efforts. The GAO (2005) recognized that: The human is the most important element in the air transportation system and must be considered first and foremost in the design of improvements to it. Humans design, develop, deploy, manage, maintain, and operate all elements of the system. Yet, the human component is often inadequately considered when improvements are made to the system. Efforts between the FAA and NASA to establish a research partnership demonstrates a concerted effort to generate synergy in this area. Table 1 provides an excerpt of the list of initial human factors research that was derived from common internal research areas of both agencies (FAA & NASA, 2011). The reader is encouraged to review a more complete narrative for each research area in Appendix A of the reference. The FAA is mainly responsible for R&D to support near term implementation and midterm implementation, while NASA conducts much of the research to address far term implementation (GAO, 2010). However, the list does not explicitly identify human factors research with regard to cybersecurity, near term or long term. 6

7 Focus Area Workforce development Table 1. FAA & NASA initial research focus areas (FAA & NASA, 2011) Description Workforce selection and training research helps to identify new skills, knowledge, and abilities that will be needed by NextGen controllers, maintainers, and pilots. The primary focus is on the changing role of NAS operators in the NextGen environment. Workstation requirements Safety assessment and management of risks Ground operations optimization Traffic flow management and collaboration Human interaction with technology and automation Workstation functional and design requirements research focuses on improving the operations of air traffic controllers and technical operations to ensure the envelope of human performance is considered as part of safety, capacity, and performance goals. Safety risk research develops and evaluates new methods that allow credible safety evaluations of highly complex, automation intensive systems. In addition, risk and error management research investigates opportunities for pilot error with the introduction of NextGen automated systems and procedures and the associated pilot roles and responsibilities. Ground operations research addresses pilot and controller situational awareness in the use of NextGen capabilities during imprecise surface movements for improved operations in all visibility conditions. Traffic flow research develops strategies and DSTs that allow traffic flow managers to minimize disruptions caused by hazardous weather and to enhance CDM. NextGen technologies encompass research on data communications and human computer interfaces for NAS actors in the NextGen environment. It produces human factors data to determine minimum required data communications display characteristics and flight crew procedures to achieve the desired performance. 7

8 Human automation functional allocation Higher traffic levels and TBO will likely cause automation to take on a greater role in the NextGen environment. Humans, however, will continue to retain a prominent or leading role in many functions. Research in this area focuses on the proper allocation of functions to humans and automated systems, as well as human / machine roles and responsibilities within those functions. Procedure design and training Trajectory based operations NextGen implementation research centers on instrument procedures, training and personnel qualification, and single pilot research. It focuses on the special skills and knowledge needed by NAS actors to ensure the envelope of human performance considerations is adequately aligned with the development of NextGen capabilities in order to accrue intended benefits. NextGen operations research involves separation assurance and collision avoidance as well as trajectory based operations. It involves new operations under NextGen, such as pilot negotiation, selection, implementation, and monitoring of aircraft state relative to position, altitude, and time. This research explores greater levels of automation support to help mitigate controller workload. Air/ground integration Air/ground integration research provides a systems approach to air ground integration as the NAS transitions from current operations to new concepts, taking into account changes in responsibilities and procedures between controllers and pilots. NextGen will require advanced concepts and technologies, along with higher levels of automation all of which will result in changes to roles and responsibilities for pilots and air traffic controllers. These transitions, in combination with increased interaction with automation, can lead to unwanted side effects, such as increased errors, loss of situational awareness, or mode confusion (FAA & NASA, 2012). Absent from the FAA and NASA research implementation plan for NextGen is the lack of an explicit focus on cybersecurity. What should draw cause for concern is the increased risk of cyber induced events of NextGen systems and the effects on air traffic control operations. Introduction of reliable automation to the Air Traffic Control (ATC) environment should reduce task load demands on the controller (Kopardekar, Prevot, and Jastrzebski, 2009; Metzger and Parasuraman, 2005). With reliable automation and an intuitive user 8

9 interface concept, one controller should be able to effectively manage more aircraft than in today s ATC environment. While the work load will increase the human element will remain to verify and confirm air traffic requirements and provide the knowledge needed to make judgment calls. The FAA and NASA (2012) clearly recognize the contrast. The initial focus was on the tower cab working environment. Results showed that new capabilities will supplement human perception, but controllers will remain responsible for the same major job functions they currently perform and are likely to continue relying upon their own perceptions and judgments, perhaps looking to the automation to confirm a course of action (p.18). Increased reliance on automation is an integral consequence of the transition to advanced technology. What must not be masked or allowed to be diluted are the inherent attributes of cybersecurity principles that become a significant influence in the human machine interface. Essentially, the increased automation and system integration are carried over perhaps transparently and increase the cybersecurity risk to the NAS as a byproduct of human machine interaction. Recognizing the potential issues related to this topic, the following questions surface. How can this gap be addressed? What cybersecurity education is appropriate for programs that currently exist for air traffic controllers? What are the emerging cyber threats that an air traffic controller can be expected to confront in NextGen systems? The intersection of these events warrants consideration of the cybersecurity education being offered to air traffic controllers and air traffic management personnel beyond traditional cybersecurity awareness. This study scrutinizes the educational content of AT CTI programs for the presence of cybersecurity content. The study includes a discussion of the implications for practitioners and a prescriptive approach for addressing the cybersecurity education requirements for institutions participating in the AT CTI program. National Initiative for Cybersecurity Education (NICE): The NICE framework is a comprehensive catalog that lists hundreds of cybersecurity tasks and skills, organized by categories and specialty areas. The intention of the framework is to describe cybersecurity work regardless of organizational structures, job titles, or other potentially idiosyncratic conventions (Duffie, 2011). The national program manager for NICE at the National Institute of Standards and Technology (NIST), Dr. Ernest McDuffie, encouraged the DHS Task Force to take on the specific adaptation of the NICE framework for mission critical roles, expressing the sentiment that it was much needed (cited in HSAC, 2011). The FAA should consider taking the same 9

10 approach as the DHS task force to extend the cybersecurity tasks and skills commensurate with a wide breadth of aviation occupations. The FAA should leverage what has already been developed to shorten the learning curve and adopt an already existing framework that would expedite adoption and policy development. The framework can also be used to guide curriculum mapping and as the basis for curriculum development of cybersecurity content specific to the needs of the air traffic control community. Huerta (2011) encourages new ideas from the community at large We need you in the game with your best ideas as to how we secure NextGen systems, how we operate our infrastructure and how we deal with a continually evolving cyber threat. Methodology: Content analysis was used to examine the existence and integration level of cybersecurity content in educational programs from institutions participating in the Air Traffic Collegiate Training Initiative (AT CTI). The FAA has agreements with institutions in the United States that have included curricula that are considered to contain the fundamentals of aviation within the degree programs (FAA, 2012). Currently, there are 36 institutions actively participating in the AT CTI program with one additional institution Western Michigan University that will be enrolling students in the near future. Program catalogs, course syllabi, and descriptive course narratives were subjected to computer assisted content analysis for manifest variables of cybersecurity principles. The principles defined by the NIST s Generally Accepted Principles and Practices for Securing Information Technology Systems (Swanson & Guttman, 1996) provided the theoretical framework that guided the qualitative scrutiny to which the content were subjected. According to Swanson and Guttman, certain intrinsic expectations must be met whether the system is small or large or owned by a government agency or by a private corporation. The sampling frame consisted of the entire list of educational institutions (N = 36) participating in the AT CTI program as of May 17, The list of participating institutions is provided in Table 1. The units of data collection are the (1) course syllabi containing the course name and a course description and, (2) program catalog with an associated descriptive narrative. All content reviewed were in digital form and publically accessible from the World Wide Web. Only web content that was date stamped between the periods of January 1, 2008 to September 30, 2012 were considered for analysis. Two coders were utilized to perform the analysis. Inter rater agreement was determined using Cohen s Kappa. The kappa coefficient (K) measures pairwise agreement among a set of coders making category judgments, correcting for expected chance agreement (Carletta, 1996). The inter rater reliability measure was high (K =.71). A measure in the range of.61 to.80 is considered substantial (Landis & Koch, 1977). 10

11 Manifest content that indicate the presence of a cybersecurity principle were measured as a nominal variable and labeled (1) affirmative or (2) negative. Manifest content that indicate degree of integration in the program was based on the degree of freedom in course selection available to the student. The level of measurement is ordinal and labeled as (1) Very High = required core course, (2) Moderate = required elective course, (3) Low = optional elective course, or (4) None = course is non existent in the program. This approach results in a matrix of mutually exclusive categories to code the manifest variables and account for every outcome evaluated. 11

12 Table 1. List of approved AT CTI institutions (alphabetical order) Institution Aims Community College Arizona State University Broward College Community College of Beaver County Daniel Webster College Dowling College Eastern New Mexico University Roswell Embry Riddle Aeronautical University Daytona Embry Riddle Aeronautical University Prescott Florida Institute of Technology College of Aeronautics Florida State College at Jacksonville Green River Community College Hampton University Hesston College InterAmerican University of Puerto Rico Jacksonville University Kent State University LeToureanu University Lewis University Metropolitan State University of Denver Miami Dade College Middle Georgia College Middle Tennessee State University Minneapolis Community Technical College Mount San Antonio College Purdue University Sacramento City College St. Cloud State University The Community College of Baltimore County Texas State Technical College Waco Tulsa Community College University of Alaska Anchorage University of North Dakota University of Oklahoma Vaughn College of Aeronautics and Technology Western Michigan University 2 Location Greely, CO Mesa, AZ Pembroke Pines, FL Beaver Falls, PA Nashua, NH Shirley, NY Roswell, NM Daytona Beach, FL Prescott, AZ Melbourne, FL Jacksonville, FL Auburn, WA Hampton, VA Hesston, KS Bayamón, PR Jacksonville, FL Kent, OH Longview, TX Romeoville, IL Denver, CO Homestead, FL Cochran, GA Murfreesboro, TN Eden Prairie, MN Walnut, CA West Lafayette, IN Sacramento, CA St. Cloud, MN Baltimore, MD Waco, TX Tulsa, OK Anchorage, AK Grand fork, ND Norman, OK Flushing, NY Battle Creek, MI 2 Institution is pending final FAA approval to offer the listed courses in the program. 12

13 Analysis: Most of the institutions provide a basic introduction to computing course which is typically focused on the functional use of productivity applications (e.g. word processing, spreadsheet application, and graphics presentation) use of electronic mail, and Internet browser. Some curriculum provided courses that included the mention of Internet security with regard to personal privacy. Virtually no institutions indicated a course specifically containing the language of cybersecurity in the title or the course description. Further analysis of catalog descriptive narratives for each program indicated no specific inclusion of cybersecurity content within the AT CTI curriculum. There exists a strong trend across the entire program that indicates a lack of cybersecurity integration across all aspects of the ATC curriculum. While it is important to educate the next generation of cyber leaders, significant resources are already in place with National Center of Academic Excellence in Information Assurance Education (CAE/IAE), National Center of Academic Excellence in Information Assurance Research (CAE/R), or National Center of Academic Excellence in Information Assurance 2 Year Education (CAE/2Y) (NSA, 2012). In addition, STEM fields of study in the categories of Science, Technology, Engineering, and Mathematics at colleges and universities, as well as STEM high school programs, and the infrastructure for successful implementation of cybersecurity education already exists. The NICE framework can be leveraged to guide curriculum mapping across the entire AT CTI program. It is worth noting that nearly 5% of the institutions on the AT CTI list have the resident expertise and capability to start offering cybersecurity courses by virtue of being designated as an NSA National Center of Academic Excellence in Information Assurance Education (CAE/IAE), or CAE in Information Assurance Research (CAE/R), or CAE in Information Assurance 2 Year Education (CAE/2Y). Table 2 provides a list of CAE designated institutions that are also an approved AT CTI institution. Table 2. CAE/IAE institutions that are also AT CTI approved Institution Arizona State University Florida Institute of Technology Hampton University Lewis University Minneapolis Community and Technical College Purdue University St. Cloud State University The Community College of Baltimore County Designation CAE/IAE, CAE/R CAE/R CAE/IAE CAE/IAE CAE/2Y CAE/R CAE/IAE CAE/2Y Each program has elective course requirements. A cybersecurity course that exposes the student to principles of cybersecurity would be a good starting point early in the career 13

14 path of an Air Traffic Controller. A deliberate effort should be undertaken to customize and shape the cybersecurity curriculum content to meet the emerging needs of NextGen Air traffic controllers and aviation management. Content analysis examined the presence of cybersecurity principles in each of eight key areas. Content analysis suggests a need to better address cybersecurity risk factors, impact to air traffic control operations, and recommendation on how to recognize and mitigate cyber induced effects. Further practice and research needs should be documented and deliberately included as part of human factors research. Conclusion: Our findings suggest a potential cybersecurity skill gap in what is being offered in AT CTI academic programs and what might actually be needed with regard to the cybersecurity knowledge, skills, and abilities of Air traffic Controllers. The NICE framework provides a starting point to begin documenting cyber security knowledge, skills, and abilities specific to air traffic controllers that can be shared by a wide community. This approach will enable standardization and validation across the enterprise at large. The contrast presented here highlights salient cybersecurity issues that educators and practitioners should consider as an essential building block for further program development, delivery, and evaluation of NextGen implementation phases. As Huerta (2011) emphasized during the annual IT/ISS conference, We need you in the game with your best ideas as to how we secure NextGen systems, how we operate our infrastructure and how we deal with a continually evolving cyber threat. 14

15 References Beidel, E., and Magnuson, S. (August 2011). Government, military face severe shortage of cybersecurity experts. National Defense Magazine. Retrieved from ment, MilitaryFaceSevereShortageOfCybersecurityExperts.aspx Carletta, J. (1996) Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2), pp Cavelty, M.D., and Suter, M. (2009).Public Private partnerships are no silver bullet: An expanded governance model for critical infrastructure protection. International Journal of Critical Infrastructure Protection, 4(2), Chertoff, M. (2009). National infrastructure protection plan: Partnering to enhance protection and resiliency. Retrieved from Chialastri, A. (n.d.). Automation in aviation. Retrieved from /37990/ InTech Automation_in_aviation.pdf Department of Homeland Security (n.d.). Cybersecurity recruitment and development programs [Brochure]. Retrieved from and development programs Duffie, E.L. (2011). National initiative for cybersecurity education: Strategic plan. Retrieved from plan_sep2012.pdf Federal Aviation Administration, and National Aeronautics and Space Administration (2011). Next generation air transportation system human factors research coordination plan. Retrieved from Finkle, J., and Randewich, N. (12 June 2012). Experts warn of shortage of U.S. cyber pros. Retrieved from media techsummit symantec idusbre85b1e

16 Flightglobal (2012). Airline safety review: Accident rates. Retrieved from features/safety review 2011/accident rates/ Government Accounting Office (31 May 2012). Critical Infrastructure Protection: DHS Could Better Manage Security Surveys and Vulnerability Assessments, GAO Government Accounting Office (January 2002). Aviation Safety: FAA and DOD response to Similar Safety Concerns. Report to the Honorable Norman Y. Mineta, Secretary of Transportation, and the Honorable Donald H. Rumsfeld, Secretary of Defense. Washington, DC: GAO. Homeland Security Advisory Council (2012). CyberSkills task force report. U.S. Department of Homeland Security: Washington, DC. Retrieved from default/files/publications/hsac%20cyberskills%20report%20 %20Final.pdf Kopardekar, P., Prevot, T., and Jastrzebski, M. (2009). Traffic complexity measurement under higher levels of automation and higher traffic densities. Air Traffic Control Quarterly, 17(2), Krebs, B. (1 October 2009). DHS seeking 1,000 cyber security experts. Washington Post. Retrieved from _ cyber_securit.html Metzger, U., and Parasuraman, R. (2005). Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors, 47(1), McCallie, D., Butts, J., and Mills, R. (2011). Security analysis of the ADS B implementation in the next generation air transportation system. International Journal of Critical Infrastructure Protection, 4(2), doi: /j.ijcip , National Security Agency (21 May 2012). NSA announces new program to prime college students for careers in cyber ops. Retrieved from 16

17 press_room/2012/new_college_cyber_ops_program.shtml Overly, S. (11 June 2012). University of Maryland, Northrop Grumman partner on cybersecurity program. Washington Post. Retrieved from capitalbusiness/university ofmaryland northrop grumman partner on cybersecurityprogram/2012/06/09/gjqa7dafuv_story.html von Solms, R., and Warren, M. (2011). Towards the human information security firewall. International Journal of Cyber Warfare and Terrorism, 1(2), doi: /ijcwt Swanson, M.; Guttman, B. (1996). Generally accepted principles and practices for securing information technology systems. Gaithersburg, MD: National Institute of Standards and Technology. Retrieved from 14/ pdf 17