Investigating information security awareness: research and practice gaps

Transcription

1 Investigating information security awareness: research and practice gaps Authors Aggeliki Tsohou 1, Spyros Kokolakis 1, Maria Karyda 1, Evangelos Kiountouzis 2 1 University of the Aegean, Dept. of Information and Communication Systems Engineering, Samos GR-83200, Greece, {agt, sak, mka}@aegean.gr, voice: , fax: Athens University of Economics and Business, Dept. of Informatics, 76 Patission Str., Athens GR-10434, Greece, eak@aueb.gr, voice: , fax: Abstract This paper aims at creating a broad picture of security awareness and the ways it has been approached and also concerns, problems or gaps that may inhibit its successful implementation, towards understanding the reasons why security awareness practice remains problematic. Open coding analysis was performed on numerous publications (articles, surveys, standards, reports and books). A classification scheme of six categories of concern has emerged from the content analysis (e.g. terminology ambiguity) and the chosen publications were classified based on it. The paper identifies ambiguous aspects of current security awareness approaches and the proposed classification provides a guide to identify the range of options available to researchers and practitioners when they design their research and practice on information security awareness. Keywords: information security awareness, information security management Introduction It is not an innovative statement that information security is not only a technical issue. Creating and maintaining information systems security requires the application of technical security controls, but also the application of administrative, procedural and managerial controls. The IS stakeholders are, in most cases, the biggest danger to an organization s IT 1

2 systems (von Solms, 2000). Therefore, security compliance is not possible without addressing the human issues of information security with proper awareness and training (Bresz, 2004). Latest information security surveys indicate the attention that has been given to the need of conducting information security awareness (ISA). To begin with, Ernst, & Young (2004) recognized the lack of security awareness by users as the top obstacle to effective information security. In the same year, ISA appeared as an issue of concern in CSI/FBI (2004) reporting that the organizations view security awareness training as important, although respondents believe their organization does not invest enough in the specific area. The exact same conclusion was made in CSI/FBI (2005), while in the same year Ernst, & Young (2005) identifies ISA as one of the measures in order to minimize the gap between the growing IS security risks and the actions taken to address them. Continuing, ISA has been identified by the Ernst, & Young (2006) as a function of the most crucial global priorities for information security that will have an accelerating impact on organizations ability to manage their risks and, ultimately, on their success. CSI/FBI (2006), reported an increase on respondents perception of the importance of security awareness training, without however the proper relevant investments. Finally, CSI (2007) specifies ISA as one of the most critical computer security issues for the next years. Apart from the raised attention for the imperative of ISA in current organizations, the analysis of information security surveys indicates that organizations have not implemented proper and effective solutions for ISA and such issues remain unresolved. In order to make sense of the reasons why security awareness practice remains problematic, a review of security awareness literature has been conducted towards revealing ISA perspectives and concerns, problems or gaps that may inhibit its successful implementation. Since an understanding of current awareness strategies was aimed, we seek answers to the fundamental questions that a journalist applies in order to investigate an issue (who, where, why, how) following the procedures of open coding in grounded theory, that is used in our research. Therefore, our first aim is to provide answers to the questions a) how ISA is perceived by researchers and practitioners, b) who is involved, c) how is it developed, implemented and evaluated, and d) in which organizational context is it situated (e.g. a broader security management process). Not a unique answer to these questions exists; there is no common definition, method and approach of ISA, since it a socially constructed concept. Thus, our second aim is to explore the way ISA is perceived and provide a broader picture of security awareness, helping practitioners, managers and academics in different ways; practitioners and managers that have to make quick decisions regarding security management and awareness, by providing an overview of current literature and academics by highlighting the way ISA has been studied and the topics that are crucial to further investigate. 2

3 The remainder of the paper is organized as follows; next section describes the research methodology used, in section three we review the ISA literature using the classification scheme proposed in section two. A discussion follows and the paper concludes with summarizing the identified issues of concern and their impact on practice and research. Study Approach Selection of publications The purpose of our analysis is to make sense of the way that practitioners and academics tackle the security awareness issue, in order to achieve a more crystallized picture of ISA perspectives and gaps. Our review focused on analyzing security awareness strategies, including campaigns, practices, programs and research studies that refer to organizational or other contexts (e.g. security awareness for internet home users). Since a broad picture of ISA was aimed we first explored information systems and information systems security journals and magazines through the aid of digital libraries (EBSCO, Elsevier Science Direct, Emerald, IEEE Electronic Library, Springer Link, ACM Digital Library), so as to collect publications that focus on ISA. In addition, conference proceedings were examined. Finally, information security standards, surveys and reports were examined. This process identified 42 information security awareness studies published in: a) Leading peer-reviewed research journals in the areas of information (systems) security; namely, Computers & Security, Information Management & Computer Security, Information Systems Security, Security Management Practices, Information Technology Learning and Performance Journal, The journal of information and knowledge management system, Logistics Information Management. b) Leading magazines that provide information about how information security management practically takes place within commercial organizations and, also, the latest developments in current security issues. They also express concerns of practitioners based on their experience of realizing ISA in organizational contexts: IEEE Security & Privacy, Network Security, and Computer Fraud & Security Bulletin. c) Surveys conducted on security issues and standards published by security firms or security institutes, i.e. Computer Security Institute, International Standards Association, National Institute of Standards and Technology, Pentasafe security technologies. d) Reports published by security organizations (European Network and Information Security Agency). 3

4 e) Chapters in Information Security books published by Springer and the International Federation for Information Processing (IFIP). f) Conference proceedings: IFIP TC11 International Conference on Information Security, Annual ACM SIGUCCS Conference on User Services, SECURE and the World Conference on Information Security Education (WISE). We excluded unpublished working papers and master s theses or doctoral dissertations that are not widely accessible. It should be noted that it was our intend to include publications that mainly focus on information security awareness issues, challenges and practices, but also publications where ISA is studied as a secondary issue, since they provide viewpoints of the way ISA is perceived (e.g. security management standards). Tables 1 and 2 present this allocation. ISA as the primary issue Benjamin et al. (2007) Hawkins et al. (2000) Security Awareness Index Report (2002) Casmir and Yngstrom (2005) Kritzinger (2006) Siponen (2000) Chen et al. (2006) Kruger and Kearney (2006) Spurling (1995) Cox et al. (2006) Maeyer (2008) Steyn etn al. (2007) Danuvasin (2008) Mathisen (2004) Thomson (1999) Dodge et al. (2007) McCoy and Fowler (2004) Thomson and von Solms (1998) Drevin et al. (2007) NIST (2003) Valentine (2006) ENISA (2006) Okenyi and Owens (2007) van Wyk and Steven (2006) Everett (2006) Peltier (2005) Vroom and von Solms (2002) Furnell et al. (2002) Power M.(2007) Yngström and Björck (1999) Furnell et al. (2006) Power R. and Forte (2006) Wood (1995) Goucher (2008) Puhakainen (2006) Hansche (2001a) Qing et al. (2007) Table 1: Sample analysis regarding topic of investigation (ISA as primary issue) ISA as secondary issue Albrechtsen (2007) Leach (2003) Bray (2002) PWHC (2006) CSI (2007) Schlienger, and Teufel (2003) Frye (2007) Stanton et al. (2005) ISO/IEC (2005) Vroom and von Solms (2004) Knapp et al. (2004) Table 2: Sample analysis regarding topic of investigation (ISA as secondary issue) Research methodology Our research methodology is based on the grounded theory techniques. Grounded theory methods specify analytic strategies and not data collection methods, and thus our technique can be applied to any other group of articles from the ISA literature. Grounded theory is a systematic interpretive method for the generation of theoretical insights from data. Open coding is the first step which refers to the process of breaking down, examining, comparing, conceptualizing 1 and categorizing 2 data (Strauss and Corbin, 1990). Open coding process requires opening up the data (in order to develop categories, properties and dimensions of them) by asking a set of four questions. As already mentioned these questions are: a) how ISA is perceived by researchers and practitioners, b) who is involved, c) how is it developed, 1 Proposing conceptual labels on discrete happenings, events, and other instances of phenomena 2 Grouping concepts together under a higher more abstract concept providing a classification of concepts 4

5 implemented and evaluated, and d) in which organizational context is it situated (e.g. a broader security management process). We have examined the literature with regard to these questions and, in sequence we have identified ISA concerns, problems or gaps that may inhibit its successful implementation. These aspects of discussion constitute the six components of our review framework. Aspects that reflect a consensus among researchers as well as practitioners are not included in the framework (e.g. academics and practitioners all seem to agree on the content of security awareness programs proposed). The classification scheme The selected publications were classified based on six issues of concern that resulted from the open coding analysis and formed our classification scheme (Fig. 1). Criterion 1: Distinction of security awareness, training and education Criterion 2: Desirable outcome Criterion 6: Conditions intervening to success Security Awareness Literature Criterion 3: Evaluation approaches Criterion 5: The role of the IS stakeholders Criterion 4: Process or product aspects Figure 1: The classification scheme First, current literature is examined regarding the terminology used. ISA is not perceived uniformly by security researchers and practitioners and thus, the distinction of security awareness, training and education is selected as the first review criterion (Criterion 1). In addition, security awareness desirable outcome becomes a base of comparison since different desirable outcomes exist across the various approaches (Criterion 2). The literature sources are also examined according to the proposed evaluation approaches of security awareness (Criterion 3). Moreover, ISA per se may be perceived differently; it could be studied under the perspective of a process conducted in an organizational context or as a product. Therefore, literature approaches are examined according to the criterion: whether process or product aspects are studied (Criterion 4). Moreover, ISA involves a number of IS stakeholders either as members of the designing team or as recipients or both. Therefore, the role of IS stakeholders in security awareness is selected as the fifth issue of interest (Criterion 5). The final criterion of the framework refers to the conditions intervening to security awareness 5

6 success, since many researchers either identify or imply the influence of different factors on security awareness success (Criterion 6). All the selected sources were examined regarding the six criteria. From the criteria, only the ones that are applicable (e.g. not all publications include or refer to the issue of security awareness evaluation) and reveal ambiguous issues are presented. The correlation of each publication with the six criteria is presented in Table 3. 6

7 Publication Criterion 1 Criterion 2 Criterion 3 Criterion 4 Criterion 5 Criterion 6 Albrechtsen (2007) Benjamin et al. (2007) Bray (2002) Casmir and Yngstrom (2005) Chen et al. (2006) Cox et al. (2001) CSI (2007) Danuvasin et al. (2008) Dodge et al. (2007) Drevin et al. (2007) ENISA (2006) Everett (2006) Frye (2007) Furnell et al. (2002) Furnell et al. (2006) Goucher (2008) Hansche (2001a) Hawkins et al. (2000) ISO/IEC (2005) Knapp et al. (2004) Kritzinger (2006) Kruger and Kearney (2006) Leach (2003) Maeyer (2008) Mathisen (2004) McCoy and Fowler (2004) NIST (2003) Okenyi and Owens (2007) Peltier (2005) Power M. (2007) Power R. and Forte (2006) Puhakainen (2006) PWHC (2006) Qing et al. (2007) Schlienger, and Teufel (2003) Security Awareness Index Report (2002) Siponen (2000) Spurling (1995) Stanton et al. (2005) Steyn etn al. (2007) Thomson (1999) Thomson and von Solms (1998) Valentine (2006) van Wyk and Steven (2006) Vroom and von Solms (2002) Vroom and von Solms (2004) Wood (1995) Yngström and Björck (1999) 48 Total number of publications Table 3: Correlation between publications and issues that will be presented 7

8 Review of security awareness literature Distinction of security awareness, training and education The different definitions and perceptions of ISA concept is one of the main obstacles that one confronts in examining relevant issues. In this criterion we consider the question of how ISA is perceived with relation to the neighbouring areas; security training and education. Although most researchers agree on differentiating ISA from training and education, a mixing of the terms used exists. Most definitions imply that ISA is the bottom level of a security learning pyramid: ISA aims at attracting the attention of all IS users to the security message, making them to understand the importance of information security and their security obligations, training aims at building knowledge and developing the relevant skills and competencies, and education aims at creating expertise (NIST , 2003, Peltier, 2005; Katsikas 2000). Analyzing the relevant publications, however, we realized that this distinction is not uniformly adopted. Many publications attempt to strictly define ISA distinctly from training and education. Hansche (2001a) explicitly adopts the distinction of awareness and training and states that security awareness is not considered the same as training... (p. 14). ENISA (2006) differentiates ISA from training and education (p. 18) and the organization is called to decide whether the program would focus solely on awareness or also training and education (p.18, 22). Maeyer (2007) defines security awareness an organised and ongoing effort to guide the behaviour and culture of an organisation in regard to security issues. Kritzinger (2006) recognises the existing terminology dubiousness and defines separately the three neighbouring terms (p. 300). Similarly, Mathisen (2004) adopts the differentiation of the three terms (p. 2). Chen et al. (2006) adopt the definition provided by NIST (2003) and thus they conceive awareness distinctly by training and education. Schlienger and Teufel (2003) define and use the terms of security awareness, training and education distinctly (p. 9) and propose a program of schooling that includes all of these elements. The exact same approach is adopted by Okenyi and Owens (2007) who differentiate the three terms and support a learning process which starts with awareness, continues with training and evolves into education. While the authors at first state that the purpose of a security awareness program is to increase awareness and facilitate understanding through training, they explicitly differentiate all terms at the end. Similarly, Power R. and Forte (2006) present a practical implementation of a successful awareness and education program; their approach includes awareness, training and education components, which are however distinct throughout the analysis. Power M. (2007) presents a case study of awareness and training specialized to 8

9 privacy issues. The described campaign involves both awareness and training, but the two processes are quite distinct. On the other hand, Siponen (2000) suggests that ISA includes training and education (p. 35). This viewpoint is also adopted by the Security Awareness Index Report (2002) where education and training are considered as aspects of ISA (p. 14). Similarly, Thomson and von Solms (1998) and Thomson (1999) regard ISA as an issue of education; security awareness is about making users aware of the value and importance of information and security procedures, which includes proper users education and training (p.20). In the same way, Qing et al. (2007) state that, Information security awareness programs are an important approach towards educating users to prevent security incidents (p.177). Spurling (1995) makes no reference of security education, but suggests that security awareness initiatives include security training (p. 25). Frye (2007) although he adopts the definitions provided by Peltier (2005), he suggests that security awareness learning includes all the three aspects of awareness, training, and education (p. 180). Likewise, Everett (2006) suggests that ISA being an obstacle to effective information security could be confronted with proper training and education (p.15). On the contrary, Yngström and Björck (1999) argue that ISA is a component of an information security training and education program. Finally, Stanton et al. (2005) examine security awareness and training as a uniform issue; e.g. with respect to training and awareness, 35% have never taken any type of security training... (p. 130). Moreover, many publications refer to the process of ISA as awareness training (Knapp et al., 2004; CSI, 2007; ISO/IEC, 2005; Goucher, 2008). McCoy and Fowler, (2004) do not differentiate the three terms and propose a framework for establishing successful security awareness programs and include in them in-person and web-based training and the goal of the programs is to educate users and change behaviour through two main avenues: security awareness training and monthly activities (p.346). Similarly, Vroom and von Solms (2002) mix the three neighboring terms since they suggest that making users information security aware includes that they are educated about the importance of securing information (p. 22) and since formal awareness program are needed in order to ensure that all users in the organization receive the proper education and training to make them aware of the security risks and threats (p.25). An obscure use of the three terms is also figured in the study of Dodge et al. (2007) and Hawkins et al. (2000). Similarly, Bray (2002) examines awareness in case of employees downsizing and suggests that security awareness is a training effort (p. 5). Equally, Steyn et al. (2007) state that user training is an important part of ICT security awareness. ISA, training, and education are also used as interchangeable terms in PWHC (2006) and Goucher (2008). In the same way, Benjamin et al. (2007) and Furnell et al. (2002) treat awareness and training uniformly. In his latter publication, Furnell 9

10 et al. (2006) focuses solely to awareness, without any reference to the neighbouring processes. Valentine (2006) presents a security awareness model as a method to outgrow Pre- Packaged or general awareness programs and develop organization-specialised programs. Although the security awareness term is not defined, it is evident through the analysis that ISA is viewed as including training and education; e.g. employees understand not only the what relative to policy, but also the why. The obscure use of awareness and training terms is also evident in Danuvasin et al. (2008) where the authors use the concept of training to refer to awareness training. They state that Training Increases the Users Security Awareness and Training or education in security awareness is an effective method for an organization to be sure that employees are aware about the threats and how to prevent them. (p. 71). Finally, Puhakainen (2006) distinguishes education, but the terms of awareness and training are tangled. First, he uses the concept of awareness training to refer to the process of conducting an awareness learning process. Second, his review of security awareness literature involves articles regarding security training (e.g. Hansche, 2001b). van Wyk and Steven (2006), also use the term awareness training but the two learning processes are conceived differently; although some organizations start off with a basic awareness training program, it isn t enough. An effective training program must be about getting software developers to change their habits. Kruger and Kearney (2006), Drevin et al. (2008), Cox et al. (2001), Wood (1995) and Leach (2003) while do not explicitly distinct the terms awareness, training and education, they avoid the confusion in using these terms in their research. Similarly, Casmir and Yngstrom (2005), Albrechtsen (2008) and Vroom and von Solms (2004) do not differentiate the three terms, but their work focuses solely to security awareness without any reference to the other two learning processes. Distinction of security awareness, training and/or education or no ambiguity No distinction or ambiguity of security awareness, training and/or education Albrechtsen (2008) Benjamin et al. (2007) Casmir and Yngstrom (2005) Bray (2002) Chen et al. (2006) CSI (2007) Cox et al. (2001) Danuvasin et al. (2008) Drevin et al. (2008) Dodge et al. (2007) ENISA (2006) Everett (2006) Furnell et al. (2006) Frye (2007) Hansche (2001a) Furnell et al. (2002) Kritzinger (2006) Goucher (2008) Kruger and Kearney (2006) Hawkins et al. (2000) Leach (2003) ISO/IEC (2005) Maeyer (2007) Knapp et al. (2004) Mathisen (2004) McCoy and Fowler, (2004) NIST , 2003 Puhakainen (2006) Okenyi and Owens (2007) PWHC (2006) Peltier, 2005 Ronald et al. (2007). Power M. (2007) Schlienger, and Teufel (2003) Power R. and Forte (2006) Security Awareness Index Report (2002) van and Steven (2006) Spurling (1995) Vroom and von Solms (2004) Steyn et al. (2007) 10

11 Wood (1995) Thomson (1999) Thomson and von Solms (1998) Valentine (2006) Vroom and von Solms (2002) Yngström and Björck (1999) Table 4: Distinction of awareness terminology in the literature The desirable outcome and methods employed The ambiguous use of the three neighboring terms of awareness, training and education may result in indistinct or excessive goals of awareness initiatives. Analyzing the selected publications to this criterion has indicated that some of the studies differentiate these terms but their stated objectives reflect more than the first level of the learning process. Other studies do not differentiate the three terms or they mix them; they present objectives that surpass the goals of raising attention or aim at changing human behavior and focus on attitudinal and behavioural theories to achieve it. To start with, although ENISA guide (2006) differentiates awareness from training and education it foreshadows a change management approach (p. 15) which strongly opposes to the aim of simply raising attention or gaining audience s commitment to security. This change is identified as a cultural change (p. 15) and refers to a change in: a) user s perceptions, b) organizational culture, c) user s behavior, d) audience s familiarity with security policies and procedures and, e) audience s interests towards security. In the same way, Power R. and Forte (2006) determined the security awareness and education program s mission to be a corporate culture change (p. 1). However to achieve this objective general ISA is not enough; they introduced changes in the organizational structure (an awareness and education hierarchy). In addition, they implemented awareness initiatives (newsletter, presentations, the establishment of a Security Day etc.), but also specialized security training and security briefing to executives. In the same way, Power M. (2007) aims at developing a privacy culture and states that changing a culture requires individual communication... ; he employs changes to the organizational structure, awareness activities (code of conduct, events, communication channel to the privacy chief) and training. On the other hand, Hansche (2001a) explicitly distinguishes the terms of awareness and training and therefore proposes a security awareness program to aim at changing end-user s actions during their work routine so as they apply good security habits (p. 16) and also change behaviors or attitudes (p. 20). However, it is not clear how these changes can be achieved through a process during which the IS end-users simply receive information (p. 1). The same perspective is adopted by NIST (2003) which also makes an explicit distinction of awareness, training and education terms; the aim of security awareness programs is to change behavior or reinforce good security practices (p.19). However, NIST (2003) introduces the concept of acclimatization for this purpose; a method to acclimatize users into 11

12 the new habits is to discuss IT security issues in the context of personal life experiences. Moreover, since people tend to tune-out messages that differ from their current practices, awareness process should be ongoing and should aim at assimilation; a process whereby an individual incorporates new experiences into an existing behaviour pattern (NIST, 1998). Chen et al. (2006) also aim at changing behavior and reinforcing good security practices. To do so they rely on e-learning techniques, those constitute active informing efforts. We should highlight that the Information Security Awareness System they build provides a two-way communication channel, since awareness material is targeted to users, personalized content is supported and also discussion forums change users role from passive recipients of information to active members of the process. In the same way, Cox et al. (2001) regard awareness raising to be an issue of changing users behavior and security understanding. They suggest the use of a) discussion sessions, b) do and don t checklists, and c) online tutorial, which is a mixture of one-way and two-way communication channels. Moreover, Maeyer (2007) states that security awareness generally aims at changing users behavior. However, this aim should be more specialized to objectives that are: a) Specific, b) Measurable, c) Attainable, d) Realistic, and e) Time-delimited. Peltier (2005) distinguishes the three neighboring terms and refers to security awareness as a process of motivation and stimulation. The awareness program provides the audience the information regarding their rights and responsibilities and the contact information in case of a concern or security incident. The IS end-users are the recipients of this information, which may (or not) raise their attention to the security concerns. Frye (2007) accepts the same distinction of awareness, training and education but perceives awareness to include all of them. The purpose of ISA is to guide the users concerning what is approved and appropriate behavior and what is not. To do so, he employs training (lectures, workshops, on-the-job training, computer-based training etc.) and also change management methods (Unfreezing, Making the Transition, and Refreezing). Similarly, security awareness is regarded by Benjamin et al. (2007) as a technique of raising the level of consciousness through the strategic placement of awareness messages. For this reason, they suggest the utilization of a video game as a powerful teaching tool, although it should be mentioned that this method is proposed for awareness and training as well. As already mentioned, Thomson and von Solms (1998), Siponen (2000), Qing et al. (2007) and Puhakainen (2006) regard awareness as including training and/or education. They focus on changing human behavior which is related to changing human cognitions, attitudes, intentions and emotions (Thomson and von Solms, 1998). Similarly, Puhakainen (2006) aims at achieving behavioural changes towards IS users compliance with IS security policies and instructions (p. 70). To do so, he employs attitudinal and instructional theories. Siponen 12

13 (2000) regards training and education as aspects of awareness which lead to gaining users commitment by taking into account aspects highlighted by behavioural theories. Qing et al. (2007) employ the elaboration likelihood model to explain attitude change. Finally, Security Awareness Index Report (2002) also considers training and education as aspects of ISA. According to the report the awareness aims at the understanding by people of their role in ensuring the security of information and information technology and their ability to make prudent decisions about security (p. 13). This definition of security awareness goes far beyond raising attention or changing attitudes; having the ability to make prudent decisions about security requires knowledge and critical judgment, attitude and training and education, as the report states. The same goal with different methodological foundations is faced by Danuvasin et al. (2008); they conduct action research in order to provide a security awareness program that is adequate to the specific organizational setting and its problems. Drevin et al (2008) make an extensive exploration of security awareness objectives. According to their view security awareness aims at reducing human error, theft, fraud, and misuse of computer assets. Their study found the ISA fundamental objectives to be in line with the acknowledged goals of ICT security, e.g. confidentiality, integrity and availability. However, additional social and management objectives emerged, such as acceptance of responsibility for actions and effective use of resources. Their framework focuses solely on specifying the ISA fundamental objectives; their research does not intent on proposing the methods to achieve these objectives. Albrechtsen (2008) adopts the view that security awareness campaigns aim at changing users behavior. His study does not propose methods of ISA, on the contrary presents a users evaluation on awareness campaigns (among others). According to this study general awareness campaigns (e.g. expert-based one-way communication directed towards many receivers) have little effect on user behaviour. Albrechtsen (2008) study suggests that user-involving approaches would be more effective. Furnell et al. (2002) also advocate the interactive approaches for achieving users familiarity with security issues. They implement a tool that engages users to security scenarios, in order to allow mistaken actions without cost to the organization. Concluding, different views of the ultimate goal of ISA exist; others regard the target of awareness to simply raising attention which may benefit information security, while others state that awareness aim is to alter behaviors or attitudes. First there is a diversity regarding the awareness ultimate target and, second, the aspect of accordance between the awareness goal and appropriate methodology has not been solved, as depicted in Table 5. Among the selected publications this section presents only the ones that state or imply the desirable goal of security awareness (see Table 3). 13

14 Publication ISA term Ultimate Goal Method Albrechtsen (2008) No reference of training and education. ISA is not defined. Changing users behavior User-involving approaches are proposed. Benjamin et al. (2007) ISA and training are treated uniformly. Raising consciousness. Interactive computer-based training. Chen et al. (2006) They adopt the definition of NIST (2003) Changing behavior and work practices. E-learning, Information Security Awareness System. Cox et al. (2001) None of the terms is defined. Changing users behavior and security understanding Discussion sessions, checklists, online tutorial. Danuvasin et al. No definition provided. Ambiguous Changing users behavior Action Research. (2008) use of awareness and training. Frye (2007) Peltier s (2005) concepts are used, but in practice ISA is perceived as including training and education. Users understand their role in security. Training. Change management methods Drevin et al. (2008) ENISA (2006) Furnell et al. (2002) Hansche (2001a) Maeyer (2007) NIST (2003) Peltier (2005) Power M. (2007) Power R. and Forte (2006) Puhakainen (2006) Qing et al. (2007) Security Awareness Index Report (2002) Siponen (2000) Thomson and von Solms (1998) Security awareness, training and education are not defined. However, training and education are considered different. ISA is not defined. Training and education are considered different. Security awareness, training and education are not defined. Awareness and training are treated uniformly. ISA is strictly differentiated from training. ISA heightens the importance of IS security and the possible negative affects of a security breach. ISA is an organised and ongoing effort to guide the behaviour and culture of an organisation in regard to security issues. Awareness is not training. The purpose of awareness presentations is simply to focus attention on security, allowing individuals to recognize IT security concerns and respond accordingly. ISA distinguished by training and education. It is a process of stimulation, motivation and reminding the audience what is expected of them. ISA is not defined. Awareness and training are differentiated. Security awareness, training and education are not defined. The program includes awareness, training and education components. A process of improving user s security behavior. ISA is mixed with training. No definition provided. Mixing of education and awareness terms. Training and education considered to be aspects of ISA. The understanding by people of their role in ensuring the security of information and information technology and their ability to make prudent decisions about security. Training and education considered to be aspects of ISA. It is a state where users in an organization are aware of (ideally committed to) their security mission. No definition provided. Mixing of education and awareness terms. In line with the Security Management objectives: confidentiality, integrity and availability. Social and management objectives. Changing organizational culture, users behavior and perceptions Advancing users familiarity. Highlighting security issues and responsibilities. Changing users behaviors, attitudes and work habits. Changing users behavior. Changing behavior and work habits. Motivation and Stimulation. Developing a privacy culture. Changing corporate culture. Changing users behavior Changing users behavior towards decision-making. People s ability to make prudent decisions about security. Making the right decisions requires the right combination of training, critical thinking (judgment) and attitude. Raise the level of awareness - Minimize user related faults - End-user commitment Changing ideas and behavior of the user and user s attitude. Table 5: Correlation of security awareness goals and methods No method provided. Effective communication Interactive computer-based training. Transmission of information. Awareness campaigns. Assimilation through repetition. Transmission of information. Changes in structure. Awareness initiatives. Specialized security training. Changes in structure. Awareness initiatives. Specialized security training. Briefing to executives. Universal constructive instructional theory Elaboration likelihood model Motivational theories Elaboration likelihood model Training and education, Knowledge, Behavior and Attitude Persuasion approaches Behavioral theories Social psychology methods. 14

15 Evaluation approaches of security awareness Conducting the evaluation of the security awareness process prerequisites answering the question of what to evaluate (or as others say, measure). There are several views with regard to this question; the subject of evaluation may be the awareness process itself, the resulting change, the level of audience s awareness or an ultimate Return of Investment. Notwithstanding, development, implementation and evaluation of the program should not be isolated; the evaluation phase should be in accordance with the goal of the security awareness process. To begin with, Mathisen (2004) regards ISA to be the understanding of importance of information security and the display of according behavior. Raising the state of awareness leads to better attitudes and behavior regarding information security; which is a change that refers to the individual level. He selects a number of metrics for awareness that represent the good security behavior, e.g. number of reported security incidents or number or number of hits to security web pages, but these metrics focus only on the organizational level. On the contrary, Kruger and Kearney (2006) aim at assessing ISA and according to their approach questions that test the knowledge, attitude and behavior of respondents are employed; therefore they focus on the individual level as their intended goal of awareness. Their analysis results on quantified levels of security awareness (e.g. the overall awareness was measured as 65%). Frye (2007) uses a checklist regarding the awareness application and a questionnaire of multiple choice and open-ended questions regarding user s behavior in hypothetic scenarios. Likewise, Security Awareness Index Report (2002) defines awareness goal to be the empowerment of users to make prudent decision regarding information security. In order this to happen, three factors are identified and become the evaluation s subject: knowledge, perception and attitude, education and training; thereby focusing again to the individual level as the intended goal. Knowledge, attitude and behavior of staff, are also the evaluation criteria of awareness levels, according to Steyn et al. (2007). Yngström and Björck (1999) highlight the difficulty of decision-makers to carry out a costbenefit analysis of security education and training programs, since Return-Of-Investment is unclear, and thereby it is difficult to take a justified investment decision. While the quantification of security awareness, presented above, would facilitate the decision-making of managers, measuring levels of security awareness is considered as difficult to be interpreted by themselves. As they state, the interpretation of such measurements is useful only in a defined organizational context (e.g. a bank) and in comparison with other measurements. They also examine the solution of measuring the effectiveness of the programs by evaluating the users knowledge before and after the program, but this knowledge does not signify that 15

16 they will actually use it. They propose that the impact of such programs is measurable only outside the finite domain of knowledge or behavior. It is measurable in the technical and procedural elements of the IS in which it is reflected (e.g. the adoption of a security policy or adoption of a password policy); which ultimately result in lower costs or increased revenue. In the same way, Okenyi and Owens (2007) argue that the evaluation of ISA cannot be straightforward; effective security awareness is depicted in the presumed beliefs, behaviors, capabilities and actions; for example when security is integrated into enterprises functions and processes. Cox et al. (2001) apply a more short-term evaluation approach; they view their discussion session to be successful since it gained the attention of the participants and stimulated a vigorous discussion. Hansche (2001a) suggests an analysis, whether quantitative or qualitative, for the task of evaluation regarding the specific needs of the program. The subject of her evaluation includes as an example the impact on the attendants perception for the program and its echo, the working habits of the users, the security incidence number and the quality of the passwords used. In general, it is stated that the evaluation should focus on the degree that the intended goals where achieved. NIST (2003) merges the evaluation of awareness and training together and presents a number of techniques such as interviews of the employees. In the same way, Danuvasin et al. (2008) use users interviews to evaluate the program s impact on their behavior. A very different approach is implemented by Dodge et al. (2007), who design and implement a system of exercises designed to evaluate the behavior of users to phishing mails; the targets are recipients of phishing mails (e.g. with attachments or encouragements of entering sensitive information) which can be handled successfully or not. PWHC (2006) coincides ISA to security policy compliance and itemize as methods used for monitoring security policy compliance: a) monitoring activity and logging unusual events, b) software that detects, reacts to and records security policy violations, c) periodic audit of security processes, d) automated scans. ENISA (2006) suggests a number of criteria for security awareness evaluation: a) process improvement, b) attack resistance (event scenarios), c) efficiency and effectiveness (number of security incidents) and, d) internal protections (implemented controls). It is the first attempt, to our knowledge, that the subject of the evaluation is the process itself. Aspects included are whether top management is committed to the process, the number of attendants and the effect of the program. In addition, the guide suggests the internal protections evaluation that focuses on the awareness of IS users other than end-users. This is uncommon into security awareness evaluation approaches, because usually the roles of the IS users included into the target audiences are confused and the end-users are regarded as being a greater threat to security than other stakeholder groups (e.g. IS administrators). 16

17 In practice, according to the results of current surveys (CSI, 2007) organizations use computer-aid knowledge tests, staff reports, number of security incidents or helpdesk, while it is noticeable that many of the respondent organizations (35%) make no effort to measure the effectiveness of ISA on the organization. It should be mentioned that the analysis based on this criterion also presents a subset of the publications studied, since not all of them deal with the aspect of ISA evaluation (see Table 3). Security Awareness: Process and Product Perspectives Our next criterion refers to whether researchers focus on the ISA process or product aspects or both. Many publications focus on the ISA process; therefore describe the steps or methods for its implementation. ENISA (2006) adopts a framework of the overall process of such an initiative divided into three main phases: a) Plan & Assess, b) Execute & Manage, and c) Evaluate & Adjust. NIST Special Publication (2003), defines awareness as the overall process of conducting a security awareness program to raise attention on security by a) design, b) development, c) implementation and d) post-implementation activities. Valentine (2006) proposes a different security awareness model which includes the a) assessment phase, b) identification phase, and c) education phase. Maeyer (2007) also targets to process aspects, as he defines ISA as an ongoing effort (p.50). Vroom and von Solms (2002) also, adopt a similar process perspective depicted in a security awareness program model (p.31) including several steps, such as educating top management, using international security standards, developing and implementing the program, etc. Thomson and von Solms (1998) and Thomson (1999) perceive ISA as a continuous process, which involves programs that will continually remind users of security issues and will inform them of any new ones. Power and Forte (2006) describe the case of an awareness and education program, which primary includes setting the mission and the means, establishing a global security team and defining appropriate content and engaging delivery; which is followed by a three phases program (awareness tasks, seminars, briefings to executives). Finally, Schlienger and Teufel (2003), argue that security awareness and training programs are part of the information security culture management process. According to their approach awareness and training programs lead from become aware to stay aware and end up in be aware, that ultimately changes the security culture. Other researchers focus on the process aspect of ISA, without however defining the steps of its conduction. Hansche (2001a) refers to ISA as the activities of heightening the importance of information systems security and the possible negative effects of a security breach or failure (p. 14). In the same way, Peltier (2005), regards awareness as a process to stimulate, motivate and remind the audience of what is expected of them (p. 39). Drevin et al. 17

18 (2008) adopt the view that security education, training and awareness form part of the process to educate staff on information security and thereby focus on process aspects of ISA. Kritzinger (2006) regards ISA as a continuously updated and renewed process that ensures all stakeholders understand their role and responsibility towards securing the information they work with and are aware of security threats and how to prevent them from happening (p.26). Spurling (1995) regards security awareness as a process that fits in with the culture of the organization and aims at gaining a long-term commitment to security (p. 20). Casmir and Yngstrom (2005) regard ISA as a complicated process that requires good planning and commitment (p. 162). Likewise, Everett (2006) argues that true improvements in security awareness and activities require a change in behavior of people and often in an organization s overall culture (p. 16). He uses the term awareness training to refer to the overall process of security awareness. Furnell at al. (2002) state that it is important security awareness to occur in the first instance and as an ongoing factor of an organization s operation. Similarly, Power M. (2007) regard awareness as a long-term exercise. On the opposite many publications focus their attention on the products of ISA. Siponen (2000) considers security awareness as a state where the users in an organization are aware of ideally committed to their security mission as expressed in end-user given guidelines (p. 31). Vroom and von Solms (2004) imply that security awareness/consciousness is a state where the organization is more in line with information security (p. 196). Bray (2002) regards security awareness as an effort to raise the security consciousness of employees and result to security-aware employees (p. 5). Security Awareness Index Report (2002) defines security awareness as the understanding by people of their role in ensuring the security of information and information technology and their ability to make prudent decisions about security (p. 9). ISO/IEC (2005) standard adopts the same perspective referring to awareness training in order to increase the level of security awareness. Likewise, Hawkins et al. (2000) make an analysis of the state of internet security awareness in the public and private sectors. In the same way, Furnell et al. (2006) include statements of revealing awareness of around just 40% or users need some awareness and practical skills in key areas which imply that they focus on the product aspects of the awareness term. Kruger and Kearney (2006) although consider ISA as a dynamic process, they target at quantifying measurable states resulting from a security awareness program. The measurement of security awareness level is also the objective of Steyn et al. (2007) study, while Wood (1995) provides a list of tools to raise awareness levels. McCoy and Fowler (2004) view ISA as the result of the process of security awareness training which aims to educate our users about the importance of information security and change the way people think (p.1) and ultimately the way they act. 18

19 A completely different approach is adopted by Puhakainen (2006). He views security awareness to be composed by a process and a product component; thus focusing on both aspects respectively. Similarly, Albrechtsen (2008) states that general awareness campaigns have little effect alone on user behaviour and awareness and thus focuses on both process and product aspects when referring to ISA. Process aspects of ISA Product aspects of ISA Process and product aspects of ISA Casmir and Yngstrom (2005) Bray (2002) Albrechtsen (2008) Drevin et al. (2008) Everett (2006) Puhakainen (2006) ENISA (2006) Furnell et al. (2006) Furnell at al. (2002) Hawkins et al. (2000) Hansche (2001a) ISO/IEC (2005) Kritzinger (2006) Kruger and Kearney (2006) Maeyer (2007) McCoy and Fowler (2004) NIST (2003) Security Awareness Index Report (2002) Peltier (2005) Siponen (2000) Power and Forte (2006) Steyn et al. (2007) Power M. (2007) Vroom and von Solms (2004) Schlienger and Teufel (2003) Wood (1995) Spurling (1995) Thomson (1999) Thomson and von Solms (1998) Valentine (2006) Vroom and von Solms (2002) Table 6: Product vs. Process aspects of security awareness term The role of IS stakeholders Another criterion of our review refers to the roles of IS stakeholders within the ISA process. More specifically, the role of the IS end-users varies and ranges from being passive recipients of information to being actively involved into one or more awareness phases. Moreover, the role of other IS stakeholders, such as administrators, is also not clear. Finally, various terms are mixed together; the proposed frameworks refer to IS stakeholders, IS users and IS endusers without actually distinguishing the category they target. In this section we present the proposed roles of IS stakeholders in the various security awareness frameworks. To begin with, ENISA (2006) provides a framework targeted to IS users. The roles of the IS users that are involved in the proposed framework are not clearly stated; the IS users are firstly identified according to their role in the organizational structure. Continuing, the guide recommends constituting a Program Team that leads the overall process, without correlating them to the above roles or making any other suggestions. The role of this Program Team (further referred as Raising Team) is extensive through the three phases of the process or further analyzed into sub-roles (such as awareness sponsor ). However, the role of the IS users is cloudy. While during the planning phase key stakeholders (p.15) are identified and participate in the process it is not stated which stakeholder roles are included. The target groups are supposed to be actively involved and engaged during the awareness process (p. 29). On the contrary in the same page, the target groups role is limited to gaining an opportunity to ask questions and address concerns. No methods of engaging IS users in the process are suggested and through the proposed communication strategy it is implied that two 19

20 major roles exist; the awareness team (who are responsible for planning, executing and evaluating the process) and the audience (who receive information through a communication process). Some roles of IS users are clearly described in the framework proposed by Hansche (2001a); the chief information officer (CIO) and senior management are periodically informed by the awareness program s designer and approve (or not) the goals, content and other elements of the program and the IS end-users simply receive information. However, Hansche (2001a) makes several suggestions that involve IS end-users into more than just receiving information. The employees are expected to display passive or active resistance to complying with promoted security practices. Dealing with this passive or active resistance is an actual part of the ISA process that goes beyond the simple transmission of information. In the given example, security policy includes the disabling of floppy disks for security reasons and the employees fight with management for this decision. This is a process of negotiation that should be considered within the security awareness framework and opposes to the above mentioned role of the IS end-users. Moreover, the employees after the awareness program are expected to actively participate in the protection of the information they process; an outcome that is not surely aroused by the one-way communication of information which is proposed. Finally, other than the role of CIO, senior management and IS end-user, the roles of other IS users are not defined; e.g. the question whether the IS developers, analysts or administrators are members of the audience, the designing team or both remains open. NIST (2003) defines security awareness audience as all people involved in using and managing IT (p. 12). The roles of IS stakeholders are described in detail for training programs. However, regarding awareness programs there are no suggestions of IS users roles other than the audiences, who are recipients of information (p.20). Chen et al. (2006) select two senior information security managers and one training manager to determine the security awareness requirements and the overall program design. Users (including management) are not only recipients of information; they participate on security discussion using the provided forum. Kritzinger (2006) selects to group organization s stakeholders according to their job category; thus six levels of authority are specified. The model she proposes specifies the security issues (security documents and non-technical issues) that each of the six levels should be aware of and in sequel implements an adequate security awareness test. According to Thomson (1999) three security awareness program groups should be identified: top management, IT personnel and end-users. Top management is allocated the role of program s leader and establisher of awareness need(s), IT personnel are responsible for the technical security controls and endusers are recipients of security information. Vroom and von Solms (2002) adopt the same IS stakeholders grouping and form their security awareness programs model. An additional role 20