Enhancing SSL Awareness in Web Browsers

Transcription

1 LUDWIG-MAXIMILIANS-UNIVERSITÄT MÜNCHEN Department Institut für Informatik Lehr- und Forschungseinheit Medieninformatik Prof. Dr. Heinrich Hußmann Bachelorarbeit Enhancing SSL Awareness in Web Browsers Tobias Stockinger Bearbeitungszeitraum: bis Betreuer: Max-Emanuel Maurer Verantw. Hochschullehrer: Prof. Hußmann

2 Zusammenfassung Viele Internetnutzer sind sich des Sicherheitsgrades einer bestimmten Webseite oft nicht bewusst, weil ihnen in den meisten Fällen entweder das technische Know-How fehlt oder aktuelle Sicherheitsindikatoren unbemerkt bleiben. Es stellt eine große Herausforderung dar, der sich schon zahlreiche Forscher gewidmet haben, eine effektive Visualisierung des SSL Status einer Verbindung zu finden. Diese Arbeit zeigt wie das Problem zu lösen versucht wurde, indem man eine Erweiterung für den Mozilla Firefox Browser entwickelt, die sich Browser Skins zu Nutze macht, um diese Information zu vermitteln. Sowohl für Extended Validation und Domain Validation als auch für nur teilweise verschlüsselte Verbindungen werden jeweils entsprechende Personas eingesetzt, die auf einen Blick erkennen lassen, ob übermittelte Informationen abhörsicher sind. Ein weiterer Aspekt von SSL Zertifikaten sind die hierbei auftauchenden Fehler, welche wiederum den Browser veranlassen den Nutzer kurzzeitig mit Hilfe einer Warnung davon abzuhalten, die Seite zu besuchen. Der Benutzer hat hierbei die Wahl, ob er die Seite besucht und unter Umständen Opfer einer Man-in-the-Middle Attacke wird, oder ob er das Risiko gänzlich vermeidet und die Seite verlässt. Aktuelle Warnhinweise werden jedoch von den Leuten entweder nicht gelesen oder nicht wirklich verstanden. Die in dieser Arbeit entstandene Firefox Erweiterung ersetzt die standard Meldungen durch optimierte Warnungen, die eine Vorschau der blockierten Seite bieten und deren Text auf ein Minimum reduziert wurde. In einer Nutzerstudie wurde gezeigt, dass diese Ansätze vielversprechend sind, da sich Nutzer von einem derartigen Plug-in beeinflussen lassen. Mithilfe der gewonnenen Einblicke in die Präferenzen der Nutzer wurde das Plug-in überarbeitet, sodass es sich für den täglichen Einsatz eignet. Abstract Many Internet users are unaware of the security level of a certain website, because they either do not have the technical expertise or they do not notice current security indicators, respectively their absence. It embodies a tough challenge, which numerous researchers have tried to take, to find an effective visualization of the SSL status of a connection. This work demonstrates how the problem was tried to solve by developing an extension to the Mozilla Firefox browser, that utilizes browser skins to convey that type of information. Both for extended validation and domain validation as well as for only partially encrypted connections corresponding Personas are deployed, which allow to tell at a glance, whether information sent is resistant to eavesdropping. A further aspect of SSL certificates are the occuring errors, which in turn cause the browser to temporarily detain the user to visit the site by displaying a warning message. At this point, the user has the choice if she visits the website and under certain conditions becomes a victim of a Man-in-the-Middle attack, or if she avoids all risks by leaving the site. However, current warning messages are either not read or not entirely understood by people. The Firefox extension resulting from this work replaces the standard warnings by an optimized warning page which provides a preview of the temporarily blocked site and a warning text that is reduced to a minimum. A conducted user study has shown that these approaches are promising, as the users tend to get influenced by such a plug-in. Based on the therein gained insights into the users preferences the plug-in was revised, so that it is ready for daily usage.

3 Aufgabenstellung Topic: Enhancing SSL Awareness in Web Browsers Secure Sockets Layer Certificates are a commonly used technique to validate the identity of a certain web page whilst securing the communication with this server. However the way existing certificates are displayed to the user in today s browsers is somewhat doubtful. For most people not knowing what a certificate is, it is hard to understand the differences between trusted and self-signed certificates and what they should pay attention to. Result of this work should be a browser plug-in for evaluation of new methods to display current SSL certificate statuses in the browser. Finally the plug-in should also be evaluated in a user study measuring in how this new kind of visualization helps browser users to understand the meaning of certificates. Tasks: Find related work to the topic of SSL visualization Analyze the different possible SSL cases that have to be distinguished for an inexperienced user Create an idea for a browser plug-in that is able to display those cases in a simple but yet meaningful way Build a browser plug-in for Firefox replacing its current SSL certificate behavior using new ideas Evaluate the new plug-in against the standard behavior in a qualitative user study Written project thesis and presentation of the work Ich erkläre hiermit, dass ich die vorliegende Arbeit selbstständig angefertigt, alle Zitate als solche kenntlich gemacht sowie alle benutzten Quellen und Hilfsmittel angegeben habe. München, September 13,

4

5 Contents 1 Introduction 1 2 Related Work Psychological Background Security Indicators Confirmation of the Problem Propositions SSL Error Warnings Confirmation of the problem Guidelines for alternative design Different Opinions New Approach Idea for the SSL Indicator Problem Characteristics of Personas Security Indicator Design Alternative Warning Message Design Implementation Used Tools Functionality Challenges General principles Detecting encrypted connections Detecting tab switches Applying Personas Replacing Warning Messages User Study Design and Arrangements Goals and Overview Ethical Guidelines Recruitment Study Procedure Group Assignment Study tasks Hypotheses and Questions Asked Hypotheses Questions Asked Results Demographics Persona Results Warning Results Qualitative Results Protocol Evaluation and Informal Feedback Discussion Study design Habituation Scariness of Partial Encryption I

6 6 Improvements Made Warning Design Possible reasons against the plug-in Customization Interference with other Plugins Newer Firefox Versions Conclusion Summary Possible Future Features Adaptation to Other Software Outlook II

7 1 INTRODUCTION 1 Introduction Security Indicators and Warning Messages Security has without a doubt become a very important aspect of everyday Internet browsing. One can imagine what would happen, if there were not any precautions taken for safer browsing. For instance, online banking would easily be attacked and customers would lose a lot of money to hackers who eavesdrop on their connection. Without encryption every could be intercepted and read by anyone - privacy would be a non-achievable goal and misuse of personal information would occur every day. The SSL protocol - respectively the newer TLS protocol [2] - can prevent these attacks at least most of the time. It successfully uses encryption algorithms to make sure that information can only be read by the chosen communication partner. Most browsers have implemented the SSL protocol and provide the users with the necessary security in order to safely perform tasks that require to submit sensitive data to websites. But how do users know if they are safe to enter information on a certain website? Most modern Web Browsers offer users certain clues as soon as the site they are visiting uses the SSL encryption protocol. Often times there is a rather small lock icon somewhere in the browser chrome which symbolizes a secure connection. Newer versions of Mozilla s Firefox and other browsers deploy more sophisticated symbolics when it comes to security visualization. These so-called security indicators are not as flashy as they would have to be, if users were expected to notice them every time the indicators appear. Therefore their absence is very often not remarked which is underlined by current research (see section (1). The first motivation for this work lies in the improvement of the visualization of security indicators, to enable even inexperienced users to distinguish between secure and insecure connections to websites. A main goal is that this happens at a single glance and very reliably. There should be a habituation to the new indicators so users get used to check automatically that the new security indicators show up right before they enter sensitive data, e.g. to a log-in form. In the long run users could be trained to identify phishing sites even before the URL is blacklisted and would trigger a browser warning message. The back-end of the SSL architecture requires some degree of maintenance from website owners and administrators. SSL certificates have an expiration date and have to be renewed from time to time. If website administrators neglect the expiration date, users visiting the regarding website will probably face a certificate warning shown by the browser - and every browser has its own design for such warnings. Furthermore they imply a certain cost factor: If a website has different areas with different sub-domains, a wild-card SSL certificate would be required to prevent a browser warning. However, most websites using SSL only certify one concrete URL to save money. A link pointing to an SSL secured sub-domain will create a domain-mismatch error and hence result in a browser-warning which users can ignore in the most cases. On the other hand, attackers or tricksters intercept Internet traffic between the user and a secure website in a so called Man-in-the-middle attack, which will ultimately lead in a browser warning as well. This in turn should not be ignored but catch the users attention. Internet users are rather regularly confronted with warning messages dealing with a problem with the server certificate, most of them being benign. In this particular case as well user studies show that the content of the warnings is often misunderstood since users are not quite aware of the purpose of such certificates. A consequence can be a precipitate leaving of harmless sites or an inconsiderate handling of the warning in favor of a potential man-in-the-middle attack. This problem should also be addressed and tried to be solved in order to alleviate the un- 1

8 1 INTRODUCTION derstanding of certificate errors. A reliable aid is desirable to guide the users toward the safe decision on how to continue in such cases, as it would make surfing the Internet easier and safer. Overview The next chapter Related work gives a rough insight into current research on this topic. The main focus lies on different approaches and their evaluations for the problems mentioned above. Based on the observations gained through this research, a design for new security indicators and warnings is given in section 3. Therein ideas for a Firefox extension using SSL Personas and new warning designs are explained. The work proceeds in section 4 with the implementation of such an Add-on. A short overview of the functionality is given. Section 5 contains the setup and findings of a user study conducted to evaluate the effectiveness of the SSLPersonas extension. Results are also discussed and possible explanations are provided. In section 6 the improvements that were made after evaluating the feedback from the user study are presented shortly. The result is a fully functional revised version of SSLPersonas. Finally, section 7.1 summarizes the findings from this work. An outlook and possible further features of SSLPersonas is given. 2

9 2 RELATED WORK 2 Related Work Usable Security has become a subject under constant research since Web-spoofing emerged [22] and the threat has been identified [9]. There are different approaches when it comes to protect users against eavesdropping of sensitive data and make them aware of phishing attacks. An effective visualization of security in general (and SSL in specific) has to be found. The work of Rachna Dhamija and her team is - amongst others - highly relevant for current research on this topic. This section dwells on how users perceive security when using the Internet - what role does it play and how can one influence the users perception? After giving some insight into the psychological background the thesis step into the details of security indicators and warning message behavior. 2.1 Psychological Background The most effective countermeasure to visual spoofing attacks is security awareness [1]. It is a matter of psychology and as such, each person behaves differently. While one user may simply not know about the importance of security, another might be interested in encrypting Internet traffic at all time. Yet there are some characteristics that many people show. These properties are provided by Whitten and Tygar [32]. They conducted a case study in which they tried to identify problems in dealing with PGP 5.0 mail encryption. After evaluating the results, users are attributed five properties that entail certain considerations for security design in general - not only for security. Among these properties one can find the unmotivated user property which states that security is only a secondary goal and users do not want to check for security but instead count on the program to make sure they are safe. As a consequence, manuals describing certain security features of the deployed software are not read in the most cases. Knowing this, interface design ought to be self-explanatory for security matters. Dhamija et al. revisit these findings and suggest further properties[6]. Before they start looking for a new approach, they explain the golden arches property, which states that people tend to rely on familiar logos even within website content. For example if users recognize the Golden Arches 1 on a website, they are most likely to believe that the website is somehow related to that trusted brand. As a result, web content suggests trustfulness. Aside from that the limited human skills property is illustrated, which states that at some point, users do not care about warning messages. On one hand this is because they do not entirely comprehend the meaning and consequences of making the warning disappear. On the other hand browsers do show warning messages quite often, so users are likely to get bothered by them and try to find a way to entirely turn them off. Dhamija s paper mentions certain tasks in which users usually fail when it comes to safeguard security. They examined reports from the Anti-Phishing Working Group and concluded that users can not reliably distinguish browser chrome from web page content, which is often exploited by phishing attacks. Also, images of security indicators are frequently regarded as real security indicators. If these images appear within website content they are often regarded as legitimate. Furthermore, Dhamija et al. doubt that users understand the meaning of the SSL lock icon commonly used by web browsers, as well as they do not comprehend the principle of SSL certificates. 2.2 Security Indicators One major problem of current security indicators is, that they go unnoticed by a high percentage of users - when not purposefully asked to look for them [31]. Therefore one can assume that the passiveness of security indicators is their major flaw [5]. Browser developers have made strong efforts over the years to improve the noticeable presence of such indicators and their accessibility to the users [28]. 1 Popular term for the McDonald s Brand Logo 3

10 2.2 Security Indicators 2 RELATED WORK Confirmation of the Problem This particular problem was confirmed by an important user study conducted by Schechter et al. in 2007 [26]. The study was designed to find out if the absence of security indicators would be noticed as well as if role playing during a study had significant influence on the behavior of participants. Test persons were divided into three groups, two of which played a role during the study and the third used their own account information. Having meticulously followed ethical guidelines, the study showed that none of the participants noticed the absence of the HTTPS and padlock icon indicators and went on to log in to the online banking site anyhow. Furthermore the paper suggests that having participants playing roles during a study has a negative effect on their security vigilance Propositions One approach using techniques other than displaying an often misunderstood lock icon [7], was the synchronized random dynamic boundary approach by Ye et al., which makes the browser change its chrome borders in a trusted path window to signal security status [34]. Before they started to think of this approach they analyzed what measures have to be taken by an attacker to successfully spoof a secure connection. The herein gained knowledge was then used to create a higher barrier for such attacks through trusted paths for browsers and eye-catching chrome-borders. Thus they chose to have the borders of the main window blink in the same frequency as the borders of the trusted window in order to allow the user to determine whether the established connection is secure. Dhamija et al. also suggest the use of trusted path password windows as well as dynamic security skins [6]. This approach deploys user selected images that occur both in a trusted window, where the user has to log-in on browser-startup and as an overlay to web-site forms to make sure the user notices them. The intention was to make the user aware that each time she submits forms there would have to be her chosen image indicating a secure transmission. Furthermore the browser changes its skin/theme to an image that is based on a hash-value of secure sites. This approach is quite similar to what this thesis will develop in later sections. However Dhamija et al. were not able to conduct a user study to evaluate dynamic security skins, since their mock-ups had not been fully implemented yet. Therefore it seemed reasonable to use it as a starting point for this research project. Another solution recently presented by Sobey et al. utilizes a security ranking indicator similar to traffic lights [27]. However the color scheme of traffic lights was dismissed, because even a red light is a sign of increased authenticity in their approach. Thus only green is deployed for the lights. The indicator is a click-able image located left to the address bar. It replaces the Firefox standard site identity button [19] and performed quite well in a conducted user study. Participants were asked if they had noticed or even used the newly introduced indicators during their browsing taks. The methodology included the use of an eye tracker to verify the participant s answers. Results show that users are either gazers or non-gazers which means that users do either purposely check for indicators or they do not. The authors point out that the indicator is too small to be reliably noticed by all users. Lastly there are some approaches using toolbars to indicate security. A solution suggested by Herzberg et al. is TrustBar which is targeted at preventing phishing attacks [14]. It displays the brand logo of the current site at the top of the browser chrome if the site uses SSL certificates. The TrustBar significantly improved the ability to detect phishing websites during a conducted user study. The research done for this thesis is presumably closest to the dynamic security skin approach. Based on the idea of skinning the browser, section 3 will show how Dhamija s approach 4

11 2 RELATED WORK 2.3 SSL Error Warnings inspired the use of three different SSL skins. 2.3 SSL Error Warnings Since their introduction in 1995 [3], SSL certificates have required a certain degree of maintenance in order to make sure that the increase of security is also an increase in value. Administrators have to pay regard to expiration dates - if they do not, users will be warned. The website owners need to spend money on validation - if they do not, they can still use no-cost self signed certificates, but users will again be warned. If an external link points to a URL that uses a certificate but not for the particular sub-domain, users will once more be warned of the domain mismatch. Thus a valid certificate for would still result in a warning if a link points to One can see that the cases in which warnings appear are very common and users will face such warnings one day or another. The design and wording however is often a critical point for the understanding and behavior of users Confirmation of the problem Sunshine et al. find that current warnings are not understood or overridden very fast by users [29]. After elaborately investigating current behavior when facing certificate warnings, they designed new warnings which performed significantly better in a 100-participant user study. Participants were not told the purpose of the study and for their tasks, security was intentionally tried to move out of focus of attention, as it usually is not the primary goal [33]. Thus users were assigned dummy tasks like looking for the total area of Italy by using a search engine. Some of the tasks resulted in the occurrence of warning messages that participants did not expect - this evoked almost natural behavior despite the laboratory environment. The results suggest that warning design can and should be improved. However users still did not have a complete understanding of certificate errors even with the modified design. This leads to the conclusion that software ought to be capable of deciding at what level of risk warnings are necessary - benign situations should not lead to interrupting (and maybe confusing) the user. Two years earlier, Schechter et al. pointed out that although showing warning messages to users as ultimate measure in order to prevent a successful Man-in-the-middle attack, nearly half of the participants of a relevant user study nonetheless did proceed and entered their passwords to online banking sites that possibly were compromised [26] Guidelines for alternative design Biddle et al. have recently formulated a few guidelines based on the findings of preceding research [3]. They found that current design reveals certain shortcomings when it comes to convey certificate information. In the mentioned paper it is stated that users do not always understand technical terms, like the (greek-deduced!) word encryption. Furthermore messages trying to explain certificate information tend to be very long and users do not want to spend their time reading all of it (see unmotivated user property suggested by Whitten and Tygar [32]). Finally there is often misleading or confusing wording, such as the words guaranteed or secure. Biddle et al. propose to avoid all those factors and suggest a different design which differentiates between identity confidence and privacy protection by separating these aspects in the warning and reducing the text. Note that the focus of their studies lay on the information provided when the user clicked on the lock icon or any other alike security indicator. A study covering the topic of phishing warnings was conducted by Egelman et al. where they investigated the effectiveness of active and passive warnings [8]. It was made sure that participants were not primed on security, instead the study was claimed to treat online shopping behavior. When people were presented an active warning, the actual web-page content was hidden, so participants needed to take an action in order to get to the desired site or leave it. 5

12 2.4 Different Opinions 2 RELATED WORK Contrary, passive warnings only display a pop-up message while still showing the actual phishing site. It was found that active warnings significantly improve the understanding and cautiousness of users. Passive warnings however often go unnoticed and therefore are useless. Egelman et al. also provide a few recommendations for improving the design of phishing indicators which can easily be adapted to certificate warnings as well. Based on their research they state that it is most important to interrupt the users s primary task and therefore rendering the warning active. Warnings need to provide clear choices instead of displaying a rather long block of text. Lastly, preventing habituation should be done by altering the warning message design to make it distinguishable from other - potentially less serious - warnings. Interrupting the primary task was recently recommended by the W3C, too [25]. This work combines all these findings and respects most of the suggested guidelines in order to be able to create a new approach to reduce existing problems. To a lesser extent phishing will be part of the research but it will always be kept in mind. 2.4 Different Opinions Contrary to the approaches shown above, there is at least one researcher who is convinced that all endeavor to improve the users security awareness is almost futile. Herley points out that web security is not perceived as benefiting as it actually is and therefore it remains neglected [13]. The user is burdened with a vast amount of advices to read and understand, which takes a lot of time and energy to actually do. The author brings forth arguments that the time invested in looking for security clues is too much compared to the actual damage done by attacks. This is a very economical point of view. Furthermore he states that effectively all of the warning messages caused by certificate errors are false positives and therefore benign. A possible flaw in his argumentation is the fact, that although the financial damage done (e.g. by phishing) to the whole mass of Internet users is relatively small, affected individuals are not too happy when losing their money to an attacker and think they should have followed the advice. Nevertheless there lies some truth in his observations. Advice given to users is often based on the worst case. He postulates that Usable Security should save the users time, not money in the first place. As one can see in later chapters, these observations will influence the handling of certificate errors for the new approach, when it comes to speeding up the process of adding exceptions. 6

13 3 NEW APPROACH 3 New Approach This section describes how a new approach towards a solution to the above mentioned problems was developed. Insights gained from related work highly influenced the design of new security indicators as well as modified warnings. Images of both the standard and modified versions are shown in each case to make the changes more comprehensible. 3.1 Idea for the SSL Indicator Problem Considering the findings of past research (especially the unfinished dynamic security skin approach), there was the idea to use Firefox s Lightweight Themes, more commonly known as Personas, as security indicators. Thus one can use the whole upper chrome area, as well as the statusbar at the bottom, to effectively catch the user s attention when a connection is secured over SSL. The indicator would remain passive and not require any action by the user. Furthermore it takes only an instant to absorb the provided information and the user s task is not heavily interrupted by a persona change. 3.2 Characteristics of Personas These skins enable fast skinning of the browser chrome and can be chosen from a large library hosted on Mozilla s own servers [20]. Compared to "Heavyweight" Themes, personas do not change the entire look-and-feel as they leave control elements, like the navigation buttons, as they are. Furthermore installing new personas does not require a restart of the browser unlike heavyweight themes do. The feature was originally developed by Christopher Beard as a Firefox Extension and was integrated into the core of the browser by the release of version 3.6, even though with less functionality. There still is a separate Persona extension maintained by C. Beard that offers access to favorites, selecting personas from a browser icon and downward compatibility. 3.3 Security Indicator Design In this section the visual design of the indicators as well as their advantages are presented. Decisions regarding design are explained. Goals One major goal was to design the personas as convenient and modern as possible while keeping the design noticeable by all means. Dhamija s dynamic security skins are a similar approach, but generating images based on hash values, like it was proposed, often results in very edgy graphics that possibly affect the browsing experience. Such an extension should not spoil the clean looks of the browser in order to achieve wide acceptance. Color scheme There are two cases in which the Persona ought to change in the first place: Extended Validation (EV) Certificates and Domain Validation (DV) Certificates. The former procure the highest degree of authentication, as every applicant for such a certificate has to be verified personally while DV certificates only verify that one has reached a specific server - not company [4]. The level of security is equal for both cases. We chose the same color scheme as Firefox currently utilizes for the site identity button to indicate the authentication level, i.e. the persona for the EV Certificate is green (see figure 3.1), a basic SSL connection (Domain Validation - DV [35]) is represented by a blue persona (see figure bottom). A smooth gradient rounds up the background color and gives it a contemporary touch. For the EV version a certificate icon is displayed at the right hand side accompanied by a lock icon on each side. The standard SSL persona shows only one padlock located at the same spot. The visualization through a padlock icon was not dismissed since we felt that in the meantime 7

14 3.4 Alternative Warning Message Design 3 NEW APPROACH Figure 3.1: Persona header images - green for EV SSL encryption and blue for DV SSL encryption Figure 3.2: Persona header image - broken SSL status users may have become used to this representation of security. Yet, a certificate icon is introduced and its meaningfulness to be evaluated by the user study. A screen-shot of a browser window with applied EV Persona can be found in figure 3.4. Whenever a web page is only partially encrypted, Firefox shows a small padlock icon with an exclamation mark in a red dot rendered on top of it (see figure 3.3). That symbol shows up at the status-bar at the bottom of the browser window. The site identity button does not change in a way users are expected to perceive this circumstance. Therefore an additional persona image was introduced for this case as well, showing an orange alert sign containing an exclamation mark (see figure 3.2). The background color is orange to signalize that there is a problem. The reason why red was dismissed, is the effect of scaring off people, which would be an exaggerated indication (having no encryption at all is even less secure and occurs more often). A possible flaw could be that users do not reliably understand the circumstances under which this persona is shown. It is also not to go unmentioned that the first time Firefox detects a partially unencrypted connection, it displays a warning message to inform the user that form data is possibly sent unencrypted. If the user clicks the OK -Button, the default option is to never show this message again unless the user knowingly activates the check-box for future warning. 3.4 Alternative Warning Message Design When addressing the problem of designing new warning messages, the guidelines proposed by Biddle et al. were strongly taken into account and applied to revise the appearance of such warnings [3]: Technical terms: One has to try to avoid technical terms (e.g. encryption) to make information accessible for all users. Lengthy messages: To save the users time and hopefully make them read the complete Figure 3.3: Standard Firefox Image for broken SSL status 8

15 3 NEW APPROACH 3.4 Alternative Warning Message Design Figure 3.4: Screenshot of a EV certified Website with SSL Personas applied warning, it is useful to reduce the length of the core message to a minimum. However users should be able to retrieve further information on the error in order to better assess the severity of the problem. (e.g. by clicking on a button labeled in such way) Misleading or confusing wording: this is probably the most challenging guideline as certain wording may be crystal clear to one part of users but confusing for others. It was tried not to use words like guaranteed but the usage of secure could not be circumvented. Two aspects of Egelman s recommendations were particularly taken into account [8]: Clear Choices: Warnings need to provide easily identifiable options and recommendations on how to proceed. In this approach only two buttons labeled Cancel and Continue to site will be used. Herley s postulate to save the users time was also incorporated by deciding to allow users to continue to the site with only one click [13]. Preventing Habituation: There has to be at least one factor that makes the warning distinguishable from other warnings. If all warnings look the same, people are likely to ignore them and do not notice changes in wording. A preview of the blocked site can thwart habituation, as it looks different for each website and users might be willing to take a closer look at it before they continue. Color scheme As background color, Firefox warnings use a light shade of gray (see figure 3.5). A yellow border delimits the message area wherein "Larry the passport officer" can also be found [30]. "Larry" is an icon that is intended to visualize the severity of the threat. In certificate error warnings it is always yellow, in phishing attack warnings it turns red. It also appears when clicking on the site identity button, where its colors are gray, green or blue (according to the SSL state). For the new warning design, orange was chosen as background color, as this was thought to be a little more effective when it comes to catch the user s attention. The text is colored light gray on a 9

16 3.4 Alternative Warning Message Design 3 NEW APPROACH dark gray background. Furthermore it was tried to use signal colors: The certified domain is consistently colored green, whereas the current URL (the one that triggered the warning!) is kept red. Other layout decisions If the error is caused by a domain mismatch, it is safer to follow the URL that is provided by the certificate. Hence the warning message clearly tries to convince the user to do so - using only one phrase, that talks about avoiding risks. In some cases phishing websites use domains that are very similar to the forged website s URL (and also the real site s certificate) - like that used the letter v twice to appear like the letter w. The warning was designed to display URLs with a wider letter spacing in order to facilitate the recognition of these attacks. However as phishing attacks become more sophisticated, this trick has lost its effectiveness and has become rarely used [13]. Warnings caused by self-signed certificates are often seen on a university s website. As observed by Herley and others, there is only a negligible percentage of phishing attacks that rely on selfsigned certificates. Therefore the warning suggests to proceed, if the user purposely visited this site. Since the warning consists of very little text, users might be wanting more information, which is granted by clicking a corresponding link. Doing so will result in toggling an extra paragraph explaining the concept of certificates and why this message is shown at this particular moment. The new design is shown in figure 3.5 at the bottom. 10

17 3 NEW APPROACH 3.4 Alternative Warning Message Design Figure 3.5: Warning designs - Baddomain Error. Standard Firefox warning on top, redesigned version at the bottom. New warning at the bottom features site preview to prevent habituation, signal colors, reduced text, no technical terms, clear choices and recommendation 11

18 3.4 Alternative Warning Message Design 3 NEW APPROACH 12

19 4 IMPLEMENTATION Figure 4.1: Categorization of Add-Ons 4 Implementation In this section it will be explained how the design presented in the previous section was implemented. It shows how the browser extension works and what it relies on. Note that it seems more reasonable to describe the concepts and ideas rather than showing code examples, since the source code is available in the appendix and could easily be deduced from the herein presented concepts. Add-Ons for the Mozilla Firefox Browser can be divided into three main categories: Extensions, Plug-ins and Themes (see figure 4.1). Extensions enhance the browser s capabilities through overlays and scripts. They rely on the browser and cannot be run as standalone programs. Contrarily, plug-ins are usually derived from already existing pieces of software, like media players or document viewers. They only provide a type of link to the browser interface which allows the user to remain inside the browser window to view special content - without having to switch applications each time. Common plug-ins are the Adobe Flash Player or PDF viewers. Themes are targeted at styling the looks of the browser and usually have no further features. Additionally, they are not granted as many rights as extensions or plug-ins thus they cannot run program code. One can divide them into Heavy-Weight and Light-Weight Themes. The former can change the entire look and feel of the browser and can skin every single dialog. They usually bring their own button designs. On the other side, Light-Weight Themes only provide basic customization by changing the background color (or image) of both the navigation bar at the top and the footer at the bottom. The term Persona has established for such a theme (only for Mozilla software products). In this work the terms extension, add-on and plug-in will be used equivalently to express a program that adds features to the Firefox browser. The chosen name for the resulting extension is SSLPersonas. XUL and JavaScript The plug-in was programmed using mainly JavaScript and XUL (XML User-Interface Language; developed by Mozilla)[18]. Some of the certificate error page functionality relies on XHTML and CSS. XUL is a markup language which lets the programmer arrange interface elements such as buttons, spacers, menus etc. Usually Add-Ons work as overlays to the original browser chrome and thereby extending it. For example, overlays can add menu items to the bar at the top of the browser. Without any further programming, these XUL elements do not react to user interactions, but are simply lay-outing components. JavaScript provides the necessary tools to achieve interactivity: EventListeners. The Observer Pattern is most frequently used when attaching an EventListener to 13

20 4.1 Used Tools 4 IMPLEMENTATION a XUL element [10]. Events are triggered if something happens to these elements (e.g. the user clicking on it) and the EventListener then calls a previously registered function. 4.1 Used Tools Eclipse was chosen as Integrated Development Environment (IDE), enhanced with the Orangevolt EclipseXUL plug-in [11]. The plug-in provides among others wizards, editors and launch configurations for Mozilla extension programming. To pack the code into an installable.xpi file, the Firefox Add-on Extension Developer was used [15]. 4.2 Functionality There are a number of features that are required for the final plug-in. The SSLPersonas extension should change the persona image according to the current SSL state. As mentioned in section 3.3, there are 4 different states that have to be detected: No SSL, partial SSL, DV SSL and EV SSL. The personas should be individually set for each tab. The user should also be able to pick a default persona for the No-SSL case and to change it whenever she wants to. Furthermore, the plug-in should also be capable of overriding Firefox s standard certificate error warning pages and display enhanced messages, enriched with a preview of the site that is temporarily blocked as well as a clearer text-message. Once the user has decided how to proceed when facing a warning message, she should be able to execute the decision very fast and the browser should not delay the accomplishment of the task any longer. Therefore it should be made possible to either add an exception with a single click on the Continue to site button or to leave the site by clicking the Leave button. If the user opts for continuing to the site, it is visible right after clicking on the corresponding button. The extension is very passive with little interaction by the user. The only point of interaction is the settings dialog which offers the user to activate the enhanced warning pages and permanent exceptions. Everything else is running in background. UML class diagram In the appendix, an image can be found which shows an UML class diagram of the most important modules, operations and attributes (img/class_diagram.png). Note that the global variables are not actually in a package, yet this visualizes the logic structure of the components quite well. 4.3 Challenges When implementing the above mentioned requirements and features there were several challenges to take. The presented solutions form the functional principle of the SSLPersonas JavaScript file General principles The Firefox browser uses XPCOM 2 modules to organize interfaces. Those interfaces can easily be modified without needing to recompile the whole browser. Also, Firefox is able to load only the implementing classes it needs, to save resources [12]. Every class that implements such an interface has to inherit from the nsisupports interface and has to implement the QueryInterface function. All this enables keeping track of XPCOM Objects [12]. Firefox organizes its XPCOM modules into four packages: Components.classes, Components.interfaces, Components.results and Components.utils. Adding functionality to Firefox will often result in implementing interfaces in Components.interfaces. 2 XPCOM = Cross Platform Component Object Model 14

21 4 IMPLEMENTATION 4.3 Challenges Components.classes and Components.utils mostly provide predefined useful classes and functions, such as the Components.utils.import(url[,scope]) function which facilitates addressing JavaScript code modules for your own code or splitting the program-logic of large projects [16] Detecting encrypted connections One of the main functionalities of the plug-in is to detect whether the currently displayed site is secured over an encrypted channel. Once such a detection has taken place, the security indicator has to show up. In order to achieve this the nsiwebprogresslistener interface has to be implemented. It provides several functions that will be triggered when certain events occur. The relevant events in our case are LocationChange, StatusChange and SecurityChange. All other events that are passed to interface will not be taken care of. In the events mentioned above there are certain steps to be taken: The plug-in has to find out whether the event was triggered by the site loaded in the currently selected tab, so the personas are not accidentally changed when a tab with a different security status is loaded in background. The global variable gbrowser offers functions for doing so. We have to check what the new security state is. Each state is a unique numerical value (hexadecimal), which either is contained in the event or can as well be read from the gbrowser variable (the latter is a lot more elaborate). Finally, the extension needs to react to the new state, i.e. change the persona. The numerical value of the state is compared to a constant value in the nsiwebprogresslistener interface. There are four different constants representing the main security states: STATE_IS_INSECURE (no SSL), STATE_IS_BROKEN (partially encrypted content), STATE_IS_SECURE (using SSL) and STATE_IDENTITY_EV_TOPLEVEL (SSL with Extended Validation Certificate). While there are no further distinctions in the insecure, broken and toplevel state, the secure state is divided into STATE_SECURE_LOW, STATE_SECURE_MEDIUM and STATE_SECURE_HIGH. For some reason, some sites without SSL encryption trigger STATE_IS_SECURE events - the Mozilla documentation does not give further explanation on why this happens. Therefore it is necessary to check if the state matches STATE_SECURE_HIGH as well, as soon as the current state is STATE_IS_SECURE Detecting tab switches There is also a need to react whenever a user switches tabs. The security indicators have to be changed according to the currently displayed tab, as a consequence of which an Event- Listener has to be implemented that listens for the TabSelect event. It is attached to the gbrowser.tabcontainer object on start-up Applying Personas LightweightThemeManager When applying the custom designed personas, there was a challenge of making them available from local storage instead of downloading the SSL Personas each time they are needed. However, Firefox s default LightweightThemeManager makes sure that personas are downloaded each time the browser is started or the theme is changed. One can presume that this is due to the fact, that the intended usage for Personas is not to change them every minute, and therefore the network traffic is affordable. Furthermore Mozilla keeps track of daily usage of certain Personas to build a popularity ranking and for capacity planning, which is explained in the Firefox Privacy Policy [17]. This could not be done if every persona were stored permanently 15

22 4.3 Challenges 4 IMPLEMENTATION once downloaded. The standard LightweightThemeManager therefore only allows URLs for the persona images that use the HTTP or HTTPS protocol - a sanitizing function thwarts usage of the file:// or chrome:// protocol. The solution for this particular problem was to copy the original LightweightThemeManager code module and remove the sanitizing function, so we can distribute our custom persona images within the installation package and address them using the chrome:// protocol. As a result, SSLPersonas comes along with its own LightweightThemeManager, that is addressed only by the plugin when needed - the standard Theme-Manager keeps running. To enable the new LightweightThemeManager a resource:// protocol pointing to the extension has to be registered in the manifest file. Proper shut-down When the browser is closed either manually or by system shut down, the SSLPersonas extension has to make sure that the standard persona is applied. If this did not happen and the user exited the browser currently displaying a secure site, the persona would be regarded as the standard theme for the next startup and falsely indicate a secure site any way Replacing Warning Messages Functionality Firefox does not have any documentation regarding the replacement of warnings for certificate errors. However there is a preference branch 3 named security.alternate_certificate_error_page that has to point to an about: protocol page. Therefore a custom about: page has to be registered, quickly done deploying functions provided by the XPCOMUtils module to register the extension as official component. Afterwards we simply set the preference value to our custom about: page using the gprefservice variable. One-Click Exceptions As the warning page is only an XHTML document as any other website, it has restricted privileges and cannot directly modify the browser-core s behavior by executing JavaScript - for obvious security reasons. To work around this problem (as does the Firefox browser itself), we include the XUL namespace in the XHTML document and make the buttons clickable XUL-Elements. These buttons are able to trigger events, that normal HTML buttons cannot and are given IDs that the extension s JavaScript code knows. The next step is to attach an EventListener to the gbrowser variable, that listens for command- Events, which are triggered by clicking the XUL buttons. In order to guarantee the integrity of the event, the EventListener implemented first checks, if the event was triggered by one of the two buttons in the warning page by matching the IDs. The event object also has a property (istrusted - boolean) that reveals if the event was triggered by a website or browser interface element respectively the user (the latter will return false). Firefox provides classes that implement the nsicertoverrideservice interface, which accesses the certificate database and can be used to instruct Firefox to trust a certain certificate. This is done by calling the remembervalidityoverride() function with the correct parameters. One can also specify if the certificate should be trusted permanently or temporarily (until the browser is exited). The SSLPersonas plug-in provides an option in its settings-dialog which lets the user decide. The default option is to permanently add the exception, since Firefox also remembers exceptions permanently by default. 3 Firefox organizes its preferences in preference branches, where important behavior is configured. The configuration can be viewed and modified either manually by calling about:config through the address bar, or through programm code by using the preference service Object. The gprefservice object is often used for this. Extensions usually store their preferences in the extensions. branch, but it is not necessary to do so. All installed extensions can access the preference system and can change each value. Therefore a cautious treatment of the configuration is important as changes might affect other modules behavior or even increase security vulnerability. 16

23 4 IMPLEMENTATION 4.3 Challenges Further Features The warning messages use JPEG Previews provided by PageGlimpse and are gathered by an XMLHttpRequest containing an API key [23]. An animated loader image is shown while the image is queried, to give the user some feedback. The Gecko engine cannot be used to render the site and to draw it onto an HTML5 canvas afterwards. This is due to the fact, that the exception does not exist when the render process starts, and it would return an image of the warning page. It was tried to work around this problem as it would probably reduce network traffic and increase the speed, but this endeavor did not succeed. Nevertheless PageGlimpse is a very powerful alternative that generates the images fairly quickly and can be deployed without concerns. Some simple JavaScript takes care of the correct insertion of URLs into the generic warning and selects the warning text according to the error-code. Currently SSLPersonas only addresses problems regarding untrusted or unknown issuers and domain name mismatches. Yet, there are a few other errors like untrusted certificate authorities or expired certificates. The latter are more uncommon and will be taken care of in a later version. 17

24 4.3 Challenges 4 IMPLEMENTATION 18

25 5 USER STUDY 5 User Study In this section the process of evaluating the new approach towards effective security indicators and certificate error warning is portrayed. In order to obtain evidence of the effectiveness, a user study was conducted. After describing the preparations and goals, the results are presented and discussed. 5.1 Design and Arrangements Goals and Overview We asked 24 participants to take a look at 14 bookmarked sites and provide their opinion about trustfulness by filling out a paper questionnaire. First of all, it was necessary to find out whether the participants would notice the personas for each SSL state. Moreover it had to be evaluated if they perceived the personas as security indicators. A second more important goal was to determine if personas can affect the security awareness in a way that users tend to feel safer on secure sites and become considerate when faced with scarier personas. If this were successful, it would imply that users prefer flashier security indicators. The third goal deals with warning messages: Participants were exposed to certain websites that cause certificate warnings. It was investigated if a new design is understood better than the standard warning and if users can then be lead to a safe decision on how to proceed. Besides, it was interesting to hear what people think about the modified warning concerning visual design and layout. A between subjects design was used dividing the participants into two groups; one group used a Firefox browser equipped with the SSLPersonas plug-in for the browsing tasks and the other group (the control group) used an ordinary Firefox (version 3.6) with no extensions installed. The main goal was to analyze, if the SSLPersonas group performed better at the tasks than the control group, i.e. easier find hints for the trustfulness of a website and add exceptions for certificate problems Ethical Guidelines As we presented two actual phishing sites during the study we did not make the users enter any information on any of the websites but instead only asked for their impression to ensure ethical user study guidelines. The only interaction allowed was to scroll up and down, and on certificate warnings choose an option. Participants were asked to sign a consent form which stated that before the study begins, all of their questions were answered the experimenter would make a protocol of their actions the gained information will only be used for research purposes all collected data will be made anonymous the participation can be revoked at any point At the end of the study the users were debriefed and answered further questions Recruitment Participants were recruited by posting information on a forum that is mainly frequented by LMU computer science students. Furthermore a Facebook event was created and s were sent to the author s friends and acquaintances. Participants were told the study was about Security awareness and they were offered a small reward in the form of a bar of chocolate. Also, LMU students 19

26 5.2 Study Procedure 5 USER STUDY Figure 5.1: Setup of the user study were made aware that participating in this study can contribute to the HCI lecture that requires them to attend one user study of their choice. The only requirement was to be familiar with internet browsing - no reservations concerning age or used internet browser were made. Among the 24 participants, nine were recruited by the forum post, six responded to the facebook event. The other nine participants answered the , have been invited by someone else (two participants) or occasionally showed up because of the signposting in the building. 5.2 Study Procedure The study was conducted in the LMU Media-Informatics department at the center of Munich. After signing the consent form, participants were asked to read a short introductory text that informed them about the tasks they are about to perform (the handout sheet can be found in the appendix). Thus both the control and the experimental group had the same starting situation. The experimenter then proceeded to hand out the first two pages of the questionnaire that dealt with the personal experience and demographics of the participants. As soon as they were filled out, the participants were allowed to perform the computer tasks. The browser had already been started beforehand and all the participants had to do was to click on 14 numbered bookmarks in the bookmarks tool-bar. The experimenter randomized the order in which the bookmarks had to be clicked and handed the corresponding questionnaire page to the participant before each single task. The computer was a desktop computer equipped with Windows XP and an ordinary computer mouse. A photo of the setup can be found in figure

27 5 USER STUDY 5.2 Study Procedure Figure 5.2: Phishing sites chosen for the user study Group Assignment The participants were able to choose any of the open dates from an online Doodle organizer. The experimenter did not have influence on the dates chosen. Therefore an alternating group assignment could be used. The first participant was assigned to the experimental group, the next one to the control group. At the end there were an even number of participants and no further adjustments had to be made. All participants were using the same scenario Study tasks We asked the participants to perform 14 independent Internet browsing tasks by clicking on a bookmark icon. The instructions remained identical for each task, but the the questions asked differed in certain cases. They were not permitted to return to the previous task. One set of tasks (whose questions could be spread throughout the user study) focused on the perception of security of certain sites and was presented when the participants had to click on a bookmark that resulted in a change of SSL state (and thus possibly in a change of the persona). There were four different categories according to the ssl state in this set of tasks. Additionally two phishing sites that did not use the HTTPS protocol respectively any certificate were presented during the user study. These sites offered login forms and were almost perfect copies of the targeted brand s website. There were only very few cues inside the content window that these sites are phishing attacks. Figure 5.2 shows two screen-shots of the phishing sites. The second set of tasks focused on the behavior while confronted with a warning message. Another two categories were added: the domain mismatching and the untrusted issuer error. Assuming that the familiarity with certain web sites might impact on the participants assessment, it was decided to use two URLs for each category - one that participants expectedly know and one which with they are completely unfamiliar. Here is an overview of the 7 different categories of tasks: 1. EV Certificate: Two of the sites used valid EV Certificates. The control group had the ordinary Firefox security indicators: the indicator, the lock icon and the green site identity button. The experimental group additionally saw a green persona (see figure 3.1). 2. DV Certificate: two sites had basic SSL encryption. The control group saw the indicator, the lock icon and the blue site identity button. The plug-in group was presented a blue persona (see figure 3.1). 21

28 5.3 Hypotheses and Questions Asked 5 USER STUDY Type # Known Site # Unknown Site EV Certificate DV Certificate online.alandsbanken.fi Partial SSL No SSL Phishing (No SSL) 14 [ebay phishing site] 13 [hsbc phishing site] Warning (domain mismatch) Warning (untrusted issuer) 9 amazon.de 10 browser.garage.maemo.org 11 webmail.ifi.lmu.de 12 Table 5.1: Sites shown to the participants. All Sites except #7, #8, #13 and #14 used the HTTPS protocol. 3. Possible insecure content: Another two of the sites chosen did have some unencrypted content. Firefox was not instructed to pop-up a warning message. The control group saw a lock icon with an exclamation mark on top. The experimental group however was warned with an orange persona including an alert sign (see figure 3.2). 4. Login Sites without SSL: The study showed two sites that had login forms within their content but did not use any encryption. No security indicators were shown either to the experimental or the control group. These two websites were chosen to find out whether the participants were less likely to login on Non-SSL websites. 5. Non SSL Phishing sites: Two phishing sites that did not use SSL were among the bookmarks. Firefox was set up not to display a warning message, which it normally would have, since the sites were already blacklisted. 6. Warning - Domain Mismatch: Two of the bookmarks pointed to URLs that did not match the transmitted certificate URL and caused a certificate warning message to appear. Participants were allowed to make a decision how they would act in such a case, thus buttons could be clicked. The plug-in group was able to continue to the site with a single click on the according button while the control group had to click at least 3 times to finally proceed to the site. Furthermore a link was suggested by both warning designs that did not result in adding an exception but instead correcting the domain mismatch error. 7. Warning - Untrusted Issuer: The last two sites used a self-signed certificate which triggered another warning message very similar to the domain mismatch warning. However there was no clickable link that suggested the use of another URL since this is only possible if the certificate provides an URL that is different to the current one. Table 5.1 lists the exact URLs of the websites displayed during the user study. 5.3 Hypotheses and Questions Asked Hypotheses The features that were implemented hopefully entail a few improvements when browsing the web. These improvements are represented by the following hypotheses: H1: Personas indicating the presence of an SSL certificate will increase the trustfulness of a site. 22

29 5 USER STUDY 5.4 Results H2: The persona that indicates only partial encryption of website content will reduce trustfulness. H3: The absence of a positive persona will reduce trustfulness. H4: Using the newly designed warning message displaying a website thumbnail, reduced text without technical terms and easier accessible options will ease the understanding and help users choose the correct option. In order to provide evidence to these hypotheses, the asked questions were worded in such way while not revealing the intention of the question Questions Asked While looking at the site, participants of both groups were given a set of questions on a paper sheet. They were able to fill it out on their own. For all SSL state related websites (categories 1 through 5) participants were asked four questions: First of all they had to tell, if they knew the current website, allowing it to check whether the choice for known and unknown websites did hold. After that, people were asked to rate the suspiciousness on a five point Likert scale ranging from -2 ( This website seems suspicious to me ) to +2 ( This websites seems to be trustworthy ). The next question asked the participants about their willingness to login to the current site. The scale ranged from -2 ( I definitely would not login on this site ) to +2 ( I would login to this site without any concerns ). The last question concerned the obviousness of security for the site. The scale ranged from -2 ( I cannot see whether this site is secure ) to +2 ( There are enough signs that this site is secure, respectively insecure ). People were provided a commentary field in which they were asked to put every indicator they used for answering the questions, allowing us to verify our hypotheses. The sites that triggered warning messages (categories 6 and 7) had a different set of questions. People had to make an assessment and again rate the statements on a five point Likert scale ranging from -2 to +2. The questions covered the following topics: If participants understood warning message content How easy it was to understand the warning If the entire text was read Whether the text was too long or too short How severe the danger was that triggered the warning. 5.4 Results This passage lays out the findings of the user study based on the participants given answers Demographics Control Group The twelve participants that did not use the SSLPersonas extension were in average 26 years old, the youngest being 17 and the oldest being 45. They used the Internet approximately 4.2 hours in average each day (SD 2.6). Two thirds of the control group were male. Nine participants used the Internet for online banking, and all of them for shopping and communication. Among the 12 participants in this group, half of them were currently studying in computer science related disciplines. 23

30 5.4 Results 5 USER STUDY Experimental Group The demographics of the experimental group were quite similar. Almost two thirds (58 percent) were male with an average age of 23 years, the youngest being 14 and the oldest being 30 years old. They used the Internet about 3.3 hours in average each day (SD 1.89). 10 people used the Internet for shopping, 11 for online banking and all 12 for communication. Four of the participants in this group were studying computer science related disciplines at that time Persona Results Both the persona and the non-persona group were unfamiliar with all sites that were chosen for this purpose, with only very few exceptions (only in the SSLPersona group). Therefore one can state that in general the choice was successful. For the following paragraphs, scores refer to the applied five point Likert scale ranging from -2 to provides an overview of the average scores for each condition. Evidence for H1 Regarding trustfulness of the Extended Validation sites, the persona group scored an average of 1.63 whereas the control group scored only People of the experimental group were also more likely to login to the EV certified websites: The score of 1.16 is better than the control group s score of When asked if it was easy to determine if the site was secure, the plug-in group (0.96) also did better than the control group (0.45). As these generic results are not statistically significant there is however one positive exception: the unfamiliar HSBC banking site was significantly (p = 0.043) more trusted by the experimental group. Therefore the results clearly indicate (albeit qualitative) that H1 is confirmed. Evidence for H2 In case of websites loading partially unencrypted content, lower scores are better since unencrypted content can be eavesdropped or data can be lost. One can see in figure 5.3 that the plug-in achieved its best scores here - the absolute difference between the average scores is 0.96 (Trustfulness) and even 1.0 (Login Secure). These results are statistically significant (p = and p = 0.012). As a consequence, H2 is confirmed. Evidence for H3 Although the plug-in group has slightly lower (and in this case better) scores this is probably due to chance and thus not significant. Therefore H3 cannot be confirmed. Section tries to explain why this is the most difficult hypothesis to provide evidence for Warning Results The certificate errors that caused warnings did not represent any danger at all. The Participants either could leave the site (by clicking a button or simply closing the tab), add an exception to be able to visit the site or - in case of a domain mismatch error - they could also follow a link that would guide them towards the correct (certified) url. Usually adding an exception for the certificate is the best option and in the study it was considered a correct action. Choosing to follow the provided alternative link is safer and militates for the usage of this particular warning message design, but one has to keep in mind that the next time the very same URL is visited, the warning message will reappear. Self-signed Certificates If the warning message was caused by a self-signed certificate, participants had only two options (leaving or adding an exception). In the control group with Firefox s standard warning 50 percent decided to add an exception, while the plug-in group added an exception in 79 percent of the warning messages dealing with untrusted issuers. 24

31 5 USER STUDY 5.4 Results Figure 5.3: Average scores for control and experimental group. A green difference means that the the plug-in group did better Domain Mismatch The incorrect action in case of a domain mismatch warning was to leave the site, which a third of the control group and only 17 percent (a sixth) of the experimental group did. Also a quite stunning percentage of people who saw the modified version of the warning opted for the alternative link (46 percent) whereas the standard warning only made 8 percent of the participants in this group do this. Figure 5.4 visualizes the actions taken by the participants. Evidence for H4 The percentages shown above indicate that the modified warning design reduces incorrect decisions and encourages people to continue to the site either via adding an exception or by following the secure link. Additionally the questionnaire participants were asked to fill out also provides hints that the standard warning message text contains too many technical terms and the new design does not. The results are statistically significant for three of the four bookmarks that generated warnings (bookmark #9: p=0.012, #10: p=0.021 and #12: p=0.030). The warning text was found to be too long in the standard warning (statistically significant for bookmark #9: p=0.006 and #12: p=0.042). Finally bookmark #9 shows that the new design makes it easier to take an action (p=0.026). The combination of these results partially confirm H4 but due to the fact that the actions taken were not significantly more correct in the experimental group, this hypothesis cannot be fully confirmed Qualitative Results At the end of the user study each participant was shown some printed pictures. In these pictures one could see screen-shots of both a standard Firefox and a Firefox window modified by SSLPersonas. Secure Persona vs. Standard Persona First, participants were asked to take a look at a screenshot of The modified version displayed a green persona while the standard version was skinned with a persona retrieved from The question asked was which one of the two pictures makes it clearer that the website is safe. As one can see in figure 5.5, an absolute majority of 75 percent of the participants chose the plug-in equipped browser. 25

32 5.4 Results 5 USER STUDY Figure 5.4: Participants actions when facing warning messages. Suggestion = Follow the suggested secure link, GetMeOut = leave the site, Exception = continue to the site. Red color indicates an incorrect decision. Enhanced Warning vs. Standard Warning The second comparison participants had to make concerned the warning message design. They were shown screen-shots of both the modified and the standard warning before they were asked which one they preferred. Here the majority voted for the enhanced version as well (71 percent) Protocol Evaluation and Informal Feedback During the study, some statements of the participants were protocoled. Additionally people were asked to leave a comment on the questionnaire whenever they had filled out the page. There are some interesting factors one has to take into account when developing the next version of the SSLPersonas plug-in or considering the limitations of the user study. Browser Chrome Perception The overall impression was that some people do not really use browser chrome or other indicators in the first place to determine whether a specific site is secure. The comments on the paper sheets sometimes indicate that a lock icon in the content area of the window is considered a security indicator (which corroborates Wu s statements that phishing toolbars do not work [33]). Or even if the word secure appears somewhere in the page (as it was the case for one of the presented phishing sites), this site was regarded innocuous. Since the related work already pointed out this phenomenon, one site was chosen for the study that was neither in German or English language but in Finnish. It was a Finnish online banking site that used a DV certificate. Thus it was expected that participants would be forced to look at the browser chrome to make an assessment regarding the trustfulness of the site. Contrary to the expectations some participants showed a completely different behavior and uttered sentences like Honestly, I really can t tell if this site is secure, because I don t understand a single word. or even If a site is not in my language, I don t think it s safe. New Warning Design One aspect that was often mentioned was the seriousness or professionalism of the new warning design compared to Firefox s design. Participants clearly stated that the standard warning looks more serious and goes along well with the Firefox chrome whereas the 26

33 5 USER STUDY 5.5 Discussion Figure 5.5: Choices for persona indicator vs. standard indicator and new warning design vs. old warning design modified design is too colorful. Thus the warning was found not to be trustful and might as well be forged. However many participants mentioned that the website preview is useful and desirable. The reduced text was felt to be reasonable as people claimed they would not read the whole standard warning anyway. Acceptance of SSL Personas Especially the plug-in group commented that the green color of the EV certificate persona would allow them to feel safer on a web site. The two padlock icons were often mentioned. It is also remarkable that one participant claimed in the comment-field to have made use of the missing padlock icons to detect the phishing site and uttered her mistrust. That means that a habituation must have taken place during the rather short user study. Another participant figured out what the plug-in actually does and also mentioned that there was no reaction from it on the phishing sites. False indicators It is not to go unmentioned that people interpreted certain signs as security indicators even if these have nothing to do with security in general. One participant commented that the missing shortcut icon ( favicon ) on one site makes it appear unsafe. Someone else mentioned that if the site were attempted fraud he could establish claims against the companies that appear in its references section. Someone said that names like ebay, Google or PayPal were commonly known brands and therefore the sites are secure. The VeriSign Logo (which appeared within the content area) was often mentioned as a security indicator. One phishing site linked to the targeted brand s original disclaimer which was also found to be a sign of security. These are just a few examples demonstrating the still very low understanding of security indicators among technically non-adept Internet users. 5.5 Discussion Looking at the results gained through the user study, certain limitations have to be taken into account. This section covers the most important points of criticism one could bring in and explains why certain hypotheses could not be confirmed. 27

34 5.5 Discussion 5 USER STUDY Study design Considering the study design and its preparations there are some factors one could find to explain the lack of statistical significance alongside other shortcomings. Additional group It was not explained to the participants in the experimental group what the plug-in does. Therefore they were needed to interpret changes of personas themselves. If this failed, they were likely to go back to looking for security clues inside the content area, which evidently is fatal. The study design could have been enriched with a third group that would be informed about the functionality and purpose of the plug-in. That makes sense since voluntarily installing an Add-On on one s own browser presumably results from having read an outline of the functionality. Number of participants Sometimes one of the two groups performed better indeed - but without statistical significance. The significance normally should not rely on the size of the sample but having only 12 participants in each condition may limit the provability of the plug-in s effect. Study Topic Participants knew the study was about security matters. In the lab they might have behaved differently than they would at home, since the typically secondary goal security became a primary goal and might have distorted the answers given. Often times researches use a sort of an alibi study subject to circumvent priming on security. Doing so as well might have lead to different results in our study Habituation Security indicators do rely on a certain level of habituation. It is almost impossible to get used to looking for a newly introduced indicator during a 30 minute user study. Since the phishing sites were also within the randomized order of bookmarks, it is possible that they appeared right at the beginning when the participants of the experimental group had never seen an SSL Persona at this point. Thus the results regarding phishing site detection are not as convincing as they might be if the phishing sites were presented at the end of the study Scariness of Partial Encryption The participants who saw a persona on a partially encrypted site were more likely to mistrust it and felt less secure. This shows the dramatic impact of negative feedback. However, sites without any encryption are even less secure, but since they did not cause any persona change they achieved higher scores for trustfulness in the experimental group. This leads to the conclusion that a strong warning about partial encryption makes users think they are less secure - which is not the intention. Netcraft found out through surveys that in November 2009 about 1.2 million web sites used SSL certificates [21]. Compared to the estimated total number of 234 million websites in December 2009 this is a very low percentage (about 0.5%) [24]. As a consequence, warning the user on 99.5% of websites about security is absurd. For the case of no SSL encryption other criteria have to be taken into account that allow browsers to effectively present information on identity confidence (see suggestion in section 7.2). 28

35 6 IMPROVEMENTS MADE 6 Improvements Made Figure 6.1: Revised design of the certificate error warning Based on the feedback given by the participants of the user study the Add-On was adjusted and improved. 6.1 Warning Design Since the color scheme was found to be non-serious, it was assimilated to the standard Firefox design. Thus the orange background made way for a light gray. Also the rather cheerful signal colors were reduced in brightness so now they are darker and not too catchy. The background of the warning area itself is now white and the text-color is black - just like the standard warning message s colors. Furthermore the elements are now arranged vertically so that there is a clearer top-down hierarchy in the layout. The website thumbnail was chosen to go into the center of the warning and become the most important eye-catcher as this was approved by many participants of the user study. The wording and the length of the text was not changed since there was very positive feedback. A picture of the revised version can be found in figure Possible reasons against the plug-in Although now improved there are still some aspects that might hinder the usage of the SSLPersonas plug-in. In this section possible obstacles for the wide acceptance are illustrated Customization Personas are a fast way to change the looks of the browser and can be regarded as an integral part of the user s individuality. If the user chooses a certain persona she may wish to maintain it for a longer time. The SSLPersonas extension however switches the persona quite often. Security is 29

36 6.2 Possible reasons against the plug-in 6 IMPROVEMENTS MADE then prioritized over customization which every user would have to acknowledge and there will be some that do not want to. Heavy Weight Themes Furthermore some Firefox users employ heavy weight themes to give their browser a whole new look - and feel. Unfortunately, deploying SSLPersonas brings along the need to abandon these themes, which some users will not be willing to do. Likeliness to other personas The current color scheme of the SSL personas is green, blue and orange. If one uses a default persona whose colors are similar to one of the SSL personas, a change might go unnoticed and therefore not provide any improvement in SSL awareness. This is probably a shortcoming in comparison to Dhamija s dynamic security skins [6] Interference with other Plugins The Personas Rotator plug-in offers to switch the persona in an adjustable time interval. If this plug-in is installed alongside the SSLPersonas extension, changes that were caused by SSLPersonas will probably go unnoticed. Also, if the Rotator changes the persona while an SSL persona is displayed, the purpose of the SSL persona is destroyed and the user is left with the standard security indicators. Thus the habituation to SSLPersonas can be interrupted. Once SSLPersonas has taken an action, the current implementation of the Rotator stops, which results in its futileness. This flaw could easily be fixed by restarting the rotation process once the LightweightThemeManager notifies the observers Newer Firefox Versions As of writing this work, the most recent Firefox version available is 4.0b5. This release brings a massive change of the user interface. It now enables the Microsoft Windows Aero Surface to render parts of the browser chrome semi-transparent. However this thwarts the usage of personas in general and of SSLPersonas in particular. If one decides to use a persona skin for Firefox 4, the transparency feature will be disabled. Since the demands for this very feature have persisted for several years, it is expected to be highly welcome to the users and only few will relinquish it. However the effectiveness of SSLPersonas might be improved if the user decides not to use a persona for her default theme (which looks very different to a persona skinned browser chrome), but still use personas for encrypted connections. One factor that might encourage the usage of SSLPersonas even in future Firefox version is the site identity button. The beta indicates that its new design is going to be less obtrusive that it currently is. The designers chose to reduce the importance of this security indicator. SSLPersonas might become interesting for users who need a certain degree of flashiness when it comes to SSL. These speculations can be seen as a minor sticking point in the process of spreading the extension throughout the Firefox user community. 30

37 7 CONCLUSION 7 Conclusion 7.1 Summary In this bachelor thesis the idea of using lightweight browser skins as security indicators was developed as a consequence of different previous approaches to the problem of SSL awareness optimization. After implementing and testing the resulting Firefox extension, a 3-day user study with 24 participants was conducted to evaluate its effectiveness. This user study indicated that the approach is promising as both quantitative and qualitative results were better if the browser was enhanced with the SSLPersonas extension. Enriching warning messages with website thumbnails while reducing the effort of both reading the text and adding an exception has also brought out improvement for some participants. 7.2 Possible Future Features In order to provide a higher degree of customization, one could allow people to choose an own persona image for each of the different SSL states. This provides individuality to the users who might be more likely to employ the extension. If they don t like the colors and icons in the standard SSL personas, they should be able to change it with a few mouse clicks, either by picking a persona from the gallery at getpersonas.com or by using an image stored on their hard-disk. Additionally, the warning messages should cover each error code for certificate problems and use different text messages for each case. At this point, there are quite a few error codes, where the current version of SSLPersonas displays a rather generic warning text, as these errors are less likely to appear. Future versions should deal with all of them. There is one idea the author would like to bring in, that cannot be implemented yet. For future SSL versions one could imagine additional flags for the type of business of the certified website. Bank sites could then be easily distinguished from shopping sites or web-mail providers. Relying on such a flag, the SSLPersonas extension could use further images to display this very information. It could for example display a shopping cart icon next to or instead of the padlock icon while maintaining the background signal color. Currently, SSLPersonas is intended to show people that they are safe to login on certain sites. However the need for a tool that effectively prevents users from logging into suspicious websites is even bigger. This case is covered only indirectly, as there will not be a green or blue persona. There is one fairly successful extension named Web-of-Trust (or WOT) which displays a green circle in the size of a navigation button whenever a site is found to be trustful by the WOT community. Furthermore the circle turns red if the community sees a problem with the trustfulness of certain sites. An encrypted connection is not needed to cause the circle to change its color. Since this approach seems also promising but the representation is quite unnoticeable, one might think of SSLPersonas taking care of the visualization of WOT-ranks. It is imaginable that a Persona shows the WOT logo and according colors. 7.3 Adaptation to Other Software Since the concept of using browser skins is supported by other browsers as well, the SSLPersonas plug-in could be ported to those. The browser is required to support lightweight themes to be able to switch the looks very fast - besides having an interface for extensions, evidently. Google Chrome (respectively the chromium project) is also capable of changing its looks quickly. The Opera browser does support themes although there is no particular distinction between heavyweight and lightweight themes - it does change its looks quickly even for seemingly heavyweight skins. Although Firefox seems to become the most used web browser, users that prefer other software should not be excluded and therefore the adaption is reasonable. Also it is imaginable to use such a plug-in for other software from the Mozilla Application Suite, 31

38 7.4 Outlook 7 CONCLUSION like the client Thunderbird or the personal information suite Seamonkey, which are also able to render websites. One could think of the usage of SSL personas whenever Thunderbird establishes a secure connection to an IMAP server. A lot of possible applications can be found for the concept behind SSLPersonas. 7.4 Outlook The extension can and will be improved over time. As it is published under Mozilla Public License (MPL), all of the Firefox users in the whole world will be able to download and use it free of charge. In the first week after it completed the review process 4 SSLPersonas was downloaded more than a thousand times, and rated 4 stars (of five). It was even credited in a twitter status update by the Personas developing team (among others). Some users might provide further feedback on how they experience the usage of SSLPersonas. Based on these reviews there will be a lot of potential to improve and enhance the current version of the plug-in. 4 all Firefox add-ons have to be reviewed by a Mozilla Editor 32

39 Contents of CD This work in PDF Format: A digital copy of this bachelor thesis. All images from this work: All images that are embedded in this work are provided in higher resolution in either JPEG or PNG file format. Source Code: The JavaScript Source, HTML and XUL files as well as manifest and RDF files. Installable Firefox Extension: An XPI File of the current version of the SSLPersonas extension. Future versions can be found at To install the extension, either open it with Firefox or drag and drop the file on any open Firefox window. User Study consent form: the exact consent form and introductory text participants of the user study were handed. 33

40 34

41 References [1] ADELSBACH, A., GAJEK, S., AND SCHWENK, J. Visual spoofing of ssl protected web sites and effective countermeasures. In Conference (ISPEC 2005), LNCS pp Copyrights Springer-Verlag, Heidelberg Berlin Cryptographic Attacs and Security Flaws on SSL - Denise Doberitz (2005), Springer. [2] ATTACKS, D. P., DHAMIJA, R., AND TYGAR, J. D. Phish and hips: Human interactive proofs to detect phishing attacks. In In Human Interactive Proofs: Second International Workshop (HIP 2005 (2005), pp [3] BIDDLE, R., VAN OORSCHOT, P. C., PATRICK, A. S., SOBEY, J., AND WHALEN, T. Browser interfaces and extended validation ssl certificates: an empirical study. In CCSW 09: Proceedings of the 2009 ACM workshop on Cloud computing security (New York, NY, USA, 2009), ACM, pp [4] CABFORUM. Certification authority/browser forum - extended validation ssl certificates. visited [5] CANNELLA, S., POLIVY, D. J., SHIN, M., STRAUB, C., AND TAMASSIA, R. Secure visualization of authentication information: A case study. Visual Languages - Human Centric Computing 0 (2004), [6] DHAMIJA, R., AND TYGAR, J. D. The battle against phishing: Dynamic security skins. In SOUPS 05: Proceedings of the 2005 symposium on Usable privacy and security (New York, NY, USA, 2005), ACM, pp [7] DHAMIJA, R., TYGAR, J. D., AND HEARST, M. Why phishing works. In CHI 06: Proceedings of the SIGCHI conference on Human Factors in computing systems (New York, NY, USA, 2006), ACM, pp [8] EGELMAN, S., CRANOR, L. F., AND HONG, J. You ve been warned: an empirical study of the effectiveness of web browser phishing warnings. In CHI 08: Proceeding of the twentysixth annual SIGCHI conference on Human factors in computing systems (New York, NY, USA, 2008), ACM, pp [9] FELTEN, E. W., BALFANZ, D., DEAN, D., AND WALLACH, D. S. Web spoofing: an internet con game. In In Proceedings of the 20th National Information Systems Security Conference (1996). [10] GAMMA, E., HELM, R., JOHNSON, R., AND VLISSIDES, J. M. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional, Amsterdam, NL, [11] GERSMANN, L. Eclipsexul. visited [12] HALLARAKER, O., AND VIGNA, G. Detecting malicious javascript code in mozilla. Engineering of Complex Computer Systems, IEEE International Conference on 0 (2005), [13] HERLEY, C. So long, and no thanks for the externalities: the rational rejection of security advice by users. In NSPW 09: Proceedings of the 2009 workshop on New security paradigms workshop (New York, NY, USA, 2009), ACM, pp [14] HERZBERG, A., AND JBARA, A. Security and identification indicators for browsers against spoofing and phishing attacks. ACM Trans. Internet Technol. 8, 4 (2008),

42 [15] MIELCZAREK, T. Extension developer firefox addon. de/firefox/addon/7434/. visited [16] MOZILLA. Components.util.import function. [17] MOZILLA. Firefox privacy policy. firefox-en.html. visited [18] MOZILLA. Xul. visited [19] MOZILLA. Site identity button. Identity+Button, visited [20] MOZILLA. Personas - getting started. started, visited [21] MUTTON, P. 24 of the 100 top https sites now safe from tls renegotiation attacks _top_https_sites_now_safe_from_tls_renegotiation_attacks.html. visited [22] OLLMANN, G. The phishing guide (part 1). Phishing.html, visited [23] PAGEGLIMPSE. easy website thumbnails. visited [24] PINGDOM. Internet 2009 in numbers. internet-2009-in-numbers/, visited [25] ROESSLER, T., AND SALDHANA, A. Web security context: User interface guidelines visited [26] SCHECHTER, S. E., DHAMIJA, R., OZMENT, A., AND FISCHER, I. The emperors new security indicators: An evaluation of website authentication and the effect of role playing on usability studies. In In Proceedings of the 2007 IEEE Symposium on Security and Privacy (2007). [27] SOBEY, J., BIDDLE, R., OORSCHOT, P. C., AND PATRICK, A. S. Exploring user reactions to new browser cues for extended validation certificates. In ESORICS 08: Proceedings of the 13th European Symposium on Research in Computer Security (Berlin, Heidelberg, 2008), Springer-Verlag, pp [28] STAIKOS, G. Web browser developers work together on security. org/2005/11/22/web-browser-developers-work-together-security, visited [29] SUNSHINE, J., EGELMAN, S., ALMUHIMEDI, H., ATRI, N., AND CRANOR, L. F. Crying wolf: An empirical study of ssl warning effectiveness. usenix security, [30] VAMOSI, R. Meet larry, firefox s friendly passport officer _ html, visited [31] WHALEN, T., AND INKPEN, K. M. Gathering evidence: use of visual security cues in web browsers. In GI 05: Proceedings of Graphics Interface 2005 (School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada, 2005), Canadian Human-Computer Communications Society, pp

43 [32] WHITTEN, A., AND TYGAR, J. D. Why johnny can t encrypt: a usability evaluation of pgp 5.0. In SSYM 99: Proceedings of the 8th conference on USENIX Security Symposium (Berkeley, CA, USA, 1999), USENIX Association, pp [33] WU, M., MILLER, R. C., AND GARFINKEL, S. L. Do security toolbars actually prevent phishing attacks? In CHI 06: Proceedings of the SIGCHI conference on Human Factors in computing systems (New York, NY, USA, 2006), ACM, pp [34] YE, Z. E., SMITH, S., AND ANTHONY, D. Trusted paths for browsers. ACM Trans. Inf. Syst. Secur. 8, 2 (2005), [35] ZUSMAN, M., AND SOTIROV, A. Sub-prime pki: Attacking extended validation ssl. In Black Hat Security Briefings (2009). 37

44 Appendix Nutzerstudie - Sicherheitsbewusstsein bei der Internetnutzung (Handout) Einführung: Nach der Beantwortung einiger allgemeiner Fragen zu Ihrer Person, werden Sie vom Studienleiter aufgefordert, eine Reihe von Websites aufzurufen, die Sie als nummerierte Lesezeichen unter der Adressleiste des Browsers finden. Die Reihenfolge wird Ihnen vorgegeben und ist zufällig. Sehen Sie sich bitte die Webseiten genau an. Der Studienleiter wird Ihnen einen Fragebogen pro Seite zur Beantwortung geben. Die Fragen beziehen sich auf Ihre persönlichen Eindrücke. Nutzen Sie bitte bei ihren Einschätzungen alles was auf dem Bildschirm zu sehen ist. Bitte beachten Sie: Es kann sein (oder auch nicht), dass sich der Browser (Firefox) anders verhält, als Sie es vielleicht gewöhnt sind. Einverständniserklärung Studie: SSL Awareness Institution: LFE Medieninformatik, LMU Name: Geburtsdatum: Ich wurde über den Ablauf und den Zweck der Studie aufgeklärt und meine Fragen zur Studie wurden vollständig beantwortet. Ich habe mich freiwillig dazu entschlossen, an der Studie teilzunehmen und bin damit einverstanden, dass der Studienleiter ein Protokoll anfertigt. Die gewonnenen Informationen dürfen nur für Forschungszwecke eingesetzt werden. Mir ist bewusst, dass diese Studie streng vertraulich ist. Alle persönlichen Daten und individuellen Ergebnisse werden nur in anonymisierter Form verwendet. Eine nachträgliche Zuordnung der Angaben zur Person ist nicht möglich.. Ich weiß, dass ich meine Teilnahme an der Studie jederzeit widerrufen kann. Datum: Unterschrift:

45 39