Institutionen för datavetenskap

Transcription

1 Institutionen för datavetenskap Department of Computer and Information Science Final thesis A Method for Evaluating the Persuasive Potential of Software Programs By Ammu Prabha Kolandai LIU-IDA/LITH-EX-A--12/056 SE Linköpings universitet SE Linköping, Sweden Linköpings universitet Linköping

2 Linköping University Department of Computer and Information Science Final Thesis A Method for Evaluating the Persuasive Potential of Software Programs by Ammu Prabha Kolandai LIU-IDA/LITH-EX-A--12/056 SE Supervisor: Eva L. Ragnemalm Examiner: Magnus Bång

3

4 Upphovsrätt Detta dokument hålls tillgängligt på Internet eller dess framtida ersättare från publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår. Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och för undervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. För att garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och administrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman i den omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sammanhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende eller egenart. För ytterligare information om Linköping University Electronic Press se förlagets hemsida Copyright The publishers will keep this document online on the Internet or its possible replacement from the date of publication barring exceptional circumstances. The online availability of the document implies permanent permission for anyone to read, to download, or to print out single copies for his/hers own use and to use it unchanged for noncommercial research and educational purpose. Subsequent transfers of copyright cannot revoke this permission. All other uses of the document are conditional upon the consent of the copyright owner. The publisher has taken technical and administrative measures to assure authenticity, security and accessibility. According to intellectual property law the author has the right to be mentioned when his/her work is accessed as described above and to be protected against infringement. For additional information about the Linköping University Electronic Press and its procedures for publication and for assurance of document integrity, please refer to the webpage: Ammu Prabha Kolandai.

5

6 Abstract Today, web, Internet, mobile and other ambient technologies engage in persuasive interaction with people. The technology designed to reinforce and change user s attitude or behaviors or both through persuasion and social influence are persuasive technologies. Recently, persuasive systems and services are becoming increasingly abundant and ubiquitous. Evaluation of these systems is a challenging endeavor and typically requires a simple and efficient method. Nielsen suggests heuristic evaluation as a method for intuitive, inexpensive and easy evaluation of a user interface design. The purpose of this thesis is to examine heuristic evaluation as a method to evaluate the persuasive potential of software programs. The Heuristics used were operationalized version of Fogg s persuasive design principles. Software programs MS Word, MS PowerPoint, Counter- Strike, The Sims, Stone and Plan Eat Smile were chosen with the goal of measuring their ability to change behavior or attitude of users. The Evaluation was performed on the software programs using three evaluators and the test results indicated that they possess significant persuasive potential. It was concluded that the set of heuristics which were developed can be used to perform summative heuristic evaluation and the method used was helpful in evaluating the persuasive potential of software programs.

7

8 Acknowledgements First and foremost, I would like to thank my supervisor Eva L. Ragnemalm for providing me this thesis opportunity. Her guidance and suggestions certainly made this project a lot easier to complete. I would like to thank my examiner, Magnus Bång for his presence and feedback. Thanks to Johan Åberg, for assisting me with Plan Eat Smile. I would like to especially thank my evaluators Florent and Hongzhan, who gladly offered me their time. Special thanks go to my Dad, Mom and brother Siddharth for their endless love and encouragement throughout my studies.

9

10 Table of Contents 1 Introduction Background Purpose Method Thesis Structure Literature Review Defining Persuasion Persuasive Technology Captology Types of Persuasion Levels of Persuasion Functional Triad Credibility and computers Computers as persuasive Tools Computers as a Persuasive Media (Simulation) Computers as a persuasive social actors Applications of Persuasive Systems Evaluation of Persuasive Systems Heuristic Evaluation Usability inspection methods vs. usability testing Why Heuristic Evaluation Ten Usability Heuristics General Procedure Test Objects Tools Microsoft Word Microsoft PowerPoint Computer Games Counter-Strike The Sims Educational Programs STONE Plan Eat Smile... 29

11 4 Test Conduction Test Procedure Operationalization of the principles into heuristics Test Participants Evaluator A Evaluator B Evaluator C Test Analysis & Discussion Test Observations Test Results Evaluators Agreement Comparative Study Conclusion Bibliography Appendix Appendix A: Heuristics Appendix B: Evaluation Feedback Appendix C: Results Summary Appendix D: Data Calculation... 82

12 Table of Figures Figure 1: Fogg's Captology... 7 Figure 2 : Types of Persuasion... 8 Figure 3 : QuitNet, an online smoke cessation program... 9 Figure 4 : Fogg's Functional triad Figure 5 : Key dimensions of Credibility Figure 6 : Nike Sports Kit, a persuasive tool Figure 7 : Baby think it over, a persuasive simulator Figure 8 : Smokey Bear, a persuasive social actor Figure 9 : Alcohol 101 Plus, online program against alcohol drinking Figure 10 : Powerhouse, a game for energy saving Figure 11 : Shower calendar Figure 12 : Usability problems Vs. No: of Evaluators Figure 13 : Screenshot: Microsoft Word Figure 14 : Screenshot: PowerPoint presentation Figure 15 : Screen shot: Counter-Strike game Figure 16 : Screenshot: Sims Game Figure 17 : Screenshot: Stone Figure 18 : Screenshot: Plan Eat Smile Figure 19 : Mean: MS Word Figure 20 : Mean: MS PowerPoint Figure 21 : Mean: Counter-Strike Figure 22 : Mean: The Sims Figure 23 : Mean: Stone... 44

13 Figure 24 : Mean: Plan Eat Smile Figure 25 : Evaluator Agreement: MS word Figure 26 : Evaluator Agreement: MS PowerPoint Figure 27 : Evaluator Agreement: Counter-Strike Figure 28 : Evaluator Agreement: The Sims Figure 29 : Evaluator Agreement: Stone Figure 30 : Evaluator Agreement: Plan Eat Smile Figure 31 : Mean for group of principles... 54

14 List of Tables Table 1 : Types of Credibility Table 2 : Evaluator A Table 3 : Evaluator B Table 4 : Evaluator C Table 5 : Heuristic Evaluation Form Table 6 : Five-Point Rating Scale Table 7 : Overall Mean Calculation... 37

15

16 1 Introduction 1.1 Background In today s world, people interacting with computer-based devices and systems are commonplace. We use numerous interactive products such as mobile phone, the web, calculator, video game, ticket machine, ATM and many more in our daily life. Most products require users to interact with them in order to perform their tasks e.g. playing music from an ipod or purchasing tickets online from the web. Generally, interactive products or devices are engineered as systems to accomplish a specific function or set of functions and it may not be necessarily created with users in mind. The practice of designing interactive digital products, environments, systems, and services for people s use is referred as Interaction design. It is observed as the fundamental to all disciplines, fields, and approaches that are concerned with research and designing computer based systems. The main objective of interaction design is to develop interactive products that are efficient, easy and enjoyable to use from the user s perspective. In other words, it is about building user experiences that enhance and augment the way people work, communicate, and interact (Newman & Lamming, 1995). According to ISO definition, user experience is a person's perceptions and responses that result from the use or anticipated use of a product, system or service. It is essential that the designers should know about users, technologies, and interactions between them in order to develop an effective user experience. Persuasive technology has emerged as a new discipline with the ability to influence all aspects of the way in which users interact with computers. Persuasion is intended communication that affects how others think, feel, or act toward some object, person, group or idea (Cegala, 1987). Systems which are designed to intentionally change or reinforce user s attitude or behavior through persuasion process, without involving coercion or deception are known as persuasive systems. Fogg labels this phenomenon as persuasive technology (Fogg, 2003). Persuasive design shapes a product's user experience so that the user's actual interaction with the product changes their behavior. Persuasive technologies 1

17 can be categorized by their functional roles such as tools, media, or social actors or as more than one at once which is referred as Functional Triad. It is based on how people view, use or respond to the computing technologies. Recently, Persuasive technology has rapidly established a wide range of applications in many fields of research and development like marketing, education, entertainment, health care, safety and environment. These persuasive applications are increasingly developed to have systems that can motivate and provide opportunities for positive change in behavior. Though persuasive technologies are found in many domains, considerable recent interest has centered on behavior change in health and environment domains. Here are some examples of persuasive technologies: Mind bloom is an online life game that encourages users to set goals, priorities and targets at improving the quality of life by building meaningful relationships. The game represents a user s life as a Life Tree whose branches include: Career, Creativity, Finances, Health, Lifestyle, Relationships, and Spirituality. It motivates user to take steps to fulfill his or her goals, the Life Tree grows and the user is rewarded with seeds that can be used to unlock new features. Mind bloom also incorporates a social networking aspect where users are encouraged to share their trees with friends (Kientz et al., 2010). Pos or not is an online game to raise awareness about HIV/AIDS. It shows real photographs of real people and asks the player to choose whether the person pictured has HIV positive or negative. The game result proves that it is really hard to assume a person by their looks and it raises the consciousness of the users about their body, health and motivates to get tested and to know more about HIV. Also, it demands users for postal code to locate the nearest testing centers. Users are encouraged to join the game by uploading photos and sharing their personal details (Inc, 2009). Code red rover is an interactive home safety website that offers games, activities, home safety checklists, safety tips for kids, parents, and teachers. It motivates children to make safe, smart choices in and around the home by providing home safety checklist, and template for creating escape plan. It also provides sources to know and read about other kids who are home safety heroes, or to submit a story or artwork (Council, 2004). More applications of persuasive systems will be discussed in the section 2.4. The framework and principles developed by Fogg, offers valuable means for understanding persuasive technologies but they seem to be too general when applied to design and evaluate persuasive systems. Therefore, evaluation of persuasive technologies requires a fully-operational prototype and long-term deployment (Oinas-Kukkonen & Harjumaa, 2008). Ever since it was 2

18 first introduced by Jakob Nielsen, Heuristic evaluation has been a classic and popular usability inspection method in the field of human-computer interaction. Practitioners have stated that heuristic evaluation as an efficient method to discover significant usability problems at various stages of the design process (Nielsen & Mack, 1994). Heuristic evaluation method has been widely adopted, and lately, there has been progress in the development and validation of more specialized heuristics for evaluating technologies. This thesis work helps to explore the use of summative heuristic evaluation as the method to design and evaluate persuasive technologies. 1.2 Purpose The main objective of this thesis is to examine summative heuristic evaluation as a method to evaluate the Persuasive potential of software programs. Based on Fogg s design principles, a set of heuristics are designed and developed to be more appropriate for evaluating technologies which aim at persuading users. The specialized heuristics are categorized as providing system credibility, Computer as a persuasive tool, computer as a persuasive medium and computer as a persuasive social actor. The software programs used for evaluation are tools MS Word and MS PowerPoint, computer games The Sims and Counter-Strike, and educational programs STONE and Plan Eat Smile. Using three expert evaluators, the method of heuristic evaluation is performed. This thesis shows how efficient is the heuristics developed and how effective is the method used in assessing the software programs which results in behavior change. Following Research questions will be addressed during this thesis: Does the different heuristics which are developed exhibit different behavior? Does the specialized heuristics developed can be used to evaluate different software technologies? What insights can we get about the evaluated software using the method? How effectual is the method heuristic evaluation to assess the software programs? 3

19 1.3 Method This section describes the steps and procedure used to explore heuristic evaluation as a method to examine the persuasive potential of software programs. First, the research was initiated with literature study about persuasive technology, Fogg s design principles and the method heuristic evaluation. Specialized heuristics were developed based on persuasive principles presented by Fogg. Six software programs such as MS Word, MS PowerPoint, Counter-Strike, The Sims, Stone and Plan Eat Smile were chosen with the goal of measuring their persuasive potential. It also helps to find whether the set of heuristics developed can be applied to evaluate the software technologies. Tools MS Word, MS PowerPoint is designed with no persuasive intentions and is expected to exhibit low persuasiveness on the heuristics scale. Games Counter-Strike, the Sims are expected to be persuasive by its nature but it is tough to predict. Counter-Strike being a violent game and Sims being a non-violent game, they were chosen in order to identify whether the nature of the game affects the persuasiveness and results in behavior change. Educational Programs Stone, Plan Eat Smile are assumed to engage persuasive strategies to reinforce learning. These software programs were selected because an effective method used for evaluation should be able to discriminate the programs based on their persuasive features and exhibit diverse results. Using the method of heuristic evaluation, three expert evaluators performed the test evaluation on the test objects i.e. six software programs. Each expert reviewed the software individually and reported with a feedback. Once the evaluators have worked their way through the goals set and provided their ratings and comments, it was recognized as results. This result was used to determine the persuasiveness of the technologies. Consequently, this helps to show how helpful is the method used in assessing the persuasive potential of the software programs. 1.4 Thesis Structure The thesis is organized as follows: Firstly, literature review is presented with good understanding of the topics: persuasion, persuasive Technology, functional triad, applications and evaluation of persuasive systems. Secondly, description about the test objects MS-Word, MS-PowerPoint, The Sims, Counter-Strike, Stone, and Plan Eat Smile. Thirdly, explanation about test conduction where it discusses about the procedure, specialized heuristics and test participants used to perform the evaluation. Finally, the test analysis, discussion and conclusion of this thesis work are presented. 4

20 2 Literature Review 2.1 Defining Persuasion Fogg defines persuasion as an attempt to shape, reinforce, or change behaviors, feelings, or thoughts about an issue, object, or action (Fogg, 2003). It is a form of interaction that targets at changing the way people think, feel or act. The user experience is a significant factor that should be included in the persuasion process. Researchers show considerable interest in the study of persuasion and theories from social psychology as they can be used for predicting user intentions and behavior (Harjumaa & Oinas-Kukkonen, 2007). The study of persuasion helps to improve the persuasion process and acts as a key element in designing successful persuasive systems. The information systems and services that target at persuading and motivating people by altering their attitudes and behaviors are known as persuasive systems. It is important that persuasive systems should not involve the concept of coercion and deception. Coercion implies force or compulsion and deception involves misinformation and act of deceiving people. Three successful goals of a persuasive system are reinforcement, changing, shaping of attitudes and behaviors. A reinforcing outcome means the strengthening of present attitudes, making them more impervious to change. A changing outcome means alterations in a person s response to a problem, e.g. to social questions. A shaping outcome means the creation of a pattern for a situation when such one does not exist earlier. In many cases, a shaping outcome may have a greater probability of success than interaction that aims at behavior change. Different purposes may involve the use of different persuasion strategies and techniques. Since computing technology is prevalent, it allows persuasive messages through technology to be interactive instead of altering and adjusting the design of interaction based on the characteristics or actions of the persuaded party. Researchers and industry both have developed a number of applications that use software, mobile technologies, games, and websites that encourage and motivate people. The web, internet and mobile 5

21 technologies increase the likelihood for persuasive communication as they reach people simple and faster. Foursquare and Mint are good examples of putting the concept of persuasion into practice. Foursquare is a location based social networking website for mobile devices such as smart phones. Once registered, the application persuades users to posts their location at a venue which they call "check-in" and to connect with friends. Users are also convinced to add pictures, personal information to have a profile and to choose to have their check-ins posted on their accounts on Twitter, Facebook, or both. Users are awarded with points at each check-in and it also offers badges like Newbie, Adventurer, Explorer etc., by checking into various venues and cities. Mint is a free web based application for personal financial management. It persuades people by simplifying the tasks of managing all their financial accounts through a single user interface. It has options like budget, cash flows, alerts, advices where it provides a good overview of the user s bank accounts, sending reminder to pay bills on time and suggestions to maintain a good budget. The application can also be accessed from mobile and other digital services like ipad, Tablets that act as an easy and fast way to track user s bank transactions and expenditure. Both applications use persuasive strategies like reduction, rewards, suggestions which will be discussed in section Persuasive Technology Persuasive technology is a recent research field that integrates both computing and human psychology aspects. It is defined as the technology designed to change attitudes or behaviors of the users through persuasion and social influence, but not through coercion (Fogg, 2003). While the concept of persuasive technology can sometimes comprise a range of definitions, persuasive technologies can also be categorized by whether they change attitude and behaviors through direct interaction or through a mediating role (Oinas-Kukkonen & Harjumaa, 2008). Most popular persuasive technology research centers on health, safety, environment and education. Persuasive technology can also be used in other domains like marketing, religion, politics, diplomacy, and management. A variety of interfaces have been used for persuasive technologies, including interactive web-based applications, computer and video games, computerized toys and many more. On the other side, persuasive technology could also be used 6

22 unethically. For instance, People might be convinced to buy products which they really do not desire or need. As computers are machines, they do not possess objectives of their own. There is a golden rule which is used to guide is that the creators of a persuasive technology should never ask to persuade anyone of something they themselves would not consent to be persuaded of (Berdichevsky & Neuenschwander, 1999). It can be briefly said as that designers of persuasive technology should not create any product that persuades someone to do or think something that they, the designers would not want to be persuaded of themselves. Designers are the ones who build, distribute, or adopt the technology and who have the intention to influence someone s attitudes or behavior. It is considered that the responsibility of the designer is to develop ethical and reliable technologies and systems. The following section explains Captology, types and levels of persuasion Captology B.J. Fogg is the first scientist to express the concept of "Captology," a word he devised to illustrate the overlap between persuasion and computers. Fogg defined Captology as design, research, and analysis of interactive computing products created for the purpose of changing people s attitude or behaviors. According to Fogg, persuasion is based on intentions and not on results whereas Captology focuses on endogenous intent built into products not on exogenous intentions (Fogg, 2003). Figure 1: Fogg's Captology- Computing technology and Persuasion Overlap, Adapted from (Fogg, 2003) 7

23 When a technology is deliberately designed with the purpose to persuade and it has planned persuasive effects, the technology inherits endogenous intent. For example, energy conservation game which targets the user with a purpose to change the behavior and thoughts towards energy saving. When someone personally chooses to use a technology with the intention to change his or her own thoughts or actions, the technology is said to inherit exogenous intent. For example, Zune mp3 player is not a persuasive product by design but a person buys the product to play music and considers that listening to music encourages him/her to stay active and happy Types of Persuasion Understanding a persuasive system needs a detailed study of the persuadee, message, channel and context. Persuadee is a human information processor which observes at the individual attributes that can either retard or propel persuasion. Persuader is one who or that which persuades, influences, or leads on. Messages are conveyed through channels, with nonverbal and verbal social interactions. The situation in which the message is delivered is referred as context and it may include the surrounding environment or broader culture. Originally, persuasion has been regarded as a communication process in which a persuader sends a persuasive message to a persuadee or audience (individuals or groups to which the message is delivered). The aim of the persuader is mostly to convince the persuadee with the idea of altering the attitudes or behavior, even though always letting the persuadee with the power of decision and control. Figure 2 : Types of Persuasion, adapted from (Fogg, 2003) 8

24 Three types of persuasion exist: Interpersonal persuasion, human-computer persuasion and computer-mediated persuasion (Fogg, 2003). Interpersonal persuasion happens when two or more people interact with each other, involving e.g. verbal and non-verbal behavior, feedback and coherence of behavior. Human-computer persuasion is the study of how people are persuaded when interacting with computer technology e.g. Apple laptop persuades people by their design. Computer-mediated persuasion means that people are persuading others through computermediated communication, e.g. discussion forums, , or social network systems like Facebook, Google+, Twitter and more. Persuasive systems may employ either human-computer persuasion or computer mediated persuasion Levels of Persuasion The technology can persuade on two levels, macro and micro. Macrosuasion refers to products that are used exclusively for persuasion. The overall target of the product would be to persuade and to have behavioral change. For instance, Figure 3 QuitNet shows good example for Macrosuasion which is developed to motivate users to quit smoking. Figure 3 : QuitNet, an online smoke cessation program (Health, 1995 ) QuitNet is an online smoke cessation program which helps people to quit smoking. It has applications like Quit Gadget that helps to calculate the money and lifetime that users have saved by quitting. Quit Wizard and self-assessment quizzes persuade the user to plan and pick a date to 9

25 stop smoking. QuitNet encourages users by providing certificates for achievement of each quitting milestone and gadget stats. The website displays the list of active users celebrating their quitting anniversaries and also sends greeting cards and s for appreciating smoke quitters (Fogg, 2003). Microsuasion refers to products that include small persuasive elements meant to persuade and they do not have an overall intent. Microsuasion can be just feedback or rating system in an online websites like e-bay, IMDB where they do not have an overall goal to persuade but they incorporate in smaller persuasive amounts to achieve a completely different target (Fogg, 2003). E-Bay is an online shopping website which has a feedback mechanism where both buyers and sellers are encouraged to rate one another at the end of each transaction. It motivates users to be trustworthy and responsible in order to have a good feedback, seller rating and these ratings are valued by the customers. It also encourages sellers by giving reward like ebay top-rated seller badge. IMDB (Internet Movie Database) has a rating scale where the users are influenced to provide their feedback and reviews for a movie, series or television show. To encourage this, it displays overall rating in a star, Meta score, Top 500 features which are valued by the viewers. 2.3 Functional Triad B.J.Fogg proposes the Functional Triad as a classification of three basic ways that people view or respond to computing technologies. The persuasive technology can function as tools, media or social actors based on the functional role of the computing device or system. Figure 4 : Fogg's Functional triad, adapted from (Fogg, 2003) 10

26 Firstly, the computer as a persuasive tool can encourage people by performing some behavior easier or more capable to do, or guiding people through a process or performing calculations or measurements that motivates. Secondly, the computer as a persuasive medium can influence by permitting people to explore cause-and-effect relationships, or affording people with vicarious experiences that motivate or helping people rehearse activities. Thirdly, the computer as a social actor can persuade by rewarding people with positive feedback, or exhibiting a target behavior and providing social support (Fogg, 2003) Credibility and computers It is significant for the designers of persuasive technology to understand the concepts of credibility. Credible products have the ability to change thoughts and behaviors, to motivate and persuade. Fogg refers credibility as believability; Credible people are believable people; credible information is believable information. Basically, researchers describe credibility as a perceived quality made up of multiple dimensions which does not reside in an object, a person, or a piece of information. Therefore, in discussing the credibility of a computer product, one is always discussing the perception of credibility. If the credibility for a product is low, the potential for behavior change is also considerably low. Although studies vary on how many dimensions contribute to credibility evaluations, the vast majority of scholars identify two key components of credibility: Trustworthiness Expertise In evaluating credibility, a person makes an assessment of both trustworthiness and expertise to attain at an overall credibility valuation. Figure 5 : Key dimensions of Credibility, adapted from (Fogg, 2003) 11

27 Trustworthiness is described as system that is observed as truthful, fair and unbiased will have increased powers of persuasion. The three main aspects of trustworthiness are credibility of source, arguing against own interests, familiarity. Website provides information about its products rather than simply marketing information. Expertise is a system that is viewed as incorporating expertise (knowledge, experience, and competence) will have increased powers of persuasion. Website updates information on a regular basis without obsolete data. Four Types of Credibility Fogg s four types of credibility are presumed, reputed, surface and earned. TYPES Presumed credibility Surface credibility Reputed credibility Earned credibility EXPLANATION When user trusts someone or something because of general assumptions in their mind. User makes initial assessments of the system credibility based on a firsthand inspection. Recommendations from credible third-party endorsements, reports, or referrals. Performance consistent with expectations (positive). Table 1 : Types of Credibility Credibility perceptions can strengthen or weaken over time, but once lost, credibility may be hard to regain. (Near) Perfection is a principle that emphasizes computing technology will be more persuasive when it never commits errors or mistakes (or very few and easily explained) (Fogg, 2003). 12

28 2.3.2 Computers as persuasive Tools As tools, persuasive technology increases the ability of users to perform a target behavior by simplifying target. Persuasive principles suggested by Fogg are reduction, tunneling, tailoring, suggestion, self-monitoring, surveillance and conditioning. Reduction means using technology to simplify complex tasks. The aim of this strategy is to make the desired behavior more tempting by making it easier for the user. For instance, some online shops don t make you fill out forms during registration, instead request you to provide your personal number where it automatically fills out your details for signing up new account. Tunneling means guiding the user through a process or experience provides opportunities to persuade. Tunneling makes it easier to go through a process for the users. For designers, it controls what the user experiences - the content, possible pathways and the nature of the activities. Software installation provides a simple example of tunneling technology. It helps you through the process, step by step. Tailoring means tailoring information to the user's specific situation, interests, personality or needs increases the persuasive effect. Tailoring has been acknowledged as one of the most potential persuasive strategies in e-commerce. They discount coupons, offer newsletters to keep customers informed of new products and promotions. Suggestion means intervening at a time when the target behavior is/would be relevant or at opportune moments, which means interacting with the user at the right time and in the right context and place. Application for healthier eating habits advises children to eat fruits instead of chocolate at a snack time. Self-monitoring means systems that allowing people to monitor themselves and learn from information on their actions in order to change their behavior. Simulations can persuade by enabling them to observe immediately the link between the cause and the effect. Heart rate monitor displays user s heart rate and the duration of the exercise. Surveillance technology is based on the overt monitoring of other people s behavior and it increases desire to fulfill perceived expectations. For example, Parental Controls in antivirus software like Norton where it helps to track children s online activities. 13

29 Conditioning means using operant conditioning to have immediate positive reinforcement after target behavior. The levels, game points and rewards provided in online games like angry birds makes the player to continue the game and play more. Example of a persuasive tool Nike + IPod Sports kit Nike ipod Sports Kit is a tool that has attempted to motivate people to exercise more. It uses a design that is compatible with the ipod and running shoes. It implies persuasive strategies reduction, suggestion and self-monitoring to have behavioral change. Figure 6 : Nike Sports Kit, a persuasive tool It keeps track of distance covered in a work-out, time to complete individual goals and other fitness benchmarks. It also provides auditory directions for working out, satisfying the role of a virtual personal trainer (Wai & Mortensen, 2007 ) Computers as a Persuasive Media (Simulation) Computer as a persuasive medium discovers the current and potential use of the simulations. Cause and Effect relationships are a scenario that provides simulation and persuades by enabling them to observe immediately the link between the cause and its effect. For example, before and after pictures of people who have lost weight are shown on a weight losing Website. Virtual Rehearsal is defined as a system providing means with which to rehearse a behavior can enable people to change their attitudes or behavior in the real world. Online shopping provides a 14

30 virtual dressing room where you can customize a model to look alike you, and then "try on" clothes. Virtual reward is a system that provides virtual rewards for users in order to give credit for performing the target behavior. Punch cards provided at Subway, a card is stamped with each qualifying purchase. Once you have the required number of punches, you get a free sandwich. This type of reward program persuades people to eat more in subway. Simulations in real-world contexts provide persuasive information vicariously through life-like settings. Baby think it over is a good example. Example of a persuasive medium - Baby Think It Over Baby Think It over Program (BTIO) designed to educate teens about parenting responsibilities. It is a baby stimulator (hi-tech doll) that looks like a human baby and simulates many infant needs. Participants are required to care the baby stimulator, just as they would care an actual baby. It implies persuasive strategies virtual rehearsal, cause and effect relationships and simulations in real-world. Figure 7 : Baby think it over, a persuasive simulator It helps teenagers to understand how much attention a baby needs. In a study of 100 and more young students who participated in this program, 95% of them were not ready to accept the responsibility of parenting. This shows strongly the persuasive potential of BTIO as a simulated object (Fogg, 2003). 15

31 2.3.4 Computers as a persuasive social actors The third corner in the functional triad which persuade by providing a variety of social cues that elicit social responses from their human users. Attractiveness is a System that is visually attractive for its users is likely to be more persuasive. If appearance of the website or design of a game is appealing, it tends to be more persuasive. Similarity is a System that imitates users in some specific way. A good example of similarity, when a user is allowed to alter the graphical layout of an application or personalize his/her character. Praise leads the user to be more receptive to the giver of praise. A System can use praise through words, images, symbols, or sounds as a way to provide positive response for a user. Reciprocity is defined as returning a favor. When the customer is ready to fill a feedback form/ survey then the system is said to be reciprocative. Authority is a system that refers to people in the role of authority. Website displays the logo of ISO 9001 certification. Example of a persuasive actor Smokey Bear Smokey Bear is an online website and mobile application for the prevention of forest fires. Smokey is a fictional character (mascot) of the United States Forest Service created to educate people about the dangers of wildfires. Figure 8 : Smokey Bear, a persuasive social actor (Ad Council, n.d.) 16

32 The role of the smokey motivates people to know the importance of wildlife conservation. The application provides step by step guide for wildfire prevention, smokey s past history, kids' section features "Bear Facts" about Smokey's species, campfire games and stories. It also displays the map of current wildfires across America (Ad Council, n.d.). 2.4 Applications of Persuasive Systems First, the most significant application domain for persuasive systems is health and wellness. For instance, health care technologies can be developed to promote physical fitness, encourage people towards healthy behavior, perhaps delay or prevent medical problems and early death (Kientz et al., 2010). Some popular persuasive applications for health are: Alcohol 101 Plus: Alcohol 101 plus is an interactive online program which fights against alcohol drinking and drunk driving. It aims to help students make safe and responsible decisions about alcohol on college campuses. It has various interactive programs to persuade students in a virtual campus like virtual bar, DUI (driving under the influence), alcohol and the brain. Virtual Bar, Virtual Brain shows how alcohol can influence body functions, activities and actions whereas DUI creates awareness about the issues, legal work, penalties caused as a result of drunk driving. Figure 9 : Alcohol 101 Plus, online program against alcohol drinking (Council, n.d.) 17

33 Figure 9 shows the interactive program in alcohol101plus. It uses the technology of virtual rehearsal and cause and effect where it provides a motivating simulation environment to rehearse a behavior and shows the consequences of user s actions that motivate people to change their attitudes and behaviors in real world (Council, n.d.). Stay Teen: Stay teen is an online website to prevent teen and unplanned pregnancy. It conducts quiz, polls campaign and also includes videos encouraging teens stay teens and avoid teenage pregnancy. It has featured games and videos which help to evaluate the health of their relationships and know more about birth control measures. Pocket Pikachu: It is a digital pet as a technology that persuades individuals to be physically active on a consistent basis. The user had to walk, run, or jump anything to activate the pedometer. HIV roulette: It is a kiosk that simulates the health threats of having unprotected sexual contact. It provides the user with hypothetical choices about sexual behavior. It uses the cause and effect technology to show how these choices would affect their chances of contracting HIV, the virus that causes AIDS. Another common application area is energy conservation. New design products and systems are also developed to persuade human behavior towards the aim of saving resources efficiently like saving water and reducing the use of electricity through individual or feedback systems. The Powerhouse: It is a computer game designed to persuade teenagers to have an energy aware lifestyle and to promote efficient use of energy in their homes. The game tries to influence players interest in energy-related issues and improve their knowledge on energy-consuming activities in their houses. In the game, player manages a simulated domestic environment with seven characters. Almost all activities in the game need electrical energy such as taking a shower, washing clothes, and watching television. The objective is to direct the characters to perform the appropriate energy-efficient actions. The game engages many persuasive strategies like cause and effect relationships provides instant 18

34 Figure 10 : Powerhouse, a game for energy saving, source (Bång et al., 2006) feedback for the actions performed, virtual rewards like energy and money meter and conditioning provides sound and visual effects to motivate user to play more (Bång et al., 2006). Shower Calendar: It is a persuasive system for reducing consumption of water used for showering. It introduces a little competition to get people to change their behavior. While in shower, it helps to track water use against everyone else you live with (Laschke et al., 2011). Figure 11 : Shower calendar, adapted from (Laschke et al., 2011) 19

35 2.5 Evaluation of Persuasive Systems In order to study user s thoughts and behavior, theories and methods from social psychology have been applied. These theories were modeled for predicting the user acceptance of the information systems rather than providing systematic analysis and design methods to develop software technologies. Fogg s design principles and framework, which offers first and most utilized theories of persuasive technology but these concepts, are too limited for evaluating persuasive systems. Fogg s principles do not explain how the proposed principles can be modified into software requirements and further implemented as actual system features (Oinas- Kukkonen & Harjumaa, 2008). However, there has not been an agreed method for evaluating persuasive systems and it is seen as a very difficult task for several reasons. The main reason concerns the fact that determining the efficacy of persuasive technologies needs a research method to assess whether the persuasive strategies used are functional and effective. A typical research study involves an intervention where variables of interest are measured before and after using the system for target and control groups. The problem is that statistical research approach represents a bottleneck: they are expensive and resource intensive, in terms of the amount of work required, techniques for data collection and the subsequent statistical analysis. Moreover, an additional problem, in terms of system development is that iterative evaluations cannot be done as part of the design process. Therefore, the method for evaluating persuasive systems as described above requires a fully operational prototype and long term deployment. For this reasons, there is a need for quick and inexpensive evaluation method for assessing how successful a technology may be in helping to promote behavior change or the likelihood of longterm adoption (Ragnemalm et al., 2011) Heuristic Evaluation Heuristic evaluation is an informal usability evaluation method in which several evaluators comment on interface design judging based on usability principles termed heuristics. It involves small set of evaluators who analyze whether each dialogue element conforms to the heuristics or not. The main goal of heuristic evaluation is to identify the usability problems in the design so that they can be attended as part of an iterative design process. 20

36 It is considered as an efficient evaluation method as it discovers many usability problems not found by 31 user testing; since it is bounded to specific tasks as user testing is. The evaluation method reveals about twice as many minor problems as major problems in absolute numbers (Nielsen & Mack, 1994). There are two ways to distinguish heuristic evaluation based on their purpose. When it is more intentional on finding problems, it is formative evaluation. This type of evaluation provides design feedback, mostly in the form of a list of problems and recommended solutions. Heuristic evaluation can also be used in a summative manner; it provides an overall assessment of a single interface or a comparison of multiple interfaces, often in the form of numerical data which is statistically analyzed. It also centers on the effectiveness, efficiency, and user satisfaction of a product. This thesis centers on the summative heuristic evaluation of the persuasive potential of software programs. Some researchers use the term heuristic evaluation in a bit different meaning than Nielsen and Molich. For example, they use the term to specify an evaluation that user interface specialists conduct depending on their experience with similar systems without any specific strategies. In this thesis, the term heuristic evaluation stands for an inspection using heuristics relying solely on the evaluators' expertise Usability inspection methods vs. usability testing According to Nielsen Usability is a quality attribute that assesses how easy user interfaces are to use. Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. It focuses on determining a human-made product's competence to meet its planned purpose. Usability testing can be regarded as an exclusive usability method, since it offers direct feedback on how real users use the system. However, real users can be difficult or expensive to recruit in order to test all aspects of all the versions of an evolving design. Nielsen conforms that many usability issues are overseen by user testing, but that user testing also discovers problems that are ignored by inspection. 21

37 Usability testing is in contrast with usability inspection methods where experts use different techniques to evaluate a user interface and does not entail users. Usability inspection is the common name used for a set of methods that are all based on having experts review a user interface or examine usability-related aspects of a user interface. Usability evaluators can be usability specialists or experts who should have a broad background in usability evaluation and human computer interface (HCI) design. An example would be in evaluating a website a person who is an expert in HCI and web designing (Nielsen & Mack, 1994) Why Heuristic Evaluation Heuristic evaluation is a popular discount usability technique that enables a few expert evaluators to examine a technology artifact for finding the usability problems. It is easy to learn, cheap and it does not require advance planning so it is also referred as discount usability method. In addition, it can be used very early or any part of the development process, since it does not require a running version of the system. An important advantage of using this technique is to provide some quick and relatively inexpensive feedback to designers. Feedback can be obtained early in the design process as they can be attended to as part of an iterative design. Drawbacks of using this method, it requires a certain level of knowledge and experience to apply the heuristics effectively. Trained usability experts are sometimes hard to find and can be expensive. It is not a thorough method to detect each and every problem and does not provide a systematic way to fix the problems found (Nielsen & Mack, 1994) Ten Usability Heuristics Rolf Molich and Jakob Nielsen developed a set of heuristics that are most likely used in the field of interface design. In 1994, Nielsen came up with a refined set of ten heuristics for user interface design. They are called "heuristics" because they are more in the nature of rules of thumb than specific usability guidelines. Nielsen's Ten Heuristics 1. Visibility of system status 2. Match between system and the real world 3. User control and freedom 4. Consistency and standards 5. Error prevention 22

38 6. Recognition rather than recall 7. Flexibility and efficiency of use 8. Aesthetic and minimalist design 9. Help users recognize, diagnose, and recover from errors General Procedure Number of Evaluators In general, heuristic evaluation is difficult for a single individual to do because one person will never be able to find all the usability problems in an interface. Nielsen recommends using three to five evaluators since one does not gain that much additional information by using large numbers. The evaluation is performed by having each individual evaluator examine the interface alone. In figure 12, Curve shows the proportion of usability problems in an interface found by heuristic evaluation when various numbers of evaluators are added. The curve represents the average of six case studies of heuristic evaluation. The heuristic Evaluation involves essential three stages: a pre-evaluation training session, actual evaluations and debriefing session (Nielsen, 1992). Figure 12 : Usability problems Vs. No: of Evaluators, Source (Nielsen, 1992) Briefing on the Method, the domain and the scenario During training session, a short lecture given to the evaluators as they knew very little about the topic. 23

39 Actual Evaluations Each evaluator is asked to evaluate the interface within a given period of time. Evaluators are expected to find as many usability problems in the interface as possible, including both major and minor. The evaluators were also asked to state what established usability principle was violated by each usability problem and to classify the problem as major or minor. Later, all evaluations completed by the evaluator are allowed to communicate and their results are collected. This assures independent and unbiased outcomes from each evaluator. Debriefing session Debriefing session (If needed) is conducted after the last evaluation session in order to deliver some design advice. The participants in the debriefing should contain the evaluators, and representatives of the design team. A debriefing is a good chance for discussing the positive aspects of the design, since heuristic evaluation does not otherwise address this important issue. Severity rating phase Severity ratings can be used to assign the important resources to resolve the most serious problems and can also provide a rough estimation of the need for additional usability efforts. During a heuristic evaluation session, it is tough to get good severity estimates from the evaluators as they are more focused on finding new usability problems. It is recommended that severity ratings can be collected by sending a questionnaire to the evaluators after the actual evaluation sessions, listing the complete set of usability problems that have been discovered, and asking them to rate the severity of each problem. Nielsen also indicates that severity ratings from a single evaluator are too unreliable. As more evaluators are asked to judge the severity of usability problems, the quality of the mean severity rating increases rapidly, and using the mean of a set of ratings from three evaluators is satisfactory for many practical purposes (Nielsen & Mack, 1994). The following 0 to 4 rating scale can be used to rate the severity of usability problems: 0 = I don't agree that this is a usability problem at all 1 = Cosmetic problem only: need not be fixed unless extra time is available on project 2 = Minor usability problem: fixing this should be given low priority 3 = Major usability problem: important to fix, so should be given high priority 4 = Usability catastrophe: imperative to fix this before product can be released 24

40 3 Test Objects Tools MS word, MS power point, computer games The Sims, Counter-Strike, educational programs STONE and Plan Eat Smile are the test objects evaluated in this project. The hypothesis was that when presented to a summative heuristic evaluation, tools are expected to rate low on a persuasive heuristics scale. Due to the reason, tools do not have planned intentions to persuade and cause behavioral change. MS Word and MS PowerPoint are tools that perform predefined set of functions like creating and editing files. Thus a computer tool which is aimed at performing some text manipulation functions is assumed to exhibit very low persuasive potential. According to Fogg, computer games have natural persuasive power and they employ different types of persuasive strategies to have a high quality gaming experience. Games are developed to be persuasive but it is hard to predict whether they have persuasive potential to motivate people. If games have persuasive potential to cause behavioral change, they should rate high when evaluated against Fogg s principles. Educational programs are expected to rate high on a persuasive heuristics scale because learning is reflected by change in observed behavior. Thus, an application intended for educational purposes must have persuasive potential to reinforce learning; otherwise no change of behavior would result. The method used should be capable of distinguishing and providing insights about these types of software based on their persuasiveness. 3.1 Tools Microsoft Word Microsoft Office Word (MS Word) is a word processor developed by Microsoft Corporation. Word is regarded as one of the most widely used product of Microsoft s office suite. Word includes a built-in spell checker, a thesaurus, a dictionary, and utilities for manipulating and editing text like transferring, copying and pasting texts. 25

41 Figure 13 : Screenshot: Microsoft Word MS Word being an extensive tool with several features so, it was decided to assess limited features as it is unnecessary to examine all functions. Certain scenarios were mainly focused for evaluation such as Spell checking, importing files in other formats, Ribbon customization, preview and zoom, mail-merge and macros facility Microsoft PowerPoint Microsoft PowerPoint (MS PowerPoint) is a presentation software program that is also a part of the Microsoft Office package. PowerPoint uses a graphical approach to do presentations in the form of slide shows that accompany the oral delivery of the topic. This program is an effective tool when used for training purposes and widely used in business and classrooms. Figure 14 : Screenshot: PowerPoint presentation 26

42 MS PowerPoint has various functions and features so, selected features and scenario was motivated for evaluation such as designing the slides, adding animations, and slideshow. 3.2 Computer Games Counter-Strike Counter-Strike 1.6 is a tactical first-person shooter video game. Players can join either the terrorist team, the counter-terrorist team, or become spectators. The scoreboard displays team scores in addition to statistics for each player: name, kills, and deaths. The scoreboard also specifies whether a player is dead, carrying the bomb, or is the VIP (very important person), although information about players on the opposing team is hidden from a player until death, as this information can be significant. Killed players become "spectators" for the duration of the round. While there are several maps, the missions are limited to kill/escort a VIP to a helipad, prevent/cause destruction of a bombsite, and prevent/rescue hostages. The game wins by killing everyone in the opposite team or ends with the death of everyone on a given team. Each team tries to complete their mission target and/or eliminate the opposing team. Figure 15 : Screen shot: Counter-Strike game 27

43 3.2.2 The Sims The Sims 3 is a strategic life-simulation video game by Electronic Arts. It is a simulation of the daily activities of one or more virtual persons called "Sims" in a suburban household near SimCity. There can be life stages in The Sims: infant, child, Teenage, adult and old person. The objective of the game is to keep the Sims happy by watching the motive bars to see what they want/need, Control the Sims social life, when they eat and sleep, and what type of job they have. The player can design, build and decorate their homes, build up their skills to help them advance in their job, create a family of Sims to live together, and watch them interact, throw parties, get married, have babies, make friends. Figure 16 : Screenshot: Sims Game 3.3 Educational Programs STONE STONE (Simple Tool for Online Education) has been developed at the Department of Computer Science, Linköping University, Sweden. It is used to teach students unfamiliar with local 28

44 computer environment by providing introductory courses such as handling of common printers. Later after reading, it examines students' knowledge by conducting online tests. Figure 17 : Screenshot: Stone The test has the outline of the course and user needs to keep track of the progress. Each course in STONE consists of a number of steps. In the figure 17, each cell shows a step in the course UNIX. The steps that are green are completed and gray steps, it means they are not accessible because the prerequisites are not met yet. If there are blue steps, it shows they are active and in line to be studied Plan Eat Smile Plan Eat Smile is a meal planning application which mainly attempts to support its users to change their food consumption behaviors by influencing their perceived behavioral control. It educates people to consume healthy food by providing recipes, plan and to shop online from food stores. 29

45 Figure 18 : Screenshot: Plan Eat Smile It comprises of a daily planner designed to help users schedule and freeze an entire month of meals for a family. Users can choose from various meal recipes with different nutritional requirements including vegetarian, Meat and seafood. It helps to make grocery shopping easier and helps to create a free, customizable, online shopping list. Meal planning database helps to find a recipe for the meal. Once you have found a recipe you like, the necessary ingredients can be easily added to your shopping list. You can also order food from your nearby restaurant. 30

46 4 Test Conduction 4.1 Test Procedure Researchers have recommended the use of heuristic evaluation to inspect persuasive technologies. In this thesis work, a specialized heuristics were developed to detect lack of persuasive elements in the persuasive system that involves intentions, adoption, or prolonged effectiveness of the technologies. The main assumption is that when summative heuristic evaluation is done, the software programs may rate high on a heuristics rating scale. The unit of evaluation may be assumed to be different when evaluating persuasive technology and usability. To examine the summative heuristic evaluation as a method for analyzing the persuasive potential, six software programs are evaluated. Three evaluators are presented within the interface design, and they were asked to comment on the effectiveness of the product. During the process, it was initiated with a brief introduction about the heuristic evaluation method and set of heuristics developed for the evaluation. They were asked to analyze the programs separately and one program at a time, taking as much time as needed. The evaluation of each program was performed in alternative days mainly to minimize the influence and effects between the software programs. Each evaluator reviewed the interface individually and they provided their feedback to the heuristics with a rating and comment. The program evaluation followed the same order like MS Word, MS PowerPoint, Counter-Strike, The Sims, Stone and Plan Eat Smile. This whole process took two weeks to complete Operationalization of the principles into heuristics The design principles for persuasive systems defined by Fogg are formulated as a set of heuristics and revising them to be applicable for evaluating the software programs. These principles are divided into four sections comprising system credibility, computer as a persuasive tool, computer as a persuasive medium, and computer as a persuasive social actor. 31

47 In total, 30 heuristics were derived from the 21 principles. Appendix A shows the persuasive principles and the corresponding heuristics. Reason for developing and validating a set of heuristics designed is to be more appropriate for evaluating technologies aimed to persuade users. 4.2 Test Participants As Nielsen recommends using three evaluators, this thesis work involves three evaluators including the author to examine the software programs. The evaluation method is easy to apply so that it has possibility to have regular developers serve as evaluators, though better results are normally achieved when using usability specialists. An expert, who has knowledge in both principles and the domain make for better, more focused heuristic evaluations. Evaluators used in this project were students in computer and cognitive science whose experience of heuristic evaluation of usability was sufficient to this software evaluation Evaluator A Evaluator A is a male, 23 years old, master s student in computer science. He had only a brief introduction to the persuasive principles so the familiarity to the principles is low. Table 2 briefs the time spent by evaluator A on different programs. Programs Time Spent Level of Familiarity (Hours) Evaluation (Hours) familiarity Word >150 2 Advanced PowerPoint 60 3 Intermediate Counter-Strike >120 2 Advanced The Sims 50 3 Beginner Stone 20 4 Beginner Plan Eat Smile 12 4 Beginner Table 2 : Evaluator A 32

48 4.2.2 Evaluator B Evaluator B is a male, 26 years old Graduate Student in Cognitive Science. He had only a brief introduction to the persuasive principles so the familiarity to the principles is low. Table 3 briefs the time spent by evaluator B on different programs. Programs Time Spent Level of Familiarity (Hours) Evaluation (Hours) familiarity Word >130 2 Advanced PowerPoint 40 3 Beginner Counter-Strike > Advanced The Sims 40 3 Beginner Stone 70 3 Intermediate Plan Eat Smile Beginner Table 3 : Evaluator B Evaluator C Evaluator C is a female, 23 years old, master s student in computer systems. She is the original constructor of the heuristics so the familiarity to the persuasive principles is high. Table 4 briefs the time spent by evaluator C on different programs. 33

49 Programs Time Spent Level of Familiarity Evaluation familiarity Word >150 2 Advanced PowerPoint >120 2 Advanced Counter-Strike 30 3 Beginner The Sims 80 3 Intermediate Stone >110 2 Advanced Plan Eat Smile Beginner Table 4 : Evaluator C 34

50 5 Test Analysis & Discussion 5.1 Test Observations The three evaluators analyzed the programs based on whether the heuristic is satisfied or not. They were asked to provide feedback by selecting one response option in the five-point rating scale. The rating scale was displayed vertically with options from 1 to 5. SYSTEM CREDIBILITY # PRINCIPLES FEEDBACK COMMENTS 1.1 Trustworthiness The system that is observed as truthful, fair and unbiased will have increased powers of persuasion.(three aspects: credibility of source, arguing against own interests, familiar) 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? 1.2 Expertise The system that is viewed as incorporating expertise (knowledge, experience, and competence) will have increased powers of persuasion. 1. Does the program show competency (skill/expertise)? 1.3 Surface credibility People make initial assessments of the system creditability based on a firsthand inspection. Does the program look like it can do what it is supposed to do? 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? 1 Extremely satisfies the heuristic, 2 Very satisfied, 3 Moderately satisfies, 4 Slightly satisfies, 5 Does not satisfy 35

51 1.4 Earned credibility Performance consistent with expectations (positive) 1. Is the program behavior consistent and predictable? 1.5 (Near) perfection No errors or mistakes (or very few and easily explained) 1. Does the program make any errors? Table 5 : Heuristic Evaluation Form Table 5 shows the layout of the heuristic evaluation form. Apart from the heuristics and feedback options, it also has a column for writing comments to explain the reason for choosing a particular option. After the evaluators had completed their evaluations, the results were summarized. Appendix B shows the evaluation form done by the evaluators for the program STONE. In order to analyze the feedback provided, scores were distributed for each feedback as shown in the table 6. Each response option in the rating scale carries a score from 5 to 1. Rating scale Score Extremely satisfies the heuristic 5 Very satisfied 4 Moderately satisfies 3 Slightly satisfies 2 Does not satisfy 1 Table 6 : Five-Point Rating Scale 36

52 Scores were assigned based on the feedback provided by the evaluator. For example if an evaluator chooses the option moderately satisfies for a heuristic, it carries a score of 3. Also, if there are two heuristics for the same persuasive principle, average value was calculated. 5.2 Test Results The scores were analyzed for each heuristic, averaged for each persuasive principle. The scores were analyzed both individually for each evaluator and averaged over all three evaluators and also averaged for each group of principles. The results and calculations are clearly shown in appendix C and D. From appendix D, we can calculate mean for each group of principles for the six computer programs. It is clearly shown in the table 7. MS- Word MS- PowerPoint Counter- Strike Sims Stone Plan Eat Smile System Credibility Computers as a tool Computer as a medium Computer as a social Actor Total Mean Table 7 : Overall Mean Calculation Based on the calculations, we can compute the heuristics which are perceived as highly satisfied and unsatisfied by the evaluators. The programs were analyzed based on the scores and comments received from the evaluators. The following section elaborates the test results in detail. 37

53 Trustworthin Expertise Surface Earned (near) Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Virtual Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale MS-WORD: The first Program, MS-word is one of the most widely used programs by the evaluators. The summative mean value for the program is The evaluators provided high scores for some principles expertise, surface credibility, suggestion, attractiveness which will be discussed below. MS-Word has various tools to perform complex tasks. For instance, word provides various templates for letter formats, agendas, resume. It offers user-friendly tools like design, layout and format to edit a chart, picture and tables and it helps to send documents directly from word using the mail merge tool. It has visually pleasing design with a neat arrangement of icons and symbols representing the function of the tool. It also has a Spell checker tool to detect the misspelled or unknown words which are underlined with the red zigzag. Auto Correct feature can be used to automatically correct your misspelling and also suggests words from the dictionary that can be used instead. Font tool provides a realtime preview before any function is applied such as Font faces and sizes. Also it provides a list of recently used fonts so it really makes user s tasks easy and simple. These program tools are significant skills that justify the expertise, surface credibility and suggestion principles MS WORD MS Word Heuristics Figure 19 : Mean: MS Word The principles earned credibility, Near Perfection, Reduction, Tunneling, cause and effect, virtual Rehearsal and Authority are moderately satisfied. The heuristic corresponding to the principle Earned Credibility Is the program behavior consistent and predictable? which was expected to rate high. It was moderately satisfied heuristic due to the reason that the program s 38

54 Trustworthiness Expertise Surface Earned (near) Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Virtual Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale behavior is inconsistent for example, when a document is transferred from a computer to another, the page layout or format of the document changes. From the table 7, group of principles scores for the programs are 3.7, 3.1, 2.9, and 2.9. It is identified that the evaluators are more satisfied with the system credibility and computer as a tool principles than the other two principles. MS-POWERPOINT: The second program, MS-PowerPoint has a feedback similar to MS Word due to the fact that they have most identical features such as design and basic tools like spell checker. MS- PowerPoint is a presentation program which has some exclusive features like design, animation, transitions and slideshow. The evaluators provided high scores for the principles expertise, surface credibility, reduction, tunneling, suggestion, attractiveness compared to other heuristics MS POWERPOINT MS PowerPoint Heuristics Figure 20 : Mean: MS PowerPoint They show expertise using the tools like animation and transitions which really support users to add effects to the slides created. Also, it offers tools to add video or music to a slide, record a slideshow and more. The tools like layout, quick styles which helps to reduce the task of the user by providing different formats and styles for creating a slide. The heuristic corresponding to the principle of tunneling Does the program steer users through a sequence of steps in order to complete a task Evaluators refer that the tunneling heuristic as 39

55 high satisfied due to the fact that PowerPoint program provides a set of available themes and templates. These templates assist users with sample formats for creating plans and proposals, PowerPoint slides and presentations or agenda and it also has various categories to choose according to their purpose. The principles which are moderately satisfied are Earned credibility, Near Perfection, cause and effect, virtual Rehearsal and Authority. The heuristic corresponding to the principle of cause and effect Does the program show any effects or consequences of user s actions? Evaluators consider that it satisfies the cause and effect heuristic. Using Animation tools, the user can see the real-time effects of the options like appear, fade, wipe, float without applying the option on a particular slide so it highly saves the time of the user in trying the various available options. PowerPoint has low scores for the conditioning, surveillance, praise and virtual reward principles. The heuristic corresponding to the principle of conditioning Is positive feedback provided immediately when a good or desired action is taken or good result achieved? and it is identified that the program totally lacks the concept of praise, reward or conditioning same like MS-Word. They do not display any cheering messages and there is no positive response provided by the program when the user achieves his/her target. From the table 7, the groups of principles scores for the program are 3.7, 3.1, 2.9, and 2.9. It is identified that the evaluators are more satisfied with the system credibility and computer as a tool principles than the other two principles. COUNTER-STRIKE: The summative mean value for the third program, Counter-Strike is It is a popular online game and has added features in the latest version like new maps, characters, and weapons, as well as provides updated versions of the classic Counter-Strike content. Counter-Strike has highly satisfied principles earned credibility, expertise, near perfection, surveillance, conditioning, virtual rehearsal, rewards and praise. The heuristic corresponding to the principle virtual rehearsal Does the program allow the user to repeat actions and activities? and the evaluators has offered high scores for this heuristic and have commented that the player is allowed to play the same maps possibly infinite number of times. The program satisfies the principle, Virtual Rewards as it offers the player with funds to buy weapons/ammunition if they succeed a mission and also the player moves to the next level. The heuristic corresponding to the principle conditioning Is positive feedback provided immediately when a good/desired action is taken or good result achieved? It is also perceived that the game 40

56 Trustworthin Expertise Surface Earned (near) Reduction Tunneling Tailoring Suggestion Self- Surveillance Conditioning Cause and Virtual Virtual Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale highly fulfills the conditioning principles as it cheers the players with messages like counter terrorists win, target successfully bombed when the player does his/her part well COUNTER STRIKE Counter Strike Heuristics Figure 21 : Mean: Counter-Strike Counter-Strike has received low scores for principles trustworthiness, suggestion, simulation, attractiveness, tailoring, reciprocity and authority. The heuristic corresponding to the principle simulation Is the similarity to the real world high and evaluators think that Counter-Strike has an old design which is not much attractive when compared to new games with advanced graphics. It has some notable flaws in the design such as a player can walk through the wall sometimes or move across other player s head. Counter-Strike does not satisfy the principle Authority. Evaluators do not really know whether the game is approved by any authority or not. So, the Authority heuristic is perceived to be least fulfilled. From the table 7, the groups of principles scores for the program are 4.1, 3.8, 4 and 2.6. The system credibility and computer as a medium provides high scores compared to other two principle groups. Comparing the overall feedback provided by the evaluators A, B and C, you can recognize that Evaluators are pleased with the game. 41

57 Trustworthine Expertise Surface Earned (near) Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Virtual Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale SIMS: The summative mean value for the fourth program, The Sims is 4.15 which hold the highest mean rating among all the programs. The program has extremely satisfied the principles similarity, praise, virtual rewards, conditioning, cause and effect and expertise. The game has a really pleasing design and environment, attractive design and appears trustworthy as it provides source information when the game starts. It offers virtual rewards and praises by winning purchase rewards and funds to buy new assets or things. The game funds each player with $20,000 when the game starts. The sims can also earn promotions by achieving the skill and friendship requirements of each level, which also increases the daily wages. For instance, in Sims 3, achieving a Sim's wish contributes to the Sim's Lifetime Happiness score and mood SIMS Sims Heuristics Figure 22 : Mean: The Sims The heuristic corresponding to the principle cause and effect Does the program show any effects or consequences of user s actions? and the game visualizes most of the actions done by the player. The game has a scoring meter in the Sim panel where it displays if any activity is successfully completed. Hunger, energy, bladder, social, fun and hygiene are the scoring meters which has to be maintained to lead a healthy life. So, when a sim takes shower or eats food and then the hygiene meter displays a complete green meter. If not, it shows a red meter and displays messages for the player to remind the activities. The heuristic corresponding to the principle surveillance Is it possible to view another user s results? and the players can view the scores of the Sims in the 42

58 same game. Most of the principles are perceived to be highly fulfilled except the principles surface credibility, reduction, surveillance and authority. The heuristic corresponding to the principle simulation Is the similarity to the real world high and do things behave in a plausible way? the muscle and body weight sliders introduced has created realistic looking Sims and the player in the game can control their own Sims in activities and relationships in a similar manner to real life. Also the environment and design is identical to real world. The heuristic corresponding to the principle reduction Does the program make tasks simpler? The game has various levels with complex things that are tough to find and the steps that are difficult to follow. The heuristic corresponding to the principle tunneling is Does the program steer users through a sequence of steps in order to complete a task? it is observed that the game provides a set of tips but they do not provide any procedure to play the game. For instance, when the player has a set of activities such as paying their bills, finding a job, working out, and throwing a party but it is difficult to prioritize the activities and manage the budgets. From the table 7, the group of principle scores for the program is 4.0, 4.1, 4.5 and 4.0. It has the highest scores for the principles computer as a medium and computer as a social actor compared to the other two principles. STONE: The summative mean value for the fifth program, Stone is 3.7. Stone is a simple and intuitive educational program. The layout design is simple and the language used is comprehensible. The program does not cause any errors. It has a good structure so that everyone can work on the program with ease. Most of the principles trustworthiness, expertise, earned credibility, near perfection, reduction, tunneling and conditioning are perceived to be highly fulfilled. The heuristic corresponding to the principle Conditioning Is positive feedback provided immediately when a good or desired action is taken or good result achieved? and the evaluators has commented that the program validate the tests by displaying in result column as correct in green color letters. The virtual rewards and praise has average scores as they do not display any cheering messages to the users and the only reward it offers is when the result is pass, you are allowed to take the next test which gets unblocked automatically. The heuristic corresponding to the principle Selfmonitoring Does the program store and provides assessment of user history and results? and the program saves the users history and results. For instance, it displays in the result that These 43

59 Trustworthiness Expertise Surface Earned (near) Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Virtual Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale are archived results from an old test at den 12 September :20:46. The test was completed by <user-id> STONE Stone Heuristics Figure 23 : Mean: Stone The heuristic corresponding to the principle Similarity Is it possible to personalize your character or design environment? which has a low score from all the three evaluators as it is not possible to personalize your character. The heuristic corresponding to the principle tailoring Is the program tailored to the specific user or situation and is it possible to configure the program to the needs of a specific user? and the evaluators believe that this program is specially designed for students in the university as introductory tests. So they consider stone to satisfy the first heuristic but it does not fulfill the second heuristic as it lacks the ability to configure to a specific users. The heuristic corresponding to the principle Virtual rehearsal Does the program allow the user to repeat actions and activities? and as the program being a test tool, it does not allow the students to take the test several times and it is restricted to only three chances to pass the test. If not, the student access to the test is blocked and he or she needs to mail the teacher to unblock the test again. It has the lowest scores for the group of principle computer as a social actor compared to the other three group of principles. From the table 7, the group of principles scores for the program is 4.7, 3.7, 3.6 and

60 Trustworthiness Expertise Surface Earned credibility (near) perfection Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale PLAN EAT SMILE: The summative mean value for the last program, Plan Eat Smile is 3.4. The Plan Eat Smile website has a very attractive design with a calendar, categories of food, feedback and appealing icons to refer vegetables, fish and meat recipes. The principles Expertise, earned credibility, near perfection, reduction, tunneling, self-monitoring, conditioning, virtual rehearsal, attractiveness are perceived to be satisfied. The program has tools to plan a meal from the recipes provided in the website. If users do not feel like cooking, it provides access to order the same food in a local restaurant. The users can add their recipes, create and print shopping lists. It also has an informative tool called My feedback where the user can see the meter measuring the content of the food with three categories, Try New, fish and animal meat, vegetarian so, it benefits the user to follow a balanced diet PLANEATSMILE PlanEatSmile Heuristics Figure 24 : Mean: Plan Eat Smile The heuristic corresponding to the principle Trustworthiness Does it provide credible Information about the program and its source? and the program provides information about the source in the footnotes of the webpage. The heuristic corresponding to the principle Reduction Does the program make tasks simpler? the program simplifies the process of planning meals, providing different recipes, creating a shopping list and many more. Plan Eat Smile helps the users by providing different list of recipes for four types of food like fish, chicken, animal meat and vegetarian to choose a meal and 45

61 provides them with a good recipe to cook. It also supports to print the shopping list when you finish adding the ingredients, it sends to the nearest store to have the items collected. These features highly fulfill the principle tunneling. The heuristic corresponding to the principle Suggestion Does the program provide any suggestions for tasks and are they provided in a timely manner? the program assists when a user is allergic to an ingredient like garlic, it suggests the user to select an another recipe without garlic. The principles which have low scores are Surveillance, Praise, Virtual rewards, Similarity as the game does not offer any rewards or praises to motivate the user. It is also significant that the program does not allow viewing other user s accounts and activities. From the table 7, the group of principles scores for the program is 4.1, 3.4, 3.6 and 2.8. It has the lowest scores for the group of principle computer as a social actor compared to the other three group of principles. 46

62 5.3 Evaluators Agreement A comparison of the results of the three evaluators was also performed. Given the 5-point scale used to rate the compliance with heuristics, exact agreement is expected to be very rare. In MS- Word and PowerPoint, the evaluators agreed exactly on 11 heuristics (52 %) out of 21, and their finding varies at least one step on 10 heuristics (48%). In Counter-Strike, the evaluators agreed on 16 heuristics (76%) out of 21, finding varies at most 2 steps on 5 heuristics (24%). In Sims, the evaluators were satisfied on 19 heuristics (90%) out of 21, and finding varies at most 1 step on 2 heuristics (10%). In stone, the evaluators agreed exactly on 13 heuristics (61%) out of 21, and finding varies at most 2 steps on 8 heuristics (39%). In Plan Eat Smile, the evaluators agreed exactly on 14 heuristics (66%) out of 21, and finding varies at most 2 steps on 7 heuristics (34%). The result suggests that the evaluators understand and use the heuristics consistently. In this following section, difference of agreement between evaluators will be discussed for each program. MS WORD Evaluator A has offered low scores for the surface credibility, tailoring, reduction, and reciprocity principles. He considers that the program does what it meant to do as a tool but does not provide any additional features to provide easy effort for complex tasks. It also fails to provide tools to personalize the program to the needs of a specific user. For Instance, if you need to create an article, the program does not provide an article template but you just need to download online or you need to design one by yourself. For the principle virtual Rehearsal, Evaluator A considers the Macros tool which allows user to record set of actions performed on document, and to repeat them in future but he finds hard to use the macros tool. On the other hand, Evaluator B and C observes that MS-Word allows user to repeat actions performed several times using undo and redo options which fulfills the virtual rehearsal principle. Evaluator A finds that the program is unappealing with more icons or tools on the same ribbon. The evaluator does not wish to return a favor to the program so the principle reciprocity does not 47

63 Trustworthiness Expertise Surface credibility Earned credibility (near) perfection Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale satisfy. It is also significant that the evaluator A has less duration of usage of the programs whereas the other two evaluators are more familiar with the program MS WORD EVALUATOR A EVALUATOR B EVALUATOR C Heuristics Figure 25 : Evaluator Agreement: MS word According to the evaluators B and C, Most of the heuristics has met the satisfied criteria. Evaluator B and C are contented with most of the features in Word like language settings, page layout, and design tools based on which they gave high scores. Also, B and C wish to return a favor as they find the program useful. The evaluator C has offered low scores for only one heuristic which was based on the principle of earned credibility, indicating that the program behavior to be unpredictable. The heuristic corresponding to Near Perfection Does the program make any errors? Evaluator C provided low scores compared to the evaluators A and B due to the reason that the program crashes sometimes causing loss of data whereas evaluator A overlooked the failure concept as it has a new feature to recover files by automatic saving. MS-Word has very low scores for the conditioning, surveillance, praise and virtual reward principles. The heuristic corresponding to the principle of praise Does the program display any messages to cheer the user or does the program offer any praise when the user finishes a task and it is known that the program lack the concept of praise, reward or conditioning. They do not display any cheering messages and there is no positive response provided by the program when the user achieves his/her target. Comparing the feedback provided by the evaluators A, B and C, you can identify that Evaluator A points out that most of the heuristics is unfulfilled. It has caused in low scores for the 48

64 Trustworthiness Expertise Surface Earned credibility (near) perfection Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale majority of heuristics compared to the other evaluators. From appendix D, you can observe the mean values for the evaluators A, B and C is 2.6, 3.2 and 3.6. MS POWERPOINT The heuristic corresponding to the principle of tailoring Is it possible to configure the program to the needs of a specific user? Evaluator C has provided with high scores as it is possible to personalize the program to a specific user. For example, if a user chooses a presentation template where he/she is directed to different categories like academic, business, medical and health care presentations. So it satisfies the needs of the particular user MS POWERPOINT Evaluator A Evaluator B Evaluator C Heuristics Figure 26 : Evaluator Agreement: MS PowerPoint Evaluator A has mentioned that the heuristic is slightly satisfied due to the reason it is not designed especially for a specific user even though evaluators B and C feel that it partially satisfies the heuristic. The heuristic corresponding to the principle of Virtual rehearsal Does the program allow the user to repeat actions and activities Evaluator C observes that it has significant tools to repeat actions like slideshow where you can have a screening of the slides infinite times and design tool where you can change the themes several times and have a preview when you choose a particular theme. Whereas evaluator A comments that the program has only limited number of themes or 49

65 Trustworthiness Expertise Surface Earned credibility (near) perfection Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale animations. From appendix D, you can observe the mean values for the evaluators A, B and C is 2.6, 3.2 and 3.6. COUNTER-STRIKE The heuristic corresponding to the principle suggestion Does the program provide any suggestions for tasks and are they provided in a timely manner and Does the program provide HELP option to assist the user? and Evaluator C has given low scores for the heuristic corresponding to the principle suggestion as she considers that the game does not provide any guide/help for the players. Evaluators A and B are contented that the program provides timely suggestions like helping the player to find a bomb or how to diffuse a bomb COUNTER-STRIKE Evaluator A Evaluator B Evaluator C Heuristics Figure 27 : Evaluator Agreement: Counter-Strike Evaluator A has specified that game satisfies the principle, Near Perfection as there are no errors in the game whereas evaluator B and C considers that the game crashes sometimes. Regarding surveillance principle, Evaluator A considers that it has a camera mode where the player can view the game and watch other player s actions and activities. Evaluator B has a different view stating the possibility to view the other player results and he has mentioned that the game points can be seen when you are playing the same game. The heuristic corresponding to the principle similarity Is it possible to personalize your character or design environment and Does the program relate the user in some meaningful way? and Evaluator A has given low scores mentioning that similarity principle is not really satisfied 50

66 Trustworthiness Expertise Surface Earned (near) perfection Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale because the characteristics of the players are fixed and you can only name the characters. Counter-Strike has a mean rating of 3.6, 3.6 and 3.7 from the evaluators. SIMS This game has fulfilled 83% of the principles with slight differences between the three evaluators. The heuristic corresponding to the principle surface credibility Is the language used logical, understandable and familiar to the users? and Evaluator B feels that the language Sims speaks is strange and incomprehensible. Evaluator C has mentioned that she enjoys hearing what they speak so does not mind that the language used should be logical SIMS Evaluator A Evaluator B Evaluator C Heuristics Figure 28 : Evaluator Agreement: The Sims The evaluators B has offered high scores for the tailoring principle as it is possible to personalize your Sim as a baby, toddler, child, teen, young adult, adult, and elder. Also, you are able to configure the design of the game by building your own house and choosing where to work etc. Evaluator C thinks that the game is not tailored for any type of users and considers that the principle is partially satisfied. Even though it has a small difference, the mean values calculated for the Evaluators A, B and C are 4, 4.2 and

67 Trustworthin Expertise Surface Earned (near) Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Virtual Virtual Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale STONE The heuristic corresponding to the principle surface credibility Is the program visually attractive? and the evaluator A has provided low scores due to the reason that Stone has an old classic design which makes it less attractive. Evaluator C observes that the language used is not logical in certain parts of the text. As you are allowed to access the program in two languages, English and Swedish and when you select English in order to take the test but you can still see few Swedish in the text. So it has received low scores from A and C. Evaluator B is perceived to be highly contented with the appearance of the program as he considers the program to be simple and clear STONE Evaluator A Evaluator B Evaluator C Heuristics Figure 29 : Evaluator Agreement: Stone As the design is old and very simple, Evaluator A has given very low scores for the heuristic corresponding to attractiveness principle whereas B and C are satisfied with simple design. The heuristic corresponding to the principle surveillance Is it possible to view another user s results? and it is familiar to the evaluators that the test results are not shared and cannot be seen by other users. But the evaluator C thinks that it is fine as test results are visible only to the student who takes the test. The mean values calculated for the Evaluators A, B and C are 3.2, 4 and

68 Trustworthiness Expertise Surface Earned (near) Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Cause and Effect Virtual Virtual Rewards Simulation Attractiveness Similarity Praise Reciprocity Authority Rating Scale PLAN EAT SMILE The heuristic corresponding to the principle Similarity Is it possible to personalize your character or design environment and Does the program relate the user in some meaningful way? and evaluator A mentions that user can have his/her own account and can login with a username and password. All the information about the user is stored including the recipes and shopping lists PLANEATSMILE Evaluator A Evaluator B Evaluator C Heuristics Figure 30 : Evaluator Agreement: Plan Eat Smile The program has very low score for the surveillance principle as the user cannot view another user s activity. Evaluator C considers the point that the recipes added by a user can be viewed by another user. The program does not satisfy the praise or reward principles. As the program does not display any cheering messages to the user and does not provide any rewards like offers. It has good score for heuristic corresponding to the conditioning principle as it gives a feedback meter showing green when the user takes a meal with a balanced diet and shows red when the meal does not satisfy the diet. The mean values calculated for the Evaluators A, B and C are 2.6, 2.1 and

69 5.4 Comparative Study As a result of evaluations, we can clearly see that The Sims game has the higher mean score of 4.15 compared to other programs. The MS- Word and MS-PowerPoint has the lowest mean score of Stone, Counter-Strike and Plan Eat Smile have the mean scores of 3.7, 3.6, and 3.5 respectively. The low mean scores indicate low persuasive potential of the program and the high mean scores indicate high persuasive potential of the program. Mean for group of principles System Credibility Computers as a tool Computer as a medium Computer as a social Actor MS- Word MS-PowerPoint Counter Strike Sims Stone PlanEatSmile Figure 31 : Mean for group of principles It is also reflected that MS-Word and MS-PowerPoint has less persuasive elements that a simulator or social actor can use. Either the programs, Word and PowerPoint, did not use the principle of reward, praise or conditioning is an interesting observation. It is expected that renowned game like Counter-Strike to engage persuasive features attractiveness, expertise, virtual rewards and similarity that a simulator can use. Even though, Counter-Strike offers several praises and rewards like weapons and money, but the evaluators reveal that the praises are the repetitive messages (for instance terrorists win ) which appear in 54

70 the game every time when the player achieves the target. Counter-Strike engages less persuasive features that a social actor can use. Sims has the highest mean score of 4.1, 4.5, 4.0 for the group of principles Computer as tools, Computers as medium, Computers as social actors. As a popular game, it is perceived to have more persuasive features by fulfilling 85% of the total heuristics. It indicates that the game engages more persuasive strategies as a tool, medium and social actor. It is interesting that Sims employs an increased number of persuasive features that a social actor can use. Generally, a game is expected to act more as a simulator but Sims fulfills most principles in a prominent manner. It is observed from the graph above that Stone has the highest mean score of 4.7 for the System credibility group of principles. It can be agreed that it highly satisfies 94% of the heuristics corresponding to trustworthiness, Expertise, surface credibility, earned credibility and near perfection principles. As an online education tool, it has less persuasive elements attractiveness, praise, reward and similarity that a social actor can use. Plan Eat Smile has more persuasive features that simulator can use. It lacks the Persuasive features praise and rewards. It was also observed in the previous research that educational programs did not use the principle of reward (Ragnemalm et al., 2011). From the overall evaluation of the programs, the fact that evaluators misunderstood few heuristics is significant but not surprising. One reason for this might be that the evaluators misinterpreting the heuristics provided. The heuristic corresponding to the principle authority was mostly misjudged and ignored while evaluating the programs. The evaluators were unaware whether the programs Sims, Counter-Strike, and Plan Eat Smile are approved by an authority or not. Thus, the result may be more an Evaluator s reflection than a problem within the program. It is also evident that score of the heuristics corresponding to the principle reciprocity is mostly diverging between the evaluators. This principle was more of a personal reflection of the evaluator whether he or she wishes to do a favor for a particular program. 55

71 6 Conclusion In this thesis work, a list of 30 heuristics was developed to serve as metrics in the summative evaluation stages of the software programs. The adaptation of the abstract principles for persuasion into more specialized heuristics indicates that the principles can be expressed in a way more appropriate to evaluate the persuasive systems. The heuristics developed were used to analyze the six software programs: MS-Word, MS-PowerPoint, Counter-Strike, The Sims, Stone, and Plan Eat Smile. The evaluators were able to successfully conduct the evaluation with the heuristics and provide their feedback and comments. MS Word and MS PowerPoint are the software programs with low potential to cause behavior or attitude change. It is because both the programs lack in the persuasive features praise, rewards and conditioning. Being tools, they do not have intentions on their own and possess less persuasive features to engage people as a medium or social actor. Comparing the two games Counter-Strike and Sims, they show considerable difference of 0.6 in the mean score. Counter-Strike has failed to fulfill three principles simulation, attractiveness and suggestion whereas Sims satisfies the same three principles with its attractive design, advanced graphics and recommendation system. This makes Sims as a software program with higher persuasive potential when compared to Counter-Strike or any other programs. Being educational programs, Stone has more persuasive potential when compared to plan eat smile. Even though, plan eat smile satisfies more heuristics compared to stone, it lacks some features like tailoring, surface credibility and conditioning. Stone has a good mean score for the system credibility group of principles and exhibits significant persuasive potential which results in learning and significant behavior change in users Principles rewards and praise are absent in both the programs. The different scores and feedback indicate that the heuristics developed present distinct behavior when used for evaluation. As anticipated, the six programs were chosen because they were known to be efficient and the summary evaluations performed on them indicate the persuasive potential of the programs. The diverse test results provide good understanding about the software evaluated and show that the heuristics developed can be used to evaluate different software programs. The relatively high rate of agreement between the three evaluators also indicates that the method heuristic evaluation is highly reliable. 56

72 This implies that it is likely to use summative heuristic evaluation for the assessment of persuasive systems and the concrete heuristics developed can be applied with a high degree of consistency. This thesis work shows the ability that the evaluation technique will be useful to designers as they provide resource-intensive process and deploy a fully functional system. 57

73 Bibliography Ad Council, U.s.F.S., n.d. Smokey Bear. [Online] Available at: [Accessed March 2012]. Bång, M., TORSTENSSON, C. & KATZEFF, C., The powerhouse: A persuasive computer game designed to raise awareness of domestic energy consumption. Berdichevsky, D. & Neuenschwander, E., Toward an ethics of persuasive technology. Communications of the ACM, pp Carenini, G., Evaluating User-Tailored Evaluative Arguments: Persuasive Effectiveness and its Communicative Appropriateness. Proceedings of the IJCAI Workshop on Computational Models of Natural Argument. Cegala, D.J., Persuasive communication: Theory and practice. 3rd ed. Council, H.S., coderedrover. [Online] Available at: [Accessed March 2012]. Council, C., n.d. alcohol 101 plus. [Online] Available at: [Accessed March 2012]. Crowley, D. & Selvadurai, N., n.d. Foursquare. [Online] Available at: [Accessed June 2012]. Fogg, B.J., Persuasive Technology : Using Computers to Change What We Think and Do. Morgan Kaufmann Publishers. Fogg, B.J., A behavior model for persuasive design. ACM International Conference Proceeding Series. FOGG, B.J., Creating persuasive technologies: An eight-step design process. ACM International Conference Proceeding Series. Gustafsson, A. & Bång, M., Evaluation of a pervasive game for domestic energy engagement among teenagers. International Conference on Advances in Computer Entertainment Technology, pp Harjumaa, M. & Oinas-Kukkonen, H., Persuasion theories and IT design. Harjumaa, M., Segerståhl, K. & Oinas-Kukkonen, H., Understanding persuasive software functionality in practice. ACM International Conference Proceeding Series, pp.1-9. Health, B.U.S.o.P., QuitNet. [Online] Available at: [Accessed 12 may 2012]. Inc, M.N.O.C., posornot. [Online] Available at: [Accessed March 2012]. 58

74 Kientz, J. et al., Heuristic Evaluation of Persuasive Health Technologies. ACM International Health Informatics Symposium. Kimura, H. & Nakajima, Designing persuasive applications to motivate sustainable behavior in collectivist cultures. Psychology, pp Laschke, M., Hassenzahl, M. & Diefenbach, S., With a little help from a friend: a shower calendar to save water. In CHI '11 Extended Abstracts on Human Factors in Computing Systems., Newman, W.M. & Lamming, M.G., Interactive system design. Addison-Wesley. Nielsen, J., Finding usability problems through heuristic evaluation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp Nielsen, J. & Mack, R.L., Usability inspection methods. Nielsen, J. & Molich, R., Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.pages Nielsen, J., n.d. use it. [Online] Available at: [Accessed March 2012]. Oinas- Kukkonen, H., Behavior change support systems: A research model and agenda. Oinas- Kukkonen, H. & Harjumaa, M., Persuasive systems design: Key issues, process model, and system features. Communications of the Association for Information Systems, pp Oinas-Kukkonen, H. & Harjumaa, M., A Systematic Framework for Designing and Evaluating Persuasive Systems. Oinas-Kukkonen, H. & Harjumaa, M., Towards deeper understanding of persuasion in software and information systems. Proceedings of the 1st International Conference on Advances in Computer-Human Interaction, pp Patzer, A., n.d. Mint. [Online] Available at: [Accessed March 2012]. Ragnemalm, E.L., Bång, M. & Alin-Nilsson, I., Heuristic Evaluation of persuasive systems: the case of educational programs. The Sixth International Conference on Persuasive Technology (Persuasive 2011). Torning, K. & Oinas-Kukkonen, H., Persuasive system design : State of the art and future directions. ACM International Conference Proceeding Series Wai, C. & Mortensen, P., Persuasive Technologies Should Be Boring. Lecture Notes in Computer Science, 4744/2007, pp Yeo, A.C., Rahim, M.M. & Ren, Y.Y., Use of Persuasive Technology to Change End- Users IT Security Aware Behaviour: A Pilot Study. International Journal of Human and Social Sciences. 59

75 60

76 Appendix Appendix A: Heuristics SYSTEM CREDIBILITY PERSUASIVE PRINCIPLES HEURISTICS Trustworthiness 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? Expertise 1. Does the program show competency (skill/expertise)? Surface credibility 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? Earned credibility 1. Is the program behavior consistent and predictable? (near) perfection 1. Does the program make any errors? COMPUTER AS A PERSUASIVE TOOL PERSUASIVE PRINCIPLES HEURISTICS Reduction 1. Does the program make tasks simpler? 61

77 Tunneling 1. Does the program steer users through a sequence of steps in order to complete a task? Tailoring 1. Is the program tailored to the specific user or situation? 2. Is it possible to configure the program to the needs of a specific user? Suggestion 1. Does the program provide any suggestions for tasks and are they provided in a timely manner? 2. Does the program provide HELP option to assist the user? Self-monitoring 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? Surveillance 1. Is it possible to view another user s results? Conditioning 1. Is positive feedback provided immediately when a good/desired action is taken or good result achieved? COMPUTER AS A PERSUASIVE MEDIUM (SIMULATION) PERSUASIVE PRINCIPLES HEURISTICS Cause and Effect 1. Does the program show any effects or consequences of user s actions? Virtual Rehearsal 1. Does the program allow the user to repeat actions and activities? Virtual Rewards 1. Does the program provide any reward for actions/behavior? Simulation 1. Is the similarity to the real world high? 2. Do things behave in a plausible way? 62

78 COMPUTER AS A PERSUASIVE SOCIAL ACTOR PERSUASIVE PRINCIPLES HEURISTICS Attractiveness Similarity 1. Is the Program design visually pleasing and interesting? 1. Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? Praise 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? Reciprocity 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? Authority 1. Does the program present an authoritative image? 2. Does the program make claims (quoting an authority)? 63

79 Appendix B: Evaluation Feedback SYSTEM CREDIBILITY # PRINCIPLES FEEDBACK COMMENTS 1.1 Trustworthiness The system that is observed as truthful, fair and unbiased will have increased powers of persuasion.(three aspects: credibility of source, arguing against own interests, familiar) 1. Does the Program appear Truthful, fair, Impartial? Yes, it does what it should and I can find the information on the first page 2. Does it provide credible Information about the program and its source? 1.2 Expertise The system that is viewed as incorporating expertise (knowledge, experience, and competence) will have increased powers of persuasion. Simple intuitive and functional 1. Does the program show competency (skill/expertise)? 1.3 Surface credibility People make initial assessments of the system creditability based on a firsthand inspection. Does the program look like it can do what it is supposed to do? It s just perfect 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? 64

80 1.4 Earned credibility It s simple and works well. Layout is clear. Performance consistent with expectations (positive) 1. Is the program behavior consistent and predictable? 1.5 (near) perfection No errors or mistakes (or very few and easily explained) 1. Does the program make any errors? No COMPUTER AS TOOL # SCENARIO FEEDBACK COMMENTS 2.1 Reduction A system that reduces complex behavior into simple tasks. Simplifying the target behavior may increase the benefit/cost ratio of a behavior. 1. Does the program make tasks simpler? 2.2 Tunneling Guiding the user through a process or experience provides opportunities to persuade. 1. Does the program steer users through a sequence of steps in order to complete a task? Yes, the order for performing the tests. Yes, the steps to proceed from one test to another. 2.3 Tailoring It is uniformed and anybody can use it. 65

81 Tailoring information to the user's specific situation, interests, personality or needs increases the persuasive effect. 1. Is the program tailored to the specific user or situation? 2. Is it possible to configure the program to the needs of a specific user? 2.4 Suggestion It s intuitive. Intervene at a time when the target behavior is/would be relevant. Heuristic for Tunneling covers this. it provides static help function 1. Does the program provide any suggestions for tasks and are they provided in a timely manner? 2. Does the program provide HELP option to assist the user? 2.5 Self-monitoring Systems that provide simulations can persuade by enabling them to observe immediately the link between the cause and the effect. 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? It stores and color changes when they show results. 2.6 Surveillance No it s not possible Overt monitoring of behavior increases desire to fulfill perceived expectations. 1. Is it possible to view another user s results? 2.7 Conditioning Yes its fine Using operant conditioning - immediate positive reinforcement after target behavior. 66

82 1. Is positive feedback provided immediately when a good/desired action is taken or good result achieved? COMPUTER AS MEDIUM (SIMULATION) # SCENARIO FEEDBACK COMMENTS 3.1 Cause and Effect Systems that provide simulations can persuade by enabling them to observe immediately the link between the cause and its effect. 1. Does the program show any effects or consequences of user s actions? 3.2 Virtual Rehearsal A System providing means with which to rehearse a behavior can enable people to change their attitudes or behavior in the real world. 1. Does the program allow the user to repeat actions and activities? 3.3 Virtual Rewards A System that provide virtual rewards for users in order to give credit for performing the target behavior. 1. Does the program provide any reward for actions/behavior? It s not real-time but it does show lot of effects To some degree. It gives green mark for correct actions. 67

83 3.4 Simulation A system providing persuasive information vicariously through life-like settings. 1. Is the similarity to the real world high? Not bad. Yes, they behave like a normal online test. 2. Do things behave in a plausible way? COMPUTER AS SOCIAL ACTOR # SCENARIO FEEDBACK COMMENTS 4.1 Attractiveness Yes, It s attractive. A System that is visually attractive for its users is likely to be more persuasive 1. Is the Program design visually pleasing and interesting? 4.2 Similarity A System that imitate users in some specific way. 1. Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? No, personalization is not possible. No, it does relate the user 4.3 Praise Praise leads the user to be more receptive to the giver of praise. Yes, just it says whether it is a correct or wrong answer. 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? 68

84 4.4 Reciprocity Returning a favor. 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? Yes it is helpful. 4.5 Authority A system that refers to people in the role of authority Yes, authorized by Linköping university. 1. Does the program present an authoritative image? 2. Does the program show make claims (quoting an authority)? 69

85 Appendix C: Results Summary MS WORD # HEURISTICS EVALUATOR A EVALUATOR B EVALUATOR C 1 S Y S T E M 2 Expertise C R E D Trustworthiness 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? 1. Does the program show competency (skill/expertise)? 3 I Surface credibility B 1. Is the program visually I attractive? L 2. Is the language used I logical, understandable T and familiar to the users? 4 Y Earned credibility 1. Is the program behavior consistent and predictable? 5 (near) perfection 1. Does the program make any errors? 6 C O M P 7 U T E R Tunneling Reduction 1. Does the program make tasks simpler? 1. Does the program steer users through a sequence of steps in order to complete a task? 8 A Tailoring S 1. Is the program tailored to the specific user or A situation? 2. Is it possible to configure the program to the needs of a specific P user? 9 E Suggestion R 1. Does the program S provide any suggestions U for tasks and are they A provided in a timely S manner? I 2. Does the program V provide HELP option to E assist the user? 10 Self-monitoring 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? 1 Extremely satisfies the heuristic, 2 Very satisfied, 3 Moderately satisfies, 4 Slightly satisfies, 5 Does not satisfy 70

86 11 Surveillance 1. Is it possible to view another user s results? T 12 O Conditioning O 1. Is positive feedback L provided immediately when a good/desired action is taken or good result achieved? 13 P Cause and Effect E 1. Does the program show R any effects or S consequences of user s U actions? 14 A Virtual Rehearsal S I V E 1. Does the program allow the user to repeat actions and activities? 15 Virtual Rewards 16 M E D I U M Simulation 17 C O M P U 18 T Similarity E R A S 19 A Praise S O C I A L 21 A C T O R Authority 1. Does the program provide any reward for actions/behavior? 1. Is the similarity to the real world high? 2. Do things behave in a plausible way? Attractiveness 1. Is the Program design visually pleasing and interesting? 1.Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? 20 Reciprocity 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? 1. Does the program present an authoritative image? 2. Does the program make claims (quoting an authority)? 71

87 MS POWERPOINT # HEURISTICS EVALUATOR A EVALUATOR B EVALUATOR C 1 S Y S T E M 2 Expertise Trustworthiness 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? 3 C R E D I Surface credibility 1. Does the program show competency (skill/expertise)? 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? 4 B I L I T Y Earned credibility 1. Is the program behavior consistent and predictable? 5 (near) perfection 1. Does the program make any errors? 6 C O M P 7 U T E R Tunneling 8 Tailoring A 1. Is the program tailored S to the specific user or situation? A 2. Is it possible to configure the program to the needs of a specific user? T O O Reduction 1. Does the program make tasks simpler? 1. Does the program steer users through a sequence of steps in order to complete a task? 9 P Suggestion E 1. Does the program R S provide any suggestions for tasks and are they U A provided manner? in a timely S 2. Does the program I V provide HELP option to assist the user? 10 E Self-monitoring 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? 1 Extremely satisfies the heuristic, 2 Very satisfied, 3 Moderately satisfies, 4 Slightly satisfies, 5 Does not satisfy 72

88 11 L Surveillance 1. Is it possible to view another user s results? 12 Conditioning 1. Is positive feedback provided immediately when a good/desired action is taken or good result achieved? 13 P E R S U 14 A Virtual Rehearsal S I V E Cause and Effect 1. Does the program show any effects or consequences of user s actions? 1. Does the program allow the user to repeat actions and activities? 15 Virtual Rewards 16 M E D I U M Simulation 1. Does the program provide any reward for actions/behavior? 1. Is the similarity to the real world high? 2. Do things behave in a plausible way? 17 C O M P U 18 T Similarity E R A S Attractiveness 1. Is the Program design visually pleasing and interesting? 1.Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? A 19 Praise 20 S O C I A L Reciprocity A C T O R 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? 21 Authority 1. Does the program present an authoritative image? 2. Does the program make claims (quoting an authority)? 73

89 COUNTER-STRIKE # HEURISTICS EVALUATOR A EVALUATOR B EVALUATOR C 1 S Y S T E M 2 Expertise Trustworthiness 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? 3 C R E D I Surface credibility 1. Does the program show competency (skill/expertise)? 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? 4 B I L I T Y Earned credibility 1. Is the program behavior consistent and predictable? 5 (near) perfection 1. Does the program make any errors? 6 C O M P 7 U T E R Tunneling Reduction 1. Does the program make tasks simpler? 1. Does the program steer users through a sequence of steps in order to complete a task? 8 Tailoring A 1. Is the program tailored S to the specific user or situation? A 2. Is it possible to configure the program to the needs of a specific user? 9 P Suggestion E 1. Does the program R S provide any suggestions for tasks and are they U A provided manner? in a timely S 2. Does the program I V provide HELP option to assist the user? 10 E Self-monitoring 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? 1 Extremely satisfies the heuristic, 2 Very satisfied, 3 Moderately satisfies, 4 Slightly satisfies, 5 Does not satisfy 74

90 11 Surveillance 1. Is it possible to view another user s results? T O 12 O Conditioning L 1. Is positive feedback provided immediately when a good/desired action is taken or good result achieved? 13 P E R S U 14 A Virtual Rehearsal S I V E Cause and Effect 1. Does the program show any effects or consequences of user s actions? 1. Does the program allow the user to repeat actions and activities? 15 Virtual Rewards 16 M E D I U M Simulation 1. Does the program provide any reward for actions/behavior? 1. Is the similarity to the real world high? 2. Do things behave in a plausible way? 17 C O M P U 18 T Similarity E R A S Attractiveness 1. Is the Program design visually pleasing and interesting? 1.Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? A 19 Praise 20 S O C I A L Reciprocity A C T O R 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? 21 Authority 1. Does the program present an authoritative image? 2. Does the program make claims (quoting an authority)? 75

91 THE SIMS # HEURISTICS EVALUATOR A EVALUATOR B EVALUATOR C 1 S Y S T E M 2 Expertise Trustworthiness 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? 3 C R E D I Surface credibility 1. Does the program show competency (skill/expertise)? 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? 4 B I L I T Y Earned credibility 1. Is the program behavior consistent and predictable? 5 (near) perfection 1. Does the program make any errors? 6 C O M P 7 U T E R Tunneling Reduction 1. Does the program make tasks simpler? 1. Does the program steer users through a sequence of steps in order to complete a task? 8 Tailoring A 1. Is the program tailored S to the specific user or situation? A 2. Is it possible to configure the program to the needs of a specific user? 9 P Suggestion E 1. Does the program R S provide any suggestions for tasks and are they U A provided manner? in a timely S 2. Does the program I V provide HELP option to assist the user? 10 E Self-monitoring 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? 1 Extremely satisfies the heuristic, 2 Very satisfied, 3 Moderately satisfies, 4 Slightly satisfies, 5 Does not satisfy 76

92 11 Surveillance 1. Is it possible to view another user s results? T O 12 O Conditioning L 1. Is positive feedback provided immediately when a good/desired action is taken or good result achieved? 13 P E R S U 14 A Virtual Rehearsal S I V E Cause and Effect 1. Does the program show any effects or consequences of user s actions? 1. Does the program allow the user to repeat actions and activities? 15 Virtual Rewards 16 M E D I U M Simulation 1. Does the program provide any reward for actions/behavior? 1. Is the similarity to the real world high? 2. Do things behave in a plausible way? 17 C O M P U 18 T Similarity E R A S Attractiveness 1. Is the Program design visually pleasing and interesting? 1.Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? A 19 Praise 20 S O C I A L Reciprocity A C T O R 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? 21 Authority 1. Does the program present an authoritative image? 2. Does the program make claims (quoting an authority)? 77

93 STONE # HEURISTICS EVALUATOR A EVALUATOR B EVALUATOR C 1 S Y S T E M 2 Expertise Trustworthiness 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? 3 C R E D I Surface credibility 1. Does the program show competency (skill/expertise)? 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? 4 B I L I T Y Earned credibility 1. Is the program behavior consistent and predictable? 5 (near) perfection 1. Does the program make any errors? 6 C O M P 7 U T E R Tunneling Reduction 1. Does the program make tasks simpler? 1. Does the program steer users through a sequence of steps in order to complete a task? 8 Tailoring A 1. Is the program tailored S to the specific user or situation? A 2. Is it possible to configure the program to the needs of a specific user? 9 P Suggestion E 1. Does the program R S provide any suggestions for tasks and are they U A provided manner? in a timely S 2. Does the program I V provide HELP option to assist the user? 10 E Self-monitoring 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? 1 Extremely satisfies the heuristic, 2 Very satisfied, 3 Moderately satisfies, 4 Slightly satisfies, 5 Does not satisfy 78

94 11 Surveillance 1. Is it possible to view another user s results? T O 12 O Conditioning L 1. Is positive feedback provided immediately when a good/desired action is taken or good result achieved? 13 P E R S U 14 A Virtual Rehearsal S I V E Cause and Effect 1. Does the program show any effects or consequences of user s actions? 1. Does the program allow the user to repeat actions and activities? 15 Virtual Rewards 16 M E D I U M Simulation 1. Does the program provide any reward for actions/behavior? 1. Is the similarity to the real world high? 2. Do things behave in a plausible way? 17 C O M P U 18 T Similarity E R A S Attractiveness 1. Is the Program design visually pleasing and interesting? 1.Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? A 19 Praise 20 S O C I A L Reciprocity A C T O R 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? 21 Authority 1. Does the program present an authoritative image? 2. Does the program make claims (quoting an authority)? 79

95 PLAN EAT SMILE # HEURISTICS EVALUATOR A EVALUATOR B EVALUATOR C 1 S Y S T E M 2 Expertise Trustworthiness 1. Does the Program appear Truthful, fair, Impartial? 2. Does it provide credible Information about the program and its source? 3 C R E D I Surface credibility 1. Does the program show competency (skill/expertise)? 1. Is the program visually attractive? 2. Is the language used logical, understandable and familiar to the users? 4 B I L I T Y Earned credibility 1. Is the program behavior consistent and predictable? 5 (near) perfection 1. Does the program make any errors? 6 C O M P 7 U T E R Tunneling Reduction 1. Does the program make tasks simpler? 1. Does the program steer users through a sequence of steps in order to complete a task? 8 Tailoring A 1. Is the program tailored S to the specific user or situation? A 2. Is it possible to configure the program to the needs of a specific user? 9 P Suggestion E 1. Does the program R S provide any suggestions for tasks and are they U A provided manner? in a timely S 2. Does the program I V provide HELP option to assist the user? 10 E Self-monitoring 1. Does the program store and provide assessment of user history and results? 2. Does the program visualize actions done by the user? 1 Extremely satisfies the heuristic, 2 Very satisfied, 3 Moderately satisfies, 4 Slightly satisfies, 5 Does not satisfy 80

96 11 Surveillance 1. Is it possible to view another user s results? T O 12 O Conditioning L 1. Is positive feedback provided immediately when a good/desired action is taken or good result achieved? 13 P E R S U 14 A Virtual Rehearsal S I V E Cause and Effect 1. Does the program show any effects or consequences of user s actions? 1. Does the program allow the user to repeat actions and activities? 15 Virtual Rewards 16 M E D I U M Simulation 1. Does the program provide any reward for actions/behavior? 1. Is the similarity to the real world high? 2. Do things behave in a plausible way? 17 C O M P U 18 T Similarity E R A S Attractiveness 1. Is the Program design visually pleasing and interesting? 1.Is it possible to personalize your character or design environment? 2. Does the program relate the user in some meaningful way? A 19 Praise 20 S O C I A L Reciprocity A C T O R 1. Does the program display any messages to cheer the user? 2. Does the program offer any praise when the User finishes a task? 1. Does the Program appear helpful, entertaining, enough to return a favor (like feedback) for the system? 21 Authority 1. Does the program present an authoritative image? 2. Does the program make claims (quoting an authority)? 81

97 Appendix D: Data Calculation MS Word System Credibilit y Compute r as Tools Compute r as Medium Compute r as Social Actor HEURISTICS EVALUATO R 1 EVALUATOR 2 EVALUATO R 3 Trustworthiness Expertise Surface credibility Earned credibility (near) perfection Average Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Average Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Average Attractiveness Similarity Praise Reciprocity Authority Average AVERAGE MS PowerPoint System Credibilit y HEURISTICS EVALUATO R 1 EVALUATOR 2 EVALUATO R 3 Trustworthiness Expertise Surface credibility Earned credibility (near) perfection Average AVERAGE

98 Compute r as Tools Compute r as Medium Compute r as Social Actor Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Average Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Average Attractiveness Similarity Praise Reciprocity Authority Average Counter-Strike System Credibilit y Compute r as Tools Compute r as Medium HEURISTICS EVALUATO R 1 EVALUATOR 2 EVALUATO R 3 Trustworthiness Expertise Surface credibility Earned credibility (near) perfection Average Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Average Cause and Effect Virtual Rehearsal Virtual Rewards Simulation AVERAGE

99 Compute r as Social Actor Average Attractiveness Similarity Praise Reciprocity Authority Average Sims System Credibilit y Compute r as Tools Compute r as Medium Compute r as Social Actor HEURISTICS EVALUATO R 1 EVALUATOR 2 EVALUATO R 3 Trustworthiness Expertise Surface credibility Earned credibility (near) perfection Average Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Average Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Average Attractiveness Similarity Praise Reciprocity Authority Average AVERAGE Stone HEURISTICS EVALUATO R 1 EVALUATOR 2 EVALUATO R 3 Trustworthiness Expertise AVERAGE 84

100 System Credibilit y Compute r as Tools Compute r as Medium Compute r as Social Actor Surface credibility Earned credibility (near) perfection Average Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Average Cause and Effect Virtual Rehearsal Virtual Rewards Simulation Average Attractiveness Similarity Praise Reciprocity Authority Average System Credibilit y Compute r as Tools Plan Eat Smile HEURISTICS EVALUATO R 1 EVALUATOR 2 EVALUATO R 3 Trustworthiness Expertise Surface credibility Earned credibility (near) perfection Average Reduction Tunneling Tailoring Suggestion Self-monitoring Surveillance Conditioning Average Cause and Effect AVERAGE

101 Compute r as Medium Compute r as Social Actor Virtual Rehearsal Virtual Rewards Simulation Average Attractiveness Similarity Praise Reciprocity Authority Average Total Average Calculation SOFTWARE EVALUATOR 1 EVALUATOR 2 EVALUATOR 3 TOTAL AVERAGE S/W WORD POWERPOINT COUNTER-STRIKE SIMS STONE PLAN EAT SMILE TOTAL AVERAGE/EVALUATOR