USER EXPERIENCE AS A TEST PLATFORM FOR INNOVATIONS

Transcription

1 Lappeenranta University of Technology School of Industrial Engineering and Management Degree Program in Computer Science Francis Matheri USER EXPERIENCE AS A TEST PLATFORM FOR INNOVATIONS Examiners : Dr. Andrey Maglyas D.Sc (Tech) Supervisors: Dr. Andrey Maglyas D.Sc (Tech)

2 ABSTRACT Lappeenranta University of Technology School of Industrial Engineering and Management Degree Program in Computer Science Francis Matheri User Experience as a testing platform for innovation 19 pages 1 Figure Examiner: Dr. Andrey Maglyas Keywords: user experience, usability, innovation, evaluation method, user experience perspective Innovation enhances the application of better solutions that meet not only new requirements, but also both the unarticulated and existing market needs. This can be achieved through more effective products, processes, technologies, ideas and services that are readily available in the markets, society and government. At the same time, the existence of these products, processes, technologies, ideas and services need to be appreciated by the end user for their successful implementation and use. Therefore the experience of the user which includes behaviour, beliefs, perception, physical and psychological accomplishments and response before during and after use is very vital. This paper describes user experience, discusses the methods and elements of user experience methods that can optimally obtain responses from the users based on their experience in the use of a product, system and /or a service. The paper also focuses on how the user experience influences innovation. 1

3 TABLE OF CONTENT 1 INTRODUCTION BACKGROUND GOALS OF THE RESEARCH SCOPE OF THE RESEARCH STRUCTURE OF THE PAPER DEFINITION OF USER EXPERIENCE USER EXPERIENCE PERSPECTIVES UX as a phenomenon Experiencing A user experience Co-experience User experience as a practice UX factors Methods, tools and criteria Representing concepts and designs User experience evaluation methods Distinguishing User experience from usability evaluation methods Distinguishing evaluation from Design methods UX evaluation methods Experience clip Experiential Contextual Inquiry Emotion sampling device (ESD) Oracle Case Study CONCLUSION REFERENCES

4 1 INTRODUCTION 1.1 BACKGROUND The term user experience is widely used but understood in many different ways. This can be attributed to its multidisciplinary nature, which has consequently resulted to many definitions and perspectives of user experience with each approaching the concept from a different viewpoint. As a result myriad of existing definitions of user experience exist ranging from psychological to business perspective and from quality centric t value centric. As such there is no an all-in-one user experience definition that suits all the perspectives. Meanwhile the field of user experience has broadened its horizons leading to practitioners discussing about user experience in areas far beyond the limited context of Technological aspects like the System for instance the applications. The context of user experience has been extended to include the experience that the products create for the people who use them in the real world. The users of the product pay a great deal of attention to what it does. Though at times how user experience works can be overlooked and the consequent impact can be used to make the difference between a successful product and a failure. In addition, different users have different beliefs, perceptions, behaviours, psychological responses especially about using a particular product, system, service. This variation can range from person's perception of a product aspect. For instance if such a product is a system, the aspects can include utility, ease of use, constant change over time due to changing usage circumstances and changes to individual systems as well as the wiser usage context. From a broader perspective, user experience is not fully focused on the inner workings of a product or service rather it s about how a product or service works on the outside, where a person comes into contact with it. In this context, a number of questions can arise whose answers provide more meaning to the definition of user experience. The questions can include: what is it like to use a product or service, how does it feel to interact with the product and many more others. Therefore a clear definition of user experience would help achieve the following: Clarify the different perspectives of user experience not only to users but also to researchers and practitioners. Establish practical user experience work in commercial, industrial and government organizations. 3

5 Educate people by communicating the meaning of the term especially to those who don t know it. Teach the basics of user experience Also to advance the user experience as a research field. This paper therefore describes the core concepts of user experience which include the different perspectives of user experience and the User experience evaluation methods that can not only be used to teach users of a product, service or a system basics of user experience but also to gain the best out the same users in business or organization perspective. 1.2 GOALS OF THE RESEARCH The main objective of this research is to describe user experience as a test platform for innovations. The description entails the discussion of the core concepts of user experience which include both the user experience evaluation methods and the different perspectives of user experience. The research questions to be addressed include: How does user experience influences the interaction between a user and a product, service or a system. What are the different perspectives pertaining the user experience Are there different methods to measure and evaluate user experience from both the business and the user end. 1.3 SCOPE OF THE RESEARCH The scope of this research has been limited to experience of a user interacting with technological products and services such as system specifically computer applications and the services emanating from them. In terms of perspectives, this research is limited only to practice and phenomena perspective of the user experience therefore disregarding all other forms of perspectives such as the user experience as a study; studying for example how experiences are formed and or what a person experiences or has already experienced. 4

6 The scope of the research can further be divided into the following subsections: To describe user experience in general by discussing various definitions of the term. To discuss user experience as from different perspectives especially the practice and the phenomena approach. To analyze the different methods of measuring and evaluating user experience 1.4 STRUCTURE OF THE PAPER Section 2 contains a description of the user experience from a general point of view. This section describes various definitions of the term user experience, and depicts the same with an illustration. Section 3 describes the user experience perspectives emphasizing more on the practice and the phenomena perspective of user experience. Section 4 discusses the methods used to measure and evaluate user experience and gives a practical illustration used by big industries such as Nokia mobile company and the Oracle database companies. Section 5 provides the conclusion. 2 DEFINITION OF USER EXPERIENCE There has been a lot of recent debate about the scope of the user experience (UX) and how it should be defined [1]. According to ISO FDIS , user experience can be defined as a person perception that results from the use and/ or the anticipated user of a product, system, or a service. Basing this definition on the ISO definition that suggests that the measure of user experience is similar to measures of satisfaction in usability, it contrasts with the updated definition of usability in ISO FDIS which is: The extent to which a system, a service or a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [2]. Nevertheless, both the definitions suggest that user experience or usability can be measured during and after use of a product, service, or a system. As a result of usability evolving to user experience, the measurements of user experience broadened from easy and efficient to even more delighting. Consequently different researchers have proposed various changes to the concept of user experience. For instance Jordan has extended the user experience further by upgrading his list to include s usability, functionality, and pleasure and pride [3]. 5

7 Similarly researcher such as Norman advanced the user experience concept more through setting a goal, and engaging the users in Visceral, behavioral, and reflective level [4]. In addition the user experience definitions are supplemented by other interpretations: User experience explores how a person feels about using a particular product, experiential, affective, meaningful and valuable aspects of products use. User experience is generally understood as inherently dynamic, given the ever-changing internal and emotional state of a person and differences in the circumstances during and after an interaction with a product [5]. Therefore user experience should not only be seen as something valuable after interacting with the object but also before and during interaction. While it is relevant to evaluate short-term experiences, given dynamic changes of user goals and needs related to contextual factors, it is also important to know how (and why) experiences evolve over time. In addition, users values affect their experiences with products and services, and thus this relationship has to be considered in the design process right from the beginning [6]. These points make already make it clear that it is important to look beyond the static aspects and investigate the temporal aspects of user UX [7]. A thorough understanding of user s experiences, be they positive or negative, a product evokes is the core of user experience evaluation [5]. There seems to be a double-edged relationship with the products and services the clients use. At time these products empower us and frustrate us, at the same time they may simplify and complicate our lives, likewise they may may separate us and bring us closer together. Even though, users tend to forget that these products are made by other people and they should get credit when they work well and on the contrary get the blame when they don t. 3 USER EXPERIENCE PERSPECTIVES The field of user experience deals with studies, designing for and evaluating the experiences that people have through the use of or the encounter with a system [4]. This takes place in a specific context, which has no impact on, or contributes, to the UX. User experience can be viewed from different perspectives: it can be seen as phenomena, as a field of study, or a practice. To consider these phenomena, consider the following analogy: health as a phenomena, medicine as a field of study and doctor s work as a practice. 6

8 Each of these views can further be detailed as follows: UX as a phenomenon: Describing what UX is and what it is not Identifying the different types of UX Explaining the circumstances and consequences of UX UX as a field of study: Studying the phenomenon, for example how experiences are formed or what a person experiences, expects to experience, or has experienced Finding the means to design systems that enable particular UXs Investigating and developing UX design and assessment methods UX as a practice: Envisioning UX, for example, as part of a design practice Representing UX, for example, building a prototype to demonstrate and communicate the desired UX to others Evaluating UX Delivering designs aimed at enabling a certain UX In this paper, we mainly focus on user experience as a phenomenon and user experience as practice. 3.1 UX as a phenomenon This UX perspective attributes pivotal role to the existence of people. Therefore the notion of experience in general covers everything personally encountered, undergone or lived through. As a result, UX entails the experience the product creates for the people who use it in the real world. Thus when a product is being developed, people pay a great deal of attention to what it does. However user experience at times and how it works can be overlooked thus resulting to impact which can make the difference between a successful product and a failure [6]. In effect user experience differs from experiences in a general sense in that, it explicitly refers to the experiences derived from the systems. User experience as a phenomenon can further be described as follows: UX is a subset of experience as a general concept. UX is more specific, since it is related to the experiences of using a system 7

9 UX includes encounters with systems not only active, personal use, but also being confronted with a system in a more passive way, for example, observing someone else using a system UX is unique to an individual UX is influenced by prior experiences and expectations based on those experiences UX is rooted in a social and cultural context In addition, the following descriptions state what user experience is not: UX is not technology driven, but focuses on humans UX is not about just an individual using a system in isolation UX is not just cognitive task analysis, or seeing users as a human information processor. UX is not the same as usability, although usability, as perceived by the user3, is typically an aspect contributing to the overall UX UX design is more than user interface design UX differs from the broader concepts of brand/consumer/customer experience, although UX affects them and vice versa Despite the fact that user experience has a narrower scope than general experience, user experience is still an umbrella term that may refer to several forms of user experience [7]. For instance it illustrates the interaction between a user and a product such as a system. That interaction often involves pushing a lot of buttons, as in the case of technology products such as alarm clocks, coffeemakers, or cash registers. Sometimes, it s just a matter of a simple physical mechanism, such as the gas cap on your car. However, every product that is used by someone creates a user experience: books, ketchup bottles, reclining armchairs, cardigan sweaters [7]. For any kind of product or service, it s the little things that count.having a button click when you push it down doesn t seem like much, but when that click makes the difference between getting coffee and not getting coffee, it matters a great deal. Even if you never realized that the design of that button was causing you trouble, how would you feel about a coffeemaker that you were able to use successfully only part of the time. More specific terms may help in explaining the intended perspective. The description of three different perspectives on user experience that people may take when referring to user 8

10 experience follows in the subsequent section. In general these terms are similar to those used in experience design in general Experiencing The verb experiencing refers to an individual s stream of perceptions, interpretations of those perceptions, and resulting emotions during an encounter with a system. Each person may experience an encounter with a system in a different way. This view emphasizes the individual and dynamic nature of experiencing the encounter with a system [4]. In this regard, the designers focusing on experiencing usually pay attention to specific interaction events, which may have an impact on the user s emotion, for instance n game design, scoring a goal or the appearance of a frightening character. Evaluating of experiencing could focus on how a single person experiences the encounter with a system from moment to moment, for example measuring emotions at various moments in time to uncover which elements in an interaction may induce which emotions [3] A user experience The noun user experience refers to an encounter with a system that has a beginning and an end. It refers to an overall designation of how people have experienced (verb) a period of encountering a system. This view emphasizes the outcome and memories of an experience rather than its dynamic nature. It does not specifically emphasize its individual nature because a user experience can refer to either an individual or a group of people encountering a system together [7]. In this example, typical examples include the focus of UX design on a specific period of activities or tasks for example visiting a website. In narratives of games the example includes the building up suspense and having a happy end. Finally in the outcome after using a system the typical example include having learned a dance with a dance game. In this perspective, evaluation could focus on methods that can provide an overall measure for the experience of a certain activity or system use for instance the retrospective questionnaire method Co-experience Co-experience can also be termed as shared experience, and group experience. It refers to the situations in which experiences are interpreted as being situated and socially constructed. The emphasis is not only on encountering a system, but also on people constructing and at the 9

11 same time experiencing a situation together. If these terms are used without considering the role of a specific system in the experience, then it no longer makes sense to talk about user experience, but more appropriately about experience in general[5]. Under this perspective when focusing on socially constructed experiences, group behaviour and/or group attitude is of importance. And as a result designing with a focus on socially constructed experiences may result in, for example a platform system providing general constraints and affordances for multiple people to act and interact rather than focusing on the determined flow of interaction and outcome for one person. For evaluation, this could mean including indirect group experience, measures such as the number and nature of encounters between people. 3.2 User experience as a practice The roots of user experience design (UXD) can be found in the principles of Human Centred Design (HCD; ISO 13407:1999; revised by ISO ), which can be summarized as: Positioning the user as a central concern in the design process Identifying the aspects of the design that are important to the target user group Developing the design iteratively and inviting users participation Collecting evidence of user-specific factors to assess a design In principle, UXD is not different from HCD. However, UXD adds important dimensions to the challenge of implementing HCD in a mature form. These additions are not trivial. The main dimensions distinguishing UXD from a traditional view of HCD include UX factors; methods, tools and criteria used in UX work; representation of the UX idea; and UX positioning in the organization UX factors As discussed in the previous section, the factors affecting UX are significantly broader and more diverse than those traditionally within the scope of HCD. While traditional usability factors were largely related to performance and smooth interaction, new UX factors relate to affect, interpretation and meaning. Some UX factors, such as social and aesthetic aspects, are likely to be very different in character from the traditional concerns. This presents UX practitioners with significant challenges in terms of identifying which UX factors they need to consider when embarking on a design project. In any case, it is usual that a design team will only be able to deal with a few critical UX factors that influence the suitability of the 10

12 design for a typical usage situation. Consequently, a big challenge for design teams is to make sense of the available information during the early phases of the UXD process. Essentially this means: Scoping out the factors that are known, because evidence exists, or are thought likely to be the drivers of UX in their particular instance, Identifying those factors that are critical to the success of the design and can be satisfactorily dealt with by the design team, given their own operational circumstances, Identifying those factors that are likely to need further investigation and, if so, the form that those investigations could take Methods, tools and criteria All design teams face the challenge of making trade-offs between the various requirements that they have to meet. The intangible nature of UX makes it even more difficult to estimate the consequences of design decisions on the UX. It may be very difficult, if not impossible, for the design team to deal with some issues (e.g. social, emotional or aesthetic) in a very direct or explicit way. Design teams often have to handle them intuitively, relying on professional judgments. Design teams will need to identify applicable and feasible methods, tools and criteria that can be used to manage the UX factors throughout the process. This includes setting initial targets, managing the iterative development of design proposals, and supporting evaluation work during and after the design work. In many cases the factors may involve traditional usability issues that can be handled using conventional methods. No generally accepted overall measure of UX exists, but UX can be made assessable in many different ways. For example, there are tools for simply evaluating whether an evoked emotion is positive or negative. There are also methods and instruments specifically developed for evaluating particular UX qualities such as trust, presence, satisfaction or fun. The choice of an evaluation instrument or method depends on the experiential qualities at which the system is targeted, as well as on the purpose of the evaluation (e.g. summative or formative) and other (often pragmatic) factors such as time and financial constraints. 11

13 3.2.3 Representing concepts and designs Another big challenge is to find ways of giving people a sense of what the experience might be like before the design itself is available. Of particular importance is that a design team needs to create representations of the system to: Stimulate the participation of prospective users or their surrogates to gather feedback on design directions, enable the capture of emotional responses of people and their explanations of why, Communicate the concepts and designs to other colleagues, senior management and others who have an interest in the success of the design, Sustain the vision of the design team throughout the design process. 4 User experience evaluation methods User experience evaluation methods can be well defined by comparing user experience methods with usability methods and by distinguishing design methods from evaluation methods [9]. 4.1 Distinguishing User experience from usability evaluation methods The relationship between user experience and usability are intertwined. Attempts have been taken to demarcate the two or even dismiss the boundary between them operationally and conceptually [10]. In this regard, we take the assumption that usability is subsumed by User experience. Nevertheless the implication is that user experience evaluation entails the augmentation of methods for usability evaluation. Usability tends to focus on task performance whereas user experience focuses on lived performance [11]. User experience is subjective, objective usability measures such as task execution time and the number of clicks or errors are not sufficient measures for usability experience since the pivotal role of user experience is to determine the feeling or satisfaction of a user. On the contrary, even though the subjective component of user experience can be seen as a part of user experience evaluation, user experience addresses a range of other subjective qualities. A user s motivation and expectation plays a major role in usability than in traditional usability [12]. 12

14 4.2 Distinguishing evaluation from Design methods A sharp distinction between evaluation and design methods is at times hard to make.design methods are often called inspirational or generative methods and aim at bringing inspiration for developers when they create new products and designs [13]. The main focus of evaluation methods is to help in choosing the best design, to ensure that the development is on the right track, or to assess if the final product meets the original UX targets [14]. 4.3 UX evaluation methods There exist a myriad of user experience evaluation methods. In this paper we will focus on some of those methods. The description of the methods follows in the subsequent section Experience clip This UX evaluation method is best suited to applications especially the mobile applications. For instance when collecting information about experiences evoked by mobile applications, two users can be involved where one uses the mobile application to be evaluated and the other uses the mobile phone to capture clips about the usage and the expression the experiences [11]. In this method, it is assumed that the two users know each other very well and the usage situation is natural that is both in context and interaction. For better results, it is recommended to shoot as many clips as possible for easier elaboration of usage and their experiences. The pros of this method are its ability to permit the natural context by making use of the social interaction between users. In addition no special requirement is needed. On the contrary in this method, the quality of video is low, and there is time consuming analysis of the video material and also its ability to depend on users to determine the usefulness of the material Experiential Contextual Inquiry The method involves observing the users in real context, where the researcher or the practitioner takes the role of the apprentice. Originally the method was developed to observe and understand the work practices [13]. When focus is on user experience, the researcher pays attention to the emotional aspects of product use, not only the behavior but also the affective aspects of product use. 13

15 Contextual inquiry simply means conducting field interviews of workers in their work places while they work, observing and inquiring into the structure of their own work practice [12]. The strength of this method is that it guarantees real data in the real context of use as well as allowing the practitioners or the researchers to get the information about the environment. In addition, it guarantees the opportunity to concentrate more on what the customer does other than what they do. A disadvantage to this method is that users may be disturbed and may not have the same behavior or experience if he was not observed Emotion sampling device (ESD) Emotional sampling device is a series of questions, yielding to the emotions a user may be experiencing as a result of an event. It is based on cognitive Appraisal Theory (CAT) and Emotional Appraisal System (EAS) [14]. This UX evaluation method concentrates on the causes of emotions rather than emotion itself which as a result help to avoid the typical problems of verbal assessment of emotions [11]. ESD enhances emotion assessment without expensive equipment or hard to understand emotional terms. This is facilitated by the predefined set of questions and the resulting set of emotions which make the method quick and easier. Emotion sampling device may be demanding sometimes by requiring a functional system and a preferable real use cases which consists of questions about level of control and the match motive. 4.4 Oracle Case Study The Oracle application user experience (Application UX) team has developed a User-centered research and design process that include five steps as shown in figure 1 below: 1. Observe 2.Analyze 3.Design 4. Prototype 5. Measure Fig. 1 Oracle five-step user centered design process(gartner 2007b) 14

16 Observe Oracle application team spends time identifying who their end users will be and visit them at their work places. This initial stage takes place before commencement of any designing of the application. Their interview methods include ethnographic studies during which they watch customers in their typical work contexts, interview them as they work and request to see the tools that they use when working. Analyse The application team then searches for the consistencies in work processes across companies and customers. They conduct further studies by using the customer feedback sessions and surveys to gain deeper understanding of the customer and their work. They then create their profiles and persona to characterize the key grouping of the customers. Design Once the application team has a better understanding of the customers, their tasks and the tools, their context of work, they then design the kind of experience they think the customers need. The team is guided by prototype which explores a variety of design solutions that matches the customer mental models of how they work. Prototype The application User experience team then builds the interactive prototype of the Best solutions using the best prototyping tools. Measure Finally after the application is built, a Common Industry Format (CIF) test is conducted to benchmark its usability. The CIF tests conform to the International Organization for Standardization(ISO) standard and also incorporate the Software Usability Measurement Inventory (SUMI), a validated, industry-standard questionnaire for measuring software quality from the end user's point of view. SUMI results are compared to a database of more than 2,000 applications to see how Oracle s applications compare to a state-of-the-market profile. 5 CONCLUSION User Experience is gradually becoming recognized and established as an important part of an organization s business and strategy. This development has impacts for the user experience design practitioners who may include new organizational debates and blurred organizational boundaries. The debates concern responsibilities for the customer experience issue and the 15

17 way UX fits in at different levels within an organization. In essence for an organization to achieve and remain innovative, it needs to embrace User Experience as a guideline considering the different perspectives and thereafter re-evaluate the experience using the user experience evaluation methods. There is a wide variety of user experience evaluation methods deployed by organizations and industry. It is important to note that for the methods to be usefully employed in product development, issues such as resources and skills, ease of use, ease of data analysis, and applicability of results for the development should be considered. This paper provides some of the methods available for practitioners at this point in time. Additionally user experience needs to be much better integrated as a multidisciplinary activity into the key development processes of organizations. Also user experience practitioners need explicit areas of responsibility and to develop effective working relationships with the complementary functions and competences, thereby getting user experience work accepted as a valued part of the overall design and development effort of an organization. In the longer term the emphasis should be on positioning User experience in order to enhance innovation and also secure strategic influence over: The business directions in terms of new value propositions to be developed, The choice of designs to be developed and their contribution to the business objectives of an organization, The development of the processes used to guide the way the organization operates. 16

18 REFERENCES [1] Law, E., Roto, V., Hassenzahl, M., Vermeeren, A. & Kort, J. (2009) Understanding, Coping and Defining User Experience: A Survey Approach. In Proc. Human Factors in Computing Systems, CHI 09. April 4-9, 2009, Boston, MA, USA. [2] ISO (1998) Ergonomic requirements for office work with visual display terminals (VDTs) Part 11: Guidance on Usability. ISO. Available: [ _sept2004.pdf] [Retrieved ] [3] Jordan, P.W. (2002). Designing Pleasurable Products: An Introduction to the New Human Factors. Taylor & Francis. [4] Isomursu M., Väinämö, S., & Kuutti, K.(2007). Experimental evaluation of five methods for collecting emotions in field settings with mobile applications. International Journal of Human-Computer Studies, Volume 65(4), April 2007, pp [5] Hassenzahl, M. (2003). Understanding the relationship between user and product. In M.Blythe, C. Overbeeke, Monk, A. & Wright, P. (Eds.), Funology: From Usability to Enjoyment (pp ). Dordrecht: Kluwer. [6] Kujala, S. & Väänänen K.(2009) Value of Information Systems and Products: Understanding the Users' Perspective and Values,Journal of InformationTechnology Theory and Application (JITTA),9, 4, [7] Stone, D., Jarrett, C., Woodroffe, M., & Minocha, S.(2005) User Interface Design and Evaluation(The Morgan Kaufmann Series in Interactive Technologies). (2005) Morgan Kaufmann. 17

19 [8] Gartner, M., Valdes R. & Gootzit D.(2007b). A Value-Driven, User-Centered Design Process for Web Sites and Applications. [9] Isomursu M., Tähti, M., Väinämö, S., & Kuutti, K.(2007). Experimental evaluation of five methods for collecting emotions in field settings with mobile applications. International Journal of Human-Computer Studies, Volume 65(4), April 2007, pp [10] Kujala, S.,&Väänänen K.(2009) Value of Information Systems and Products: Understanding the Users' Perspective and Values, Journal of Information Technology Theory and Application (JITTA),9, 4, [11] Kaye, J.(2007) Evaluating experience-focused HCI. In CHI '07 Extended Abstracts on Human Factors in Computing Systems CHI '07. (2007) ACM, New York, NY, [12] Mäkelä, A.& Fulton, J.(2001) Supporting Users Creativity: Design to Induce Pleasurable Experiences. Proc. of the Int. Conf. on Affective Human Factors Design, (2001) pp [13] Gaver, B., Dunne, T., & Pacenti, E.(1999) Design: Cultural probes. Interactions, 1 (Jan. 1999), [14] Stone, D., Jarrett, C., Woodroffe, M. & Minocha, S.(2005) User Interface Design and Evaluation(The Morgan Kaufmann Series in Interactive Technologies). (2005) Morgan Kaufmann. [15] Mandryk, R., Inkpen, K., & Calvert, T. (2006). Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour & Information Technology, 25, 2,