Jason Bissict Kent Hawkings Andrew Pettey

Transcription

1 UNIVERSITY OF CAPE TOWN Stroking Not Poking Project Proposal Jason Bissict Kent Hawkings Andrew Pettey

2 Table of Contents 1 Project Description Problem Statement User experience using surface gestures vs. discrete interaction Creating a hybrid interface Creating an effective interface for users solving the application overload on mobile devices Elements required to create an intuitive and assistive gestural interface Procedures and Methods Development Platform System Testing Photo Editor Lock Screen Music Player Ethical, Professional and Legal Issues Related Work Anticipated Outcomes Experiment results System Expected Impact Recognition vs. Recall Creating a gestural interface Surface gesture interaction vs. discrete interaction Resolving the application overload Key Success Factors Project Plan Risks Failure to effectively implement gesture recognition Dependencies between group members Hardware Acquisition Failure to acquire enough participants for user experiments Group conflict Loss of Project Code

3 7.2 Timeline Resources Required Deliverables Milestones Work Allocation References Appendix A

4 1 Project Description Our project aims to explore and analyse the feasibility of creating a mobile interface through which interaction is predominantly gestural. This will be done through the design of three frequently used components of a mobile phone: A music player A lock screen A photo editor All components of the system will be developed on Google s Android platform. In particular, our system will be developed for the devices running Android 4.0 or higher. The term gesture is an extremely broad concept. For the scope of this project we will categorise gesture types based on the two primary input modalities, as suggested by Ruiz, Li and Lank (2011): Surface Gestures refer to gestures done by using one s fingers to make patterns on the surface of a touchscreen display itself. Motion Gestures refer to physical movement made by the user with the phone. The phone then uses it s built in motion sensors (i.e. accelerometer, gyroscope, etc). Through the use of these gestures the user is able to perform a command and elicit a response from the system. This forms a contrast to standard discrete input whereby the user makes discrete poking gestures on the screen that emulates the press of a button. The concepts of motion gestures, surface gestures and discrete interaction will all be used to investigate issues around how gestural interaction affects user experience. In particular we will be investigating the feasibility of creating a predominantly surface gestural interface. 2 Problem Statement Touchscreen devices are becoming ever more ubiquitous as time passes. This is no more evident than in the mobile technology industry. The majority of current smart mobile devices use touchscreen technology as their main source of input. Although touchscreen devices have been in common use for several years, gesture based interaction has not been successful in widespread use (Rico, Crossnan and Brewster, 2011). While gesture based input is prevalent in most devices (i.e. pinch zoom) users still largely rely on discrete poking gestures when interacting with touchscreen devices and are forced to navigate through large menu interfaces to find applications. The term gesture carries no formal definition largely because it is such a broad concept. Saffer (2010) defines a gesture as ``any physical movement that a digital system can sense and respond to without the aid of a traditional pointing device such as a mouse or stylus. A wave, a head nod, a touch, a toe tap, and even a raised eyebrow can be a gesture''. As such this paper will consider different types of gestures one can perform using different input modalities offered by most of today's handheld devices. In order to investigate the feasibility of a predominantly gestural interface we will be addressing the following research problems/questions: 4

5 2.1 User experience using surface gestures vs. discrete interaction This will consist of an investigation into whether surface gestures provide a better means for interacting with a mobile device as opposed to the conventional button based interactions. Conventional discrete interaction was designed originally for physical buttons. As such, it is important to determine whether surface gestures provide more efficient interaction with mobile devices. 2.2 Creating a hybrid interface This question will consist of determining what aspects of a user interface best lend themselves to each type of interaction (i.e. discrete interaction, motion gestural interaction or surface gestural interaction). This will involve the design and evaluation of an interface that is predominantly gesture based through prototyping and user experimentation. In order to investigate this, a music player application will be developed. 2.3 Creating an effective interface for users solving the application overload on mobile devices A study evaluating Icon Arrangement by Smartphone Users (Böhmer and Krüger, 2013) has created discussion around user interaction with applications and icon arrangement. The recent introduction of mobile application stores such as Apple s App Store and the Google Play Store has created an accessible, user focused environment for finding and sharing applications. Users have welcomed this new feature rich environment with open arms; however it has resulted in user s home screen becoming cluttered with endless amounts of applications. As such, finding desired applications can quickly become laborious and time consuming. By creating a surface gesture interaction based navigation system we hope to solve this issue of application overload. 2.4 Elements required to create an intuitive and assistive gestural interface It is important to determine whether a new method of interaction is better than (or at least conforms to) previous interaction standards. A particular problem introduced with gestures is the concept of recognition vs. recall. Discrete interfaces contain obvious hints as to how to interact with the interface. These hints are not inherently apparent in gestural interfaces. This could prevent a usability challenge as users will get frustrated if they keep on having to remember large amounts of gestures. Thus investigation into the adaptation of these conventional interface hints will be required. This means that one will need to test both the recall of gestures (or else find means of suggesting their structure) as well as the discoverability of new gestural techniques. 3 Procedures and Methods In order to effectively investigate the proposed research questions, a number of techniques will be employed for each component of the system. Initial Research Initial research into users expectations regarding gestural interfaces and devices will be required to form a basic understanding of the easiest way to solve each research problem. This will be achieved by performing user experiments wherein participants will be asked to what gestures best represent certain commands relevant to each component of the system. 5

6 Design Phase Each system component will follow an iterative design pattern. Based on the information collected in the initial research phase low-fidelity prototypes of each system will be made. User testing on the low-fidelity prototypes will then be conducted in order to determine the overall efficiency and ease of use of the interface. Hi-fidelity prototypes of each system will then be designed based on the reaction to the lowfidelity prototypes. A second round of user testing will then be conducted to reassess the user interfaces. Implementation Phase After evaluating the design of the interfaces focus shall be shifted to implementation of the interfaces and functionality associated with them. The final product will incorporate all the information collected throughout the initial research and design phase. Evaluation Phase Upon completion of each system final evaluation will be conducted. For the lock screen and the photo editor this will be done in two parts. Firstly, participants will be asked to perform a set of predefined tasks using the different system interfaces. The performance of the user with each interface will be quantitatively analysed. Secondly, interviews will be conducted with the participants to determine details of the system. For the music player evaluation will be carried out in the form of longitudinal studies involving a technology probe. 3.1 Development Platform The system will be implemented on the Android OS. Our development platform will be Java. In particular we will be using the Eclipse IDE as there exists the Android Development Tools (ADT) plugin that provides an integrated environment in which to build Android applications (Android Developer, 2013). 3.2 System Testing Photo Editor In order to determine which input modality is more appropriate for input user testing will be conducted. This will include tests that involve the collection of both qualitative and quantitative data. Quantitative analysis of the interfaces will be conducted by getting participants to complete a set of predefined tasks using each interface. The results of these experiments will be measured using three metric: Error Rate Success Rate Total task completion time Analysis of this data will provide insights as to which input method is more effective at allowing users to achieve tasks in a short time. Post-experiment interviews will be conducted to gather details about the user s experience during the test Lock Screen Testing the effectiveness of the lock screen application will involve a two part evaluation; initially users will be required to use the system for a minimum of a week. After the first week users will have applications mapped to gestures on their phone, from this a qualitative test can be executed, the test will involve a comparison of the time required by both systems (the current standard and the implemented 6

7 system). Finally a qualitative survey will be used to gauge user happiness and design implementation; this evaluation will determine the success of the system as a viable product for users Music Player Final evaluation of the music player will be carried out using longitudinal studies. 5 users will be given the music player to use for about a week. Qualitative data will be gathered by asking each user will be asked to keep a diary in order to document their experience with the hybrid interface. At the end of the week long period interviews will be conducted in order to pursue points of interest within the system. 4 Ethical, Professional and Legal Issues Due to the fact that our overall design approach is user-centered this implies that extensive user testing will be required. In particular, qualitative testing will require 3-5 participants and quantitative analysis will require approximately 20 users as recommended by Jakob Nielsen (Nielsen Norman Group, 2013). As such, we will require ethics clearance from the university for all of the experiments we wish to perform. Moreover, because our experiments will involve UCT students will need to apply to the Executive Director of the Department of Human Affairs for permission to access said students. 5 Related Work A study done by Mauney, et al. (2010) analysed the similarities and differences in user defined surface gestures across different cultures. This was done through an experiment whereby 340 across nine different countries were asked to define 28 common actions using gestural input. Initial analysis of study suggested that there was a large degree of agreement about gestures across the cultures. It was concluded that no gesture was favoured by any country. On the other hand the amount of agreement among non-technical users is somewhat lower as suggested in the study done by Wobbrock, Morris and Wilson (2009). Despite the somewhat disappointing lack of agreement it was noted that similar operations usually elicited similar gestures. In terms of user interface testing arguably the largest figure in this field is Jakob Nielsen. Nielsen (1994) described the four categories of evaluating user interfaces: Automatically Usability measures computed by running a user interface specification through some program Empirically The usability assessed by testing the interface with real users Formally Using the exact models and formulas to calculate usability measures Informally Based on rules of thumb and general skill and experience of the evaluators Nielsen then grouped these four categories based on the results they produce, either qualitative or quantitative. 7

8 6 Anticipated Outcomes As a result of our project we anticipate the following outcomes discussed below: 6.1 Experiment results As this is a research project the main outcome of the project is research results. Upon the completion of the project we expect to have both qualitative and quantitative results from our user experiments. We will deliver a critical analysis of the qualitative results in order to address the strengths and weaknesses of our system. In addition, statistical will be conducted in order to assess whether or not our results are statistically relevant. 6.2 System At the completion of this project we expect to have three separate components that utilise gesture interaction. Each component will be in the form of an application that a user installs onto their android mobile device. We expect to have fully functional music player with a hybrid interface. We also expect two photo editors - one of which will use surface gesture interaction while the other will simply use the conventional discrete interaction. Lastly, we expect a new lock screen interface that utilises surface gesture interaction. 6.3 Expected Impact Gestures (in particular, surface gestures) are usually defined based off of real world concepts (e.g. expanding one s fingers to enlarge an image). This inherent nature of gestures gives them the potential to provide a more intuitive and naturalistic form of user interaction. The expected impact of the results with regard to the proposed research questions are outline below Recognition vs. Recall Arguably the largest challenge with gestural interfaces is the concept of recognition vs. recall. Gestural interfaces do not contain any of the interaction hints inherent in discrete/conventional interfaces. With this in mind, affordances will need to be made for the gestures within all the components of the system. Furthermore all the gestures should be designed to be easily discoverable and memorable. Should an interface be developed which allows for these design considerations, it may be possible to use it to provide guidelines for future gesture-based applications Creating a gestural interface The design and the implementation of a hybrid interface should provide some insight into the appropriateness of both surface gestures and motion gestures within an interface. In particular, it will be interesting to determine what aspects of a user interface best lend themselves to certain types of interaction. Potentially this could provide some motivation for exploiting gestures to a greater degree within mobile interfaces Surface gesture interaction vs. discrete interaction The implementation of the two photo editor interfaces should provide some insight into the appropriateness of surface gestures as a replacement for conventional discrete interaction. Attempting to replace conventional discrete interaction with surface gesture interaction would have a massive impact on interfaces in general, and a larger impact on the way users interact with their mobile devices. If the photo 8

9 editors prove to be better with surface gesture interaction then further research into other applications would be recommended Resolving the application overload The implementation of a lock screen interface for users to input user defined gestures to open applications and apply system wide settings to the device will be used. A separate setup screen will be used to define the gestures for the lock screen, allowing users to add, remove and view gestures. Additionally a help screen will be built into the lock screen to allow users easy access to the gestures from the lock screen if they forget them. Evaluating the security aspect of the lock screen, the help can be disabled, forcing the user to know at least one gesture to unlock the phone. Special focus will be placed on creating an intuitive and efficient interface. 6.4 Key Success Factors The success of our project will be determined by the results of our user experiments. In particular the success hinges on how sound are results are. To this end we will need to design user tests that accurately allow us to gather qualitative data. In turn, the qualitative data will need to be critically analysed in order to successfully determine certain details about our system (such as inherent strengths and weaknesses). Appropriate statistical tests will also need to be carried out in order to determine the statistical significance of any quantitative data gathered. The key success factor connected to the successful implementation of the music player is in creating a music player application with a hybrid interface and determining how gestures affect interaction. The successful implementation of the photo editor is determined by whether gestures could replace conventional discrete interaction or whether gestures should be used simply as shortcuts. To determine this we need to have a fully functional photo editor that makes use of conventional discrete interaction with gestures as shortcuts and a version that only uses gestures. The key success factor connected with the successful implementation of the lock screen interface would be determining whether gestures could simplify a cluttered home screen by providing the user with gesture shortcuts and gesture based navigation through hopefully less menus. To determine this we need a lock screen interface that uses intuitive and memorable gestures mapped to phone actions. 7 Project Plan 7.1 Risks Identified risks, as well as the mitigation strategies are outlined below: Failure to effectively implement gesture recognition In order to create user friendly gestural interface it is important that the system is easily able to correctly recognise users gestures (be they surface or motion gestures). Failure to do so could result in a completely unusable interface, essentially causing our project to fail. Severity of Risk: High Risk Likelihood: Low 9

10 Mitigation strategy: Android provides multiple libraries for gesture recognition. Should these libraries prove to be inefficient proprietary gesture recognition libraries exist; most of which can be purchased at a discount rate for students Dependencies between group members If one member fails to meet a deadline/milestone it could adversely affect the progress of the other members, particularly with regard to the creation of the gestural framework. To this end, certain contingencies need to be put in place in the event that one member is unable to meet a certain deadline for any reason. Severity of Risk: High Risk Likelihood: Low Mitigation strategy: Critical aspects of the project will be compartmentalised so as to reduce the overall impact of one member failing to meet a deadline. However certain aspects of the project will still need to depend on each other. Remaining members must be prepared to absorb aspects of the work. So as not to overload the remaining members, the overall scope of the project may need to be reduced. This could take the form of removing the lost member s component from the project or narrowing the interface types tested Hardware Acquisition We will not be able to properly test and implement our system without appropriate Android mobile devices. Although two of the team members (Kent Hawkings and Andrew Pettey) already have devices running the Android platform out third member, Jason Bissict, will need to acquire one before he is able to fully develop his application. Severity of Risk: Medium Risk Likelihood: Low Mitigation strategy: Samsung Electronics and the UCT have a partnership to develop mobile applications. As such our supervisor Gary Marsden will be able to acquire an Android device for Jason Failure to acquire enough participants for user experiments In the event that not enough users are acquired for our experiments we will not be able to gather sufficient data in order to make our results significant. With particular regard to our statistical analysis; too little participants will not give us a large enough confidence interval in our results. Thus our results will not be statistically significant. Moreover, certain people might be willing to participate in our experiment but then simply not arrive at the experiment. Severity of Risk: High Risk Likelihood: Medium Mitigation strategy: We will recruit more participants than we actually need. As such, if a certain proportion does not show up to the experiment we should still have enough participants to allow our results to be statistically significant. Moreover, in order to ensure that participants show up we can offer 1 An example can be found here: 10

11 compensation for their participation. This could be in the form of paying the user a small amount for their participation or offering snacks/drinks during/after the experiment Group conflict The high stress nature of the honours project could lead to group members ultimately conflicting with each other. In addition, if certain members have different visions of certain aspects of the project this may lead to further conflict of opinions. Severity of Risk: High Risk Likelihood: Medium Mitigation strategy: Regular group meetings will be scheduled in order to ensure that all groups members are fully aware of the overall project progress and the current state of the project. Specific channels of communication will be agreed upon in order to ensure constant and efficient communication between all group members Loss of Project Code Loss of the project code could have disastrous consequences depending on how much was lost. At any rate, it would require one or more group members to completely restart a certain section; setting the project back substantially. Severity of Risk: High Risk Likelihood: Medium Mitigation strategy: All project data could be stored on a cloud server, dramatically reducing the likelihood of data loss. As an extra precaution (for instance, in the case of loss of internet access) project data could be backed up to external storage devices. 7.2 Timeline The overall timeline of the project is presented in the form of a Gantt chart which can be found in Appendix A. 7.3 Resources Required As we are developing our system for the Android platform we will require at least one Android device in order to perform the user testing and the applications. The Eclipse IDE (available for the majority of platforms) in conjunction with the Android Development Tools (ADT) plugin will be used to develop the applications. In order to effectively test our software mobile devices that run on the Android platform will be required. In particular devices running the 4.0 Android Platform or higher will required in order to exploit the full gestural capabilities. Finally, people will be required in order to perform the user testing. 11

12 7.4 Deliverables Upon completion of the project, three fully functional components/applications for Android based devices (running Android 4.0 or higher) should be developed. In addition, quantitative results gathered will be statistically analysed. Milestones Milestone Project Proposal Presentation of Project Proposals by Students Project Web Presence: Proposal, timeline/plan Initial Experiment Completion User gesture experiment Low-Fidelity Prototype Low-Fidelity Prototype Testing Completed Initial Demonstration Presentation Background/Theory Chapter High-Fidelity Prototype High-Fidelity Prototype Testing Completed Design Chapter First Implementation/Experiment/Performance Test + Writeup Final Prototype/Experiment/Performance Test + Writeup Chapters on Implementation and Testing. Final implementation completed, testing completed. Coding complete Outline of complete report Final Complete Draft of Report, Weighting for project marking decided Final demonstration Project Report Final Handin Due Date Friday, 10 May Thursday, 23 May Tuesday, 11 June Friday, 28 June Monday, 1 July Thursday, 4 July Monday, 15 July Monday, 15 July - Friday, 19 July Friday, 19 July Friday, 2 August Friday, 22 August Monday, 26 August Monday, 16 September Wednesday, 25 September Monday, 30 September Monday, 7 October Monday, 21 October Thursday, 24 October Monday, 28 October 12

13 Poster Due Web page Completion Project Demonstrations Reflection Paper Final Project Presentations Thursday, 31 October Monday, 4 November Tuesday, 5 November (at the latest) Friday, 8 November Thursday, 14 November/Friday, 15 November 7.5 Work Allocation The workload allocation is largely reflected in the research questions. The development of the three components developed for the system will be equally divided among each of the group members. The Android platform provides multiple libraries for surface gesture recognition. However a framework will need to be created in order to interpret the specific gestures we will use in our components. This framework will be developed by Jason Bissict. Because Kent Hawkings is the only one using motion gestures in his components of the system he will develop the framework required to interpret movements of the phone and translate them into commands. This will involve manipulating the accelerometer, gyroscope and magnetometer of the device to accurately interpret movements. In order to successfully execute our user experiments, research design will be required. Certain inherent similarities within each of our projects dictate that there will be many similarities within our experimental design. The same consent forms and demographic survey forms will be used across each experiment. Moreover the same overall procedure will be followed in all experiments. Naturally, each the experiments will differ in the respect that participants will be asked to perform will vary due to the differences in each of our projects. To this end, the analysis of results will vary accordingly between each project. As such, each group member will perform their own experimental design with certain aforementioned aspects requiring collaborative work. The end user components of the system are distributed as follows: Kent Hawkings will create the music player. Jason Bissict will create both versions of the photo editor. Andrew Pettey will create the lock screen navigation component. Each member will conduct their own user testing relating to their component. 13

14 8 References Android Developer Site ADT Plugin. [online] Available at: <developer.android.com/tools/sdk/eclipse-adt.html> [Accessed 7 May 2013] Böhmer, M and Krüger, A A Study on Icon Arrangement by Smartphone Users. In CHI 2013: Changing Perspectives. Paris, France. Mauney, D, Howarth, J., Wirtanen, A., Capra, M Cultural Similarities and Differences in User- Defined Gestures for Touchscreen User Interfaces. CHI 10 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, USA. pp Nielsen, J. Usability Inspection Methods In Conference Companion on Human Factors in Computing Systems. ACM, Boston, Massachusetts, USA. Nielsen Norman Group Quantitative Studies: How Many Users to Test? [Online] Available at: < [Accessed 20 May 2013]. Rico, J., Crossnan, A., Brewster, S Gesture-Based Interfaces: Practical Applications of Gestures in Real World Mobile Settings. Whole Body Interaction. London: Springer. pp Rubine, D Combining Gestures and Direct Manipulation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, USA. pp Ruiz, J., Li, Y., Lank, E. User-defined motion gestures for mobile interaction. Proceedings of the 21st SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, USA. pp Saffer, D Introducing Interactive Gestures. Designing Gestural Interfaces. California: O Reilly Media, Inc. pp Wobbrock, J. O., Morris, M. R., Wilson, A. D User-Defined Gestures for Surface Computing. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, USA. pp

15 Appendix A 15