Design of Mobile Interface for Vula Learning Management System

Transcription

1 Honours Project Report Design of Mobile Interface for Vula Learning Management System Tatenda Shumba Supervised by Professor Sonia Berman Category Min Max Chosen 1 Requirements Analysis and Design Theoretical Analysis Experiment Design and Execution System Development and Implementation Results, Findings and Conclusion Aim Formulation and Background Work Quality of Report Writing and Presentation Adherence to Project Proposal and Quality of Deliverables Overall General Project Evaluation Total Marks Department of Computer Science University of Cape Town 2012

2 Abstract This report outlines the steps taken by the researcher in implementing a new mobile interface for access to the Institutional Learning Management System at the University of Cape Town named Vula. As the number of students with internet enabled mobile phones grows there is an increased interest to find meaningful ways to apply them to education. However, the mobile platform presents several challenges which call for careful application of new and traditional user interface design techniques. Utilizing user- centered design approaches the researcher aims to produce an interface that provides useful and effective access to learning tools that students may want to access on mobiles. The design starts by gathering data using interviews and questionnaires to identify which learning tools students would want to interact with from a mobile phone. After the researcher has gained insight on what is to be designed he will use participatory design techniques and interaction design techniques to involve students in the design process as co- designers. The aim is to work closely with users to produce an interface with high usability. The interface produced was evaluated using usability tests that measured the users performance in terms of error rate when performing specially designed tasks the system was intended for. User satisfaction interviews were also used which gave data on the usability of the system. When the results were collected and analysed it was found that the interface was usable and useful but needed more work to be considered as a full system as it only included a minimum of tools. If similar design approaches could be used with the Vula SAKAI mobile access portal more students would use mobile phones to access learning resources. ii

3 Acknowledgements Firstly, I would like to thank my supervisor Professor Sonia Berman for the guidance and support throughout the duration of the project. Mr Stephen Marquard for his helping us understand how Vula works. I would also like to thank my group members George, Sascha and Don. I am grateful to have worked with such a group of enthusiastic people who liked coffee too. I would also like to thank Samsung for providing us with the test phones for developing and testing our project. Thanks to my family who made all the sacrifices for me to be here and for the support that helped me stay here. Anda for being Anda and keeping the ship steady. Last and most importantly I would like to thank God for being there from before I can remember. iii

4 Table of Contents 1. Chapter 1: Introduction Introduction Problem Definition Scope and Objectives Vula Project Aims Success Criteria Project Importance System Overview Chapter Summary Chapter 2: Literature Review Introduction Mobile learning Allure of mobile technology to Learning Mobile Learning Management Systems Moodle Blackboard Desire2Learn ecollege SAKAI Mobile Learning Management System Summary Vula PDA Portal Chapter Summary Design Methodology Introduction User Centered Design Participatory Design Introduction Participatory Design Workshops Why Participatory Design Interaction Design iv

5 3.4.1 Introduction The Approach to Interaction design Why Interaction Design Identifying User Needs and Requirement Who are the users What about the users Requirements Gathering Approach Prototyping Designs Evaluating Designs Chapter Summary Design Implementation Introduction Choosing Design Participants Design Goal First Design Iteration Second Design Iteration Objectives Preliminary Results Summary Third Design Iteration (Interactive prototype) Fourth Design Iteration Chapter Summary Interface Implementation Introduction First Prototype Final Design Interface Evaluation Introduction Test Platform and Environment Test Participants Evaluation Structure and Content Test Procedure Findings and Results v

6 7.1 Task Completion Task Errors Completion Time User satisfaction Feedback Free Response feedback on new interface Chapter Summary Conclusion and Future Work References Appendix 1: Standard Vula Tools Appendix 2: Participatory Design Information Packet Appendix 3: Nielsen s Ten Tsability Heuristics Appendix 4 Interface Usability Testing Appendix 5: Final Interface screenshots vi

7 List of Figures Figure 1: Vula Home Screen... 3 Figure 2: Sample Tools Available for a Vula site... 3 Figure 3: VulaMobi System Overview... 5 Figure 4 Moodle4Iphones two screens showing sample screens... 9 Figure 5: Official Iphone Mobile app course display and gallery function screen Figure 6: Blackboard Mobile enrolled courses page and tool site for a particular course Figure 7: Desire2learn Event details page and course screen Figure 8: ecollege Welcome page and Events page Figure 9: Oxford Mobile showing map functionality(left) and Home page Figure 10: Showing how Vula(right) is mapped to SAKAI mobile portal (left) Figure 11: Interaction Design lifecycle for project Figure 12: Students preferred tools for mobile display Figure 13: Participants collaborating to design screens Figure 14: Paper prototypes showing the login screen, home page and course site shown from left to right Figure 15: Shows change to Home screen to accommodate group member tools Figure 16: Redesigned home screen Figure 17: Showing login screen, Home screen, Courses screen and Course page Figure 18: Box to the right shows the information users wanted to accompany assignments Figure 19: Sample screen showing interface additions Figure 20: Showing Grades displayed on Vula mobile SAKAI(left) and Announcements displayed on Vula mobile SAKAI Figure 21: Final interface running on test phone Figure 22: Number of students that performed errors with either interface on a task List of Tables Table 1: Favourable Characteristics of Mobile devices in M- learning... 8 Table 2: Requirements Gathering Techniques used in this Project vii

8 1. Chapter 1: Introduction 1.1 Introduction Electronic resources and course material have come to play a central role in education. Universities the world over have started to provide course materials and academic support materials on the internet. However, most Universities in developing countries are unable to fully support access to these resources on their campuses[1]. The challenges in providing access can be attributed to the high bandwidth costs and difficulty in attaining space and funds to build expensive computer labs. For instance, at the University of Cape Town computer labs are at times unable to cope with the demand for students needing internet access. Mobile phones are increasingly able to carry media rich content, and allow greater interaction with educational material[2]. This increase in capability, coupled with the high global mobile adoption rate[1] presents an opportunity to solve the problem of limited internet access by using mobile phones to access educational resources. This project aims to develop a new mobile interface for the University of Cape Town s Learning Management System (LMS) named Vula. Learning Management System (LMS) is a broad term used for systems that organize and provide access to online learning materials to students, educators, organizations and administrators. These systems typically support access control, information sharing, communication tools, and administration of user groups[3]. The interface we developed for this project presents a subset of the Vula functionality available on the desktop but geared towards mobile access. This subset was chosen by investigating the way students at the University of Cape Town access or would envision accessing resources on mobile devices. The existing SAKAI mobile interface available is a barebones implementation which ports desktop Vula functionality to a mobile device without giving significant consideration to the way people use mobile devices. It can be argued that the existing interface focuses on functionality as opposed to usability. Studies have shown that users are no longer satisfied with systems that simply work but if long term adoption and frequent use is to be achieved systems need to have high usability and effectiveness in helping users to complete tasks of interest given their context of use[2]. This project will try and produce an interface that takes into account the users context and the peculiarities of mobile phones when accessing information. The design methodology will employ an iterative user centred design methodology in an attempt to create an interface that has high usability and allows users to act as co- designers. 1.2 Problem Definition Vula is the name of the SAKAI[4] LMS currently used at the University of Cape Town(UCT). It is used to distribute course material and spread course information to students and staff and is integral in every course offered at the university. To help increase the availability of the LMS to students a SAKAI mobile access portal allows students and staff to access Vula on mobile devices. This portal is a simple implementation which ports desktop functionality to the mobile device: only the very top level menu is customised for mobile, and as soon as a top- level tool is selected the conventional (desktop) interface is presented on all other devices. This approach however, raises issues of usability and effectiveness as due 1

9 consideration of the limitations and peculiarities of mobile phones was not considered. Issues of usability, user context, and effectiveness need to be taken into consideration when migrating services to mobile devices to ensure adoption and use. Interface designers recognize that to produce interfaces with good usability it is important to have an understanding of the ergonomics, organisational and social factors that determine how people operate[2]. Users want systems with well- designed interfaces and functionality, it is no longer sufficient that a system works but the presentation and function of the system must compel people to want to use it[5]. This project outlines steps taken by the designer in an attempt to produce a mobile interface for accessing the most effective subset of Vula tools on a mobile device. The design will follow methodologies which have been shown to produce systems with high usability and that effectively help support users in performing tasks of interest. 1.3 Scope and Objectives This project explores a new mobile interface for the Vula system at UCT to encourage students to access educational materials and LMS services more frequently from their mobile devices. The ubiquity of mobile phones and their mobility aspect could assist users to perform educational functions more easily and quickly from their mobile devices[6]. The benefits of increased mobile access would not only be limited to saving students time but would also help alleviate the crowding problem in computer labs by allowing users to access materials without needing to enter a computer lab. However, there are features and limitations associated with mobile phones as compared to desktop computers that must be taken into consideration when designing systems for these devices. These features and limitations limit the number of activities users would want to engage in on the mobile phone and also constrain the kind of effective interactions that are possible on such devices. If interfaces to these systems are not well designed no one may want to use the services available to them on the platform. The new interface we aim to develop will focus on the functions and features of Vula that users would most want to interact with on mobile devices and present them to the mobile user in a way that has high usability and encourages the user to access the system on a mobile more frequently. The interface will focus on the display of course related material only. Vula is a system that is used extensively by both students and staff. The project seeks to extend the learning management system to incorporate camera functionality and a student feedback system. These two components were developed by other group members; the focus of this work is on determining the most useful Vula functionalities for mobile use and improving student interaction with those Vula tools on mobile devices. In addition, it had to accommodate the camera and feedback tools and ensure that a single integrated interface resulted. 1.4 Vula Vula is the learning management system used at the University of Cape Town built on top of SAKAI. Vula is used to support UCT courses as well as group and society based activities. The system is intended to be available 24 hours a day seven days a week. The Vula system is flexible and has functions that help foster collaboration and creation of learning opportunities in a networked environment [7]. 2

10 Figure 1 below shows the Vula homepage available after a student logs into the system on a desktop machine. The courses associated with that particular account are displayed as tabs near the top of the window. Figure 1: Vula Home Screen The tools available for a course vary depending on what the course convenor expects students enrolled in the course to do. The possibilities include custom- built tools (e.g. the automated marker in Computer Science) and the standard Sakai tools. There are over 20 tools available some of which are shown in Figure 2, for a full list please see Appendix 1. Figure 2: Sample Tools Available for a Vula site 1.5 Project Aims The main objectives of the study are as follows. i. To investigate how students at UCT would want to interact with Vula on mobile devices and which Vula tools would be the most effective to provide for student interaction on mobile devices ii. To produce a mobile interface that facilitates access to Vula from a smartphone. The first aim involves getting an understanding of which tools students interact with as part of their course experience. By conducting workshops and interviews information on the tools students would most likely want to use was on mobile devices was gathered. In this process we attempt to identify candidate tools and functionality to incorporate in our mobile interface. 3

11 The second aim is to capitalise on the insight gained to produce an effective mobile interface. The effectiveness of this interface will be measured by how well it supports users in performing tasks desired tasks on mobiles. The usability of the system will be measured by how easily users can perform given tasks, measured in terms of time taken, number of errors and number of successfully completed steps. 1.6 Success Criteria The project will be deemed successful if i. Users performance at the given tasks improves when given the new interface compared with the existing Sakai mobile interface, in terms of speed and accuracy. ii. Users react positively to the design of the interface and navigate the system successfully without training. The interface we present does not represent a replacement for the current Vula site. It will only support a portion of the Vula functionality and can be viewed as ideally working in conjunction with existing Vula systems on desktop. The interface is meant to allow users to perform small and frequent tasks quicker than using the SAKAI mobile portal. This can also extend to allowing the users to quickly check course status allowing them to make decisions on what kind of tasks they may need to pursue further in the full desktop environment. The new mobile interface should be able to interact with and display data provided by the existing Vula LMS without any alterations being made to the existing Vula infrastructure. The implementations of the screens supporting the selected tools should take advantage of mobile phone features where applicable and employ strategies and mechanisms to reduce the impact of mobile device inadequacies as compared to desktop computers when considering access to the Vula system. As the interface is being designed for existing users who have already had some exposure to the Vula system on desktop or on mobile devices, the interface must have a clear pattern of navigation that is familiar to current users. 1.7 Project Importance Over 98% of students at UCT own cell phones, of which 85% are smart phones[8]. As courses are moving towards paperless administration more content is being moved to Vula. This puts a greater load on University computer labs as students access various course related information and material from them. If mobile phones could be used more frequently to access some of this information then the load on the desktop resources on campus could be reduced. This would also allow students to perform some tasks faster. The project does not necessarily seek to change the way people learn but to give them a viable and quick alternative to logging on to a desktop machine to get course information. Mobile devices have become more viable ways to access and create digital information. This viability warrants a further investigation into how usability can be increased on these previously specialised platforms. Traditional desktop computer user interface knowledge is not sufficient to design effective and usable interfaces for mobile devices[2]. 4

12 Desktop user interface design approaches must be adapted to the mobile context by addressing peculiar aspects of mobile interaction that distinguish it from interaction with the desktop and design constraints and issues that affect visual mobile user interfaces. If UCT is to benefit from the ubiquity of mobile devices amongst students in terms of education the mobile interfaces that access University materials must have high usability to increase adoption and frequent use. 1.8 System Overview Figure 3: VulaMobi System Overview Figure 3 above shows an architectural diagram of the system we developed. The interface that has been discussed in this document is part of a larger four member project aiming to extend the functionality of the Vula system using mobile devices. This report describes the task of providing a new interface for students to have access to chosen Vula functionality and tools with increased usability on mobile devices. In addition to the chosen existing Vula tools and functionality the interface will also support two new tools that are being developed by fellow group members to take advantage of the opportunities mobile phones present in sharing multimedia and anytime access. These two additional tools were developed independently but presented to the users on one interface. The interface and additional tools interact with Vula through an intermediary server which then forwards the request to the Vula system and returns appropriately formatted data to the mobile interface. This intermediary server was developed by the fourth member of our group. 1.9 Chapter Summary In this chapter the project was introduced and its importance was discussed. The chapter highlights how mobile devices have become ubiquitous the world over and how their increase in functionality has made them viable means of accessing information. The project aims to investigate how mobile phones can be used to support learning activities at the University of Cape Town when applied to access to its LMS named Vula. By designing and implementing a new mobile interface to access Vula which has high usability. The assumption is that 5

13 students will use their mobile phones more frequently to access Vula functions and tools where possible. The next chapter looks at why mobile phones have become a viable technology platform in supporting educational activities. It also looks at how care must be taken in designing interfaces for mobile devices as they have features, functions and limitations that differentiate how people use them from the desktop computer. The chapter will also look at other mobile LMS offerings in use in Educational Institutions around the world. The chapter will end of by a discussion on the current Vula mobile interface that is built on top of SAKAI. 6

14 2. Chapter 2: Literature Review 2.1 Introduction In the previous chapter the project was introduced and reasons for its importance were put forward. This chapter looks at related work that will help motivate some of the approaches followed in the design of the project solution. This chapter touches on mobile learning then gives a summary of why mobile phones are suited for mobile learning. The characteristics that differentiate mobile devices from desktops are discussed and examples of interfaces available for other LMS systems are presented. The chapter will end with a discussion of the current Vula SAKAI mobile interface. 2.2 Mobile learning It is widely believed that innovative applications of Information Communication Technologies (ICTs) can be employed to increase the quality of education by facilitating information sharing and supporting administrative processes [9].Learning management systems (LMS) are one such innovation that has been developed to help achieve the goal of increasing quality of education. Mobile devices have become ubiquitous and cheaper[6] The popularity of mobile devices has made people eager to find a way to apply these portable and personal handhelds for educational purposes[10]. Mobile learning is when mobile computing and electronic learning come together to present access to learning materials at any time and independent of location[11]. As mobile devices have become more powerful they can now be considered as low cost internet enabled computing devices able to support rich interactions with educational materials[10].the large mobile phone user base[6], and presence of an Institutional LMS give UCT an opportunity to increase student access to educational materials and reduce load on existing computer labs. 2.2 Allure of mobile technology to Learning Initially most mobile technologies had a small set of functions acting as a camera, phone, personal digital assistant, et cetera[12]. These functions have now been merged making mobile devices platforms for entertainment and commerce and tools for information management and media consumption. According to Woodill [13], mobiles have the following affordances which make them attractive platforms for education: mobility, ubiquity, accessibility, connectivity, context sensitivity, individuality and creativity. Even though mobile technologies were not designed as an educational device, Institutions continue to try to use them to support institutional learning[12]. There are various favourable characteristics of mobile devices that have motivated the research into the area of mobile learning. Some of these are shown in Table 1 shown below adapted from[6]. 7

15 Table 1: Favourable Characteristics of Mobile devices in M-learning In spite of all the potential and advantages to users of mobile learning and the large number of projects available, mobile learning has not been largely adopted in the long term. An explanation for this has been put forward by Wagner[11], she says the following, complicated key controls and difficult- to- read screen presentations will be tolerated only under certain very limited conditions. The rest of us aren t willing to risk having a bad experience. For broad and long- term adoption, the experience really does matter. To be able to take advantage of these favourable characteristics mobile learning applications need to be designed with high usability to compel stakeholders to use their mobile devices for learning purposes. Good usability means that learning can proceed without obstacles and might even be enhanced by the availability of certain features[14]. The current mobile portal for the Vula system was designed with the goal of making it functional on mobile devices with small screens. This adaptation has overlooked issues of usability which are of great importance to users and stakeholders. The concept of a mobile LMS is to extend the current functionality of the LMS and provide users with a system that allows them to access course information using a mobile device. If the University of Cape Town is to reap the full benefit of mobile devices as a support for learning activities, the usability of the system must be increased and tools that take advantage of the characteristics[15] of the mobile devices must be added. 2.3 Mobile Learning Management Systems There are many mobile learning management systems in use in Universities around the world. We will be focusing on those we found to be most widely used. We will discuss Moodle[16] Sakai[4], Blackboard[17], Desire2Learn[18], and ecollege[19]. 8

16 The discussion will be limited to visual user interfaces available to the user on mobiles and the aims or focuses of these interfaces in terms of the kind of tasks they are meant or support the user in performing successfully. As these University implementations need student credentials to access the full functionality the researcher will only be able to report on features that are made publicly available by the Universities themselves or the particular LMS provider in question. The goal is to get an insight into how other Universities around the world are using mobile devices to display and share educational material and to attempt to get a reference point on how we can proceed to design, develop and implement a new interface for mobile access to the Vula system here at UCT Moodle Moodle is a software package for producing Internet- based courses and web sites. It is a global development project designed to support a social constructionist framework of education. Moodle is provided freely under open source license. Since its creation in 2002 by Martin Dougiamas, the Moodle community has grown to over users who can interact and collaborate on courses[16] Moodle is an open source community based LMS there are a great number of attempts at making a mobile interface for the LMS. In the section that follows I will present the implementations that seemed representative of the display styles or theme shared by all the others. Moodle4iPhones Project This application[20] is an open source community project to create a mobile interface for the Moodle system. The project is still underway and is still being community tested and adapted based on user feedback. The interface is shown below in Figure 4. Figure 4 Moodle4Iphones two screens showing sample screens Official iphone Mobile app This application is a project being run by Moodle HQ to produce a mobile application for the Moodle LMS. This project is still underway and has not yet been released and is still being tested. It still only 9

17 supports the ios platform. The interface is shown below in Figures 5 below. This interface used icons and inly shows a subset of all the tools based on what the provider has found to be more useful to the user in the mobile context. Figure 5: Official Iphone Mobile app course display and gallery function screen Blackboard Blackboard is an enterprise company mainly dealing in the LMS field for both education and other applications. Since its creation in 1997 by Michael Chasen and Matthew Pittinsky, the number of organisations using the system has grown to over 9300 institutions in over 60 countries. The Figure 6 below shows the interface for blackboard. Figure 6: Blackboard Mobile enrolled courses page and tool site for a particular course Desire2Learn Desire2learn is an enterprise company involved in e- learning dealing in the LMS field for education. Since its creation in 1999 by John Baker, the number of institutions using the system has grown to over 450[18]. The interface is shown in Figure 7 below. 10

18 Figure 7: Desire2learn Event details page and course screen ecollege ecollege[19] is an on- demand, provider of elearning software and to learning institutions. Founded in 1996, It is used in prominent Universities in the USA. Figure 8 show a sample of the interface. Figure 8: ecollege Welcome page and Events page SAKAI In the early 2000s several major institutions in the USA most prominent amongst these being the University of Michigan, Indiana University, MIT, Stanford, the uportal Consortium chose to release their LMS under open source rather than commercial licenses. These were where gathered together to form the SAKAI Project[4]. There are now more than 350 organisations and Institutions worldwide using SAKAI to support teaching, learning research and collaboration. Since its formation the SAKAI Project has been managed by the SAKAI foundation. The SAKAI foundation is in place to facilitate the success of the community and the software. This is done by coordinating software development, quality assurance and distribution activities for the community. Sakai s goal is to develop and distribute a complete course management system that incorporates the best features of the participants existing systems and experiences. Oxford Mobile Oxford University[21] has released a new interface that works on top of the SAKAI platform. Their interface tries to bring all the tools that the user would want to use closer to the user. The implementation is totally different from the default SAKAI mobile portal. The information is 11

19 represented by icons that give affordances to their function. Figure 9 contains pictures of the interface presented to users. Figure 9: Oxford Mobile showing map functionality(left) and Home page Mobile Learning Management System Summary Various institutions have shown great interest in adapting their learning Management Systems to work better on mobile devices. The SAKAI community[4] that UCT is a part of is aware of the importance of mobile devices in education. This awareness started the Mobile SAKAI Collaboration Project[4] which is actively working to expand upon current, limited mobile functionality and to develop frameworks usable by institutions to integrate Sakai into their own mobile initiatives. All these recent developments show the importance of mobile support and in the case of the SAKAI based systems can act as points of reference in designing the UCT mobile Vula experience. Oxford mobile[21] which was launched recently is built on top of the SAKAI architecture and extends the functionality of the underlying LMS by adding support for more functions for example bus schedules, library searches webcams, et cetera. This move by Oxford shows that the Vula mobile portal that comes packaged with SAKAI may not be able to fully utilize the power of available mobile technology and this may warrant an investigation into steps that can be taken to improve and extend the functionality of the Vula mobile interface available here at school. The researcher did note that most mobile LMS adaptations made added support for cameras(upload function with iphone Moodle app) and location based information (with Oxford mobile). The layout of the tools was either in lists or using icons or a combination of both. It was also noticed that most of the course related tools were a subset of the full system tools available. The researcher will aim to make use of these observations as a starting point when pursuing the design for this project. 2.4 Vula PDA Portal Vula is UCT's online collaboration and learning environment, used to support UCT courses as well as other UCT- related groups and communities[22]. Vula is built on the SAKAI framework and is part of the SAKAI project. Vula mobile access is made possible through the SAKAI mobile access portal which is part of the SAKAI framework. It was first incorporated into SAKAI at version 2.4. This PDA portal has since been 12

20 improved but still works on the same premise of porting as much of the full desktop functionality as possible onto the mobile device. In the next few sections we will explain how the SAKAI mobile access portal is structured. The goal is to attain an understanding of the limitations and constraints associated with mobile technology at the time when the first mobile access portal for the SAKAI service was first proposed. This may help explain why the interface presents information in the way it currently does Sakai Mobile Support When the Sakai project began the Learning Management System produced from this collaboration was only accessible from desktop computers. As mobile devices became more prevalent an initiative was begun to start offering support for mobile devices. This initiative developed into what is now the SAKAI mobile access portal. At the time of development mobile market was predominantly PDAs and PocketPCs that were limited in functionality and processing power. The challenges associated with displaying learning management systems on the mobile devices were as follows [23]: Limited screen size: The 240x320 display presented a challenge for displaying static web elements without resizing. Users were then required to do a lot of scrolling to the view a page. Limited functionality: Mobile devices did not support Cascading Style Sheets[24] and iframe tags[25], which are commonly used in LMS technology. If a tool was displayed using an iframe tag, it could not be viewed on the mobile devices which also only had limited support for Javascript[26]. Interoperation between desktop web browsers and mobile browser: Web applications written for mobile devices were not fully functional on desktop machines and vice versa. The user needed to remember two different URLs to access two different versions of the web portal SAKAI Solution for Displaying Web Portal on PDA This mobile Sakai access portal ports desktop functionality to the mobile device by flattening iframes and presenting only their titles in a list on mobile devices. Figure 10 below shows a sample course on Desktop (right) being mapped to the mobile portal (left). As soon as a user selects an option on the mobile version they are redirected to the same version of the option as would appear if accessed on the desktop Vula. 13

21 Figure 10: Showing how Vula(right) is mapped to SAKAI mobile portal (left) 2.6 Chapter Summary In this chapter we talked about the allure of the use of mobile devices to increase access to educational materials. We also touched on how system usability plays a big role in whether or not a system is widely adopted. Factors that differentiate desktop interface design from mobile interface design were investigated. Mobile learning applications currently in use around the world were discussed. The chapter ended with a look at how the currently available mobile portal for SAKAI was developed and the limitations at the time that constrained the interface designers 14

22 3. Design Methodology 3.1 Introduction In the previous chapter we looked at factors that make mobile phones potentially good candidates for academic applications. We looked at how usability plays an important role in the adoption of electronic systems. An investigation into related work on mobile LMS was conducted and the chapter was concluded by a discussion on the construction of the first version of the SAKAI PDA portal. This chapter introduces the design methodologies that will be used with the aim of producing a mobile interface with high usability. We will be using two user centered design approaches and will attempt to motivate why two approaches were chosen for this project. The researcher would like to note that all work with users undertaken in this project was conducted with full signed consent forms and ethical clearance for the UCT Ethics committee. 3.2 User Centered Design User centered design is a broad set of design processes that involve end- users directly in the design process of products to influence how the design takes shape[23]. Users can be involved to differing degrees depending on the design process chosen but the important aspect is that end- users are involved. These processes place the user at the center of design. The designer then becomes a facilitator to the design process making sure that the user can make use of the product as intended and with minimum effort to learn how to use it[27] This approach was chosen for this project as it has been found to lead to more effective, efficient and safer products and contributed to the acceptance and success of products[28]. In the design of this project we will be using two User Centered Design approaches namely, participatory design and interaction design. These will be explained in the following section and the need for both approaches in this project will be substantiated. 3.3 Participatory Design Introduction Participatory design enables designers to learn user requirements and preferences early in the design process through a facilitated group design workshop. This methodology attempts to take advantage of the end- users collective generativity, when applied to design problems they may feel are associated with systems they use[28]. This approach is seen to provide more innovative solutions compared to relying solely on the designers creativity when solving design problems. This design methodology when applied to this project, implies that students as the end- users of the interface should have a say in the decisions that affect the way it is used. This allows the researcher to get an insight on unmet needs by allowing users to create their own ideal interfaces. By analysing the interfaces designed by users the researcher aims to elicit the tacit needs of the users. 15

23 Participatory design is used to design early prototypes which feed into overall project design. Participatory design workshops are most effective early in the design process when designs are less constrained by existing code or other infrastructure Participatory Design Workshops In participatory design workshops, users who represent the stakeholders in a product collaborate to come up with conceptual designs that reflect user preferences and user requirements[28]. These users need to be domain experts, for this project we can view our domain experts as students who have used the mobile Vula interface at UCT. These conceptual designs are created using generative tools. Generative tools are common materials like coloured pens, sticky notes, scissors, tape, white boards and other common types of low- tech objects[29]. These tools are kept simple to encourage communication and creativity. No participant feels intimidated by unfamiliar technology and this allows technical and non- technical participants to contribute equally. The low- tech objects also allow user designs to be created and edited quickly and easily. The workshops are conducted by a facilitator who in this project will be the researcher. The goal of the facilitator is to stimulate discussion and support dialogue around a list of topics whilst also accommodating discussion of impromptu related topics. Participants are asked to brainstorm ideas individually or in small groups then present these ideas to the whole group. These ideas are discussed, evaluated and possibly refined by the larger group. The researcher moderates design presentations and asks follow- up questions and notes key points raised and user reactions to designs put forward. The workshop produces one or more paper prototypes of a user interfaces and textual descriptions of interface elements and their behaviour. These may reveal new thoughts about application requirements or put forward unique solutions to design problems the users have been exposed to. Photographs of key stages in design can also be kept Why Participatory Design As discussed in the previous chapter Vula already has a mobile access system set in place. The researcher felt that any exploration into the design of a new interface would need to learn from the experiences users had already encountered with the existing system. This made participatory design an important choice as a starting point in the design as we would be able to evaluate how well the existing interface supported user desired tasks in a group setting. This evaluation of the existing interface will be done using conceptual model extraction. Conceptual model extraction works by showing users static images of a system and asking them how they would use screen elements to complete particular tasks[28]. This shows how well the current system performs and where design problems still hinder the usability with users. This may help reveal more user requirements which would not be apparent. The collaborative nature of this approach also provides the researcher with an opportunity to get information on context of use by allowing users to cooperatively design user- stories[28] for how they interact with their mobiles when trying to perform tasks of interest. This is desirable as it can give the researcher an approximation of the users context and design requirements without running ethnographic studies[29] which can be long and costly. Participatory design has been found to produce innovative solutions to design problems[28]. The use of participatory design can give the researcher a reasonable first approximation of what the users 16

24 needs and preferences may be when interacting with a system geared towards helping them perform specific tasks. 3.4 Interaction Design Introduction Interaction design according to Preece[28], focuses on how to identify users needs and the context of their activities, and from this understanding move to designing usable, useful, and pleasurable interactive products. Interaction seeks to address the question of how to optimize the users interactions with a product so that they match the users activities that they would want supported[28]. Interaction design focuses on making design decisions that help the system meet the users requirements and preferences on how certain tasks should be accomplished. This approach can be directed at aesthetic aspects of the system (is it pleasant to look at) or usability criteria like how efficiently it allows users to perform certain tasks of interest The Approach to Interaction design In this project we chose to use iterative interaction design using feedback from users to make incremental improvements in the system. Users are involved throughout the design process to ensure that the interface meets their requirements and meets their expectations when performing desired tasks. Effective interaction design can be separated into three activities[29]. 1. Understanding users (requirements gathering) : having a sense of users capabilities, limitations, and how they interact with systems in different contexts 2. Developing prototype designs (interactive prototypes): presenting an interaction design in such a way that it can be discussed, demonstrated and changed 3. Evaluation (iterative evaluation of interactive prototypes): assess the strengths and weaknesses of designs with the results driving improvement or total redesign if necessary. The iterative nature of our approach makes the three activities shown above complement each other. This allows interface designs to adapt along with the researcher s understanding of users, user experiences with prototypes and system strengths and weaknesses after evaluation. This means that outputs of one activity drive the next activity in a cycle until the final design is achieved. 17

25 Figure 11 below shows our development lifecycle with different activities looping until the final design is reached. Figure 11: Interaction Design lifecycle for project Why Interaction Design Participatory design which was chosen as the starting point of the design phase of this project is a good methodology for adding innovation to design problem solutions and is a good method for initially gathering user requirements and preferences. This method however, does not directly address the issue of usability of the system. Interaction design has been found to produce systems with high usability as it not only looks at the tasks users would want to perform but how these tasks can be affected by the users context. Interaction design is aimed towards allowing interfaces that give users certain experiences and aesthetic values when interacting with the interface. 3.5 Identifying User Needs and Requirement The process of understanding users for the purposes of designing an interactive system to support their activities is closely bound to the objectives the system is meant to achieve. These objectives are determined by researcher before work with users begins to allow the interaction design process to be focused. These objectives can be classified into two categories: usability goals and experience goals. Usability goals are concerned with achieving usability criteria; experience goals are concerned with providing the user with a pleasing aesthetic experience [28] Who are the users As we are using user- centered methodologies to approach the design of the new interface it is of great importance to specify who the users our interface is meant to cater for are. In relation to any interactive system we can identify three types of users: primary, secondary, and tertiary. Primary 18

26 users are those who use the system; secondary users are those who will use the system occasionally or through an intermediary; and tertiary users are people who will be affected by use of the system. For our design purposes we will be working with students who are primary users and they will be acting as domain specialists throughout the entire design process What about the users In this section we will be discussing the methods that the researcher will use to get information from the users that will be participating in the design process of the mobile interface. Data gathering is an important part of user centered design as it give a basis to elicit information about the kind of experiences and usability goals our interfaces should attempt to provide to end users. The Table 2 below adapted from Preece, et al[28], shows all the requirements gathering techniques that we will be using in reference to this project. Table 2: Requirements Gathering Techniques used in this Project Technique Purpose Stage of Design Cycle Background Interviews and Collecting data on user At the beginning of design questionnaires expectations, requirements and preferences. Participatory workshops Gaining an impression of the Early in design cycle challenges users are currently faced with when performing tasks of interest. Getting an early approximation of user requirements and preferences. Focus group Stake- holders discuss issues and requirements Early in design cycle The researcher will make use of a combination of the requirements gathering techniques presented in table 2. There are over 18 tools available to the Vula system (see Appendix 1 for full listing). It is worth noting that in most cases not all these tools are available to students as the tools included in courses are mostly decided by the lecturers who own the course sites and decide which tools to include The researcher aims to conduct activities with the participants that achieve the following: Identify which Vula tools and functionality students used most frequently during the running of a course. Identify which tools users would want to use in a mobile context? Identify which aspects of the current mobile Vula portal students have problems with? (where applicable) Allow users to propose possible features and functions that would that would be useful in the mobile context? Allow users to propose new ways of displaying data, and which related data could be displayed together (proposing new data views) Assess users experience of the currently available Vula mobile portal where the prior experience exists. 19

27 3.5.3 Requirements Gathering Approach As described previously the researcher chose to use two user centered approaches to designing the user interface namely participatory design and interaction design. Both of these approaches give the user an impression of what the users requirements and expectations for a usable interface are but provide this information in different ways. Participatory design as it will be applied to this project will have the participants taking part in workshops with the researcher as a facilitator performing a conceptual model extraction[28] on the screens depicting the existing Vula user interface. This will allow the participants to articulate the kind of tasks they feel the interface allows them to perform more easily. This discussion will also prompt participants to identify the tools on the desktop Vula they were most interested in using on the mobile device and how well the existing Vula mobile interface had helped them perform these tasks. The participants will then be given an opportunity to collaboratively design an interface that would support their needs ideally. For the interaction design sessions the researcher will use background interviews and questionnaires with the users to determine which Vula tools they use most on their mobile phone. This will be done by asking the users which tools they are familiar with and or have encountered during any course work. We will then ask students to identify which tools they would most like to see included in a mobile interface if they were to build on from the ground up. To get information on user needs we would ask the users to give reasons for why they would want to include the courses they had mentioned. Thereafter the users will be asked to evaluate the interface produced from the participatory design session. The feedback from this evaluation will shape the design of the next interface design which may be re- evaluated and the process repeated as needed until a set number of iterations has been completed or the researcher and users are satisfied with the design. When attempting to evaluate the tools identified by the users for implementation in the new interface we will consider the following: 1. How widely they were used 2. How relevant they were for mobile use 3. Whether they were implementable within the timeframe of the project The tools will be divided into four categories: 1. Essential: must be implemented in our new interface as they are key to the user experience and are relatively straight forward to implement 2. Desirable: can be implemented as extras but may require a bit more extra work than the essential tools 3. Initially unnecessary: superficial additions which may be useful but do not necessarily help our interface meet its desired goals 4. Unnecessary or unfeasible: would not make sense in respect to the goals of our project and the time frame in place, these could unfeasible because of technology constraints. We will not concentrate on tools with high amounts of user input as using the keypads and miniature keyboards can be time consuming and have a high error rate. In other words we would not be expecting the user to write and submit an assignment on their mobile device, therefore would not support such a feature with our interface. 20

28 We will concentrate on interfaces to tools that allow users to perform single discrete tasks (e.g. checking a course grade). This will allow us to avoid complicated tasks requiring extensive navigation and preservation of state information (e.g. changing password), which given unreliable mobile device network connections could leave the system in an unknown state. This problem also extends to file upload where network issues may cause sent files to be corrupted without the sender knowing (e.g. a student s assignment submission being corrupted). However downloading files is a different case if a file were corrupted it would simply not display properly and the user would be made aware of the problem as soon as they tried to access the file. 3.6 Prototyping Designs Prototyping presents a way of turning ideas into physical objects that people can see and interact with. Designers need to produce prototypes of their designs to open up their designs to critique and help them evaluate how well they understand the needs of the users. A prototype lets you express a design idea as quickly as possible[29]. In this project the researcher made use of both low fidelity and high fidelity prototypes at different phases of design [29]. The main purpose of prototypes in the design process is to have physical representations of systems that can be changed quickly cheaply and are easy to create. This allows better group design as it is easier to collaborate and allows more ideas to be tried in quick time. In the early stages the designs will be expressed using low fidelity prototypes using low tech materials like pen and paper drawings that are evaluated by users to become further refined or even in some cases redone. As these are low fidelity prototypes throwing out designs or editing them is not a costly process. As the process of refinement and redesign progresses on the low fidelity prototypes they move closer to being high fidelity prototypes which could represent the final system. User interface design is a visual process which benefits greatly from ideas being written out on paper. As the number of design iterations increases storyboarding [28] will be used to get the users to design sequences of screens for performing particular tasks. If storyboarding is successful, a high fidelity prototype is built from the refined low fidelity prototypes. This gives the users a chance to interact with a system that may show what the final system might look like. Interacting with the system in this form could still cause design changes. 3.7 Evaluating Designs Evaluation is the process of determining how well a design meets users needs[28]. In the course of the research we will be using three evaluation methods to assess how well our designs meet requirements namely quick and dirty, usability testing and heuristic evaluation. Quick and Dirty: as we were working with designing a visual interface there was a high chance that many alternate designs will be designed by participants in the workshops and focus groups. Quick and dirty evaluation provides the researcher with an option to evaluate early low fidelity prototypes with users and discard designs that fail to meet the user requirements early on in the design life cycle before costly high fidelity prototypes are attempted[29]. This method is concerned on giving the designer a quick impression of how the design decisions made have been received by designers. Usability Testing: involves measuring user performance over a set of carefully designed tasks that are representative of what the system was designed for[28]. The users performance is measured in 21

29 terms of error rate and time to complete tasks. The facilitator takes notes of how the user goes about completing these tasks and notes any errors made. This can then be followed up by user satisfaction interviews. Heuristic evaluation: is a quick inspection method that helps identify usability issues in a mobile interface[28]. It involves 3 to 4 evaluators looking at the screens associated with an interface and checking them against a set of usability guidelines. In this project we will be using Nielsen s 10 Usability Heuristics(see Appendix 3) 3.8 Chapter Summary This chapter looked at the design methodologies the researcher used when coming up with the interface design for this project. The methodologies were outlined without considerable detail being put forward on the specifics of how these designs were going to be carried out. In the next chapter the way these methodologies were applied in the context of our interface design will be outlined 22

30 4. Design Implementation 4.1 Introduction This chapter looks at documenting the implementation of design methodologies outlined in the previous chapter. The researcher started the design process with requirements gathering then moved into co- design with primary users of the Vula SAKAI mobile access interface. After adequate initial data had been collected and analysed the researcher then produced one high fidelity prototype. This high fidelity prototype was evaluated and changes to the design were made to reflect the feedback from participants. The changes to the designs were documented as low fidelity prototypes. These prototypes then underwent a heuristic evaluation and the resulting paper prototypes were considered to represent the final design. The researcher would like to note that all work with users undertaken in this project was conducted with full signed consent forms and ethical clearance for the UCT Ethics committee. 4.2 Choosing Design Participants For the purposes of these user centered design activities it was required that the participants be primary users of the existing Vula SAKAI mobile interface. This would make them domain experts and give them experience with the system which would make them good sources of knowledge on design problems by virtue of prolonged use. For this study we considered domain experts to be students in at least their second year of study at UCT who use the mobile SAKAI access portal. UCT has many Departments and the way they go about presenting course material using Vula differs, for instance Economics Department can make extensive use of tests and quizzes whilst Computer Science seldom uses it. These differences in what the Vula systems provide to students can cause a difference in what the user expectations are for the system. Given the time constraints associated with the project a comprehensive study into all students could not be undertaken so the users were constrained to being Science students and at a finer granularity Computer Science students where possible. Each activity used four to six students unless a different number is specifically noted in that activity. Participants were found in Computer labs in the Computer Science build and through posters placed in the Computer Science building. 4.3 Design Goal The goals of the interface are to provide mobile users of the Vula system access to tools that provide information and functionality that make sense in the mobile context. This will require identifying a subset of Vula tools that users would most likely want to use on mobile devices. A reorganisation of the presentation of this subset of tools and the information they deliver will be explored to enhance usability and to try and address some of the peculiarities presented by mobile phones as compared to desktop computers. The researcher concentrated on discovering which information students wanted to access most frequently from the Vula services on a mobile. To increase usability we aimed to produce an effective useful interface with a small number of tools done well and we wanted to incorporate 23

31 features of mobile phones where possible to enrich the experience (the addition of the Gallery and Feedback tool being developed by other group members). 4.4 First Design Iteration Objectives 1. To get a list of the tools the participants as end- users would want to use most on the mobile phone. 2. To get a first approximation of the design problems the participants associate with the existing Vula Mobile interface through a collaborative discussion and conceptual model extraction on a sample storyboard accomplished using the SAKAI mobile interface. 3. To get the participants to design interfaces supporting the access to tools they had previously identified as being most useful on mobile devices 4.4.2Preliminary Prior the day of the workshop participants were asked to consider the following aspects of the SAKAI mobile access portal: I. What they liked about using the SAKAI mobile access portal? II. What they did not like about using the SAKAI mobile access portal? A short statement of the workshops purpose was indicated allowing the participants to think of issues relating to the usability of the SAKAI mobile portal ahead of the actual workshop if they so wished. The researcher secured a room from which the workshop could be run. Care was taken to ensure that the room had a chalkboard, whiteboards and sufficient table space. This was done to allow the participants to have adequate writing materials to express their design ideas. The choice was made to have the participants make initial designs on the chalkboards and whiteboards as this made it easier for them to collaborate as opposed to writing on small pieces of paper. The researcher was then tasked with transferring these designs to paper prototypes. On the day of the test the researcher who acted in the role of facilitator used the following steps to run the workshop: 1. Welcome: The facilitator welcomed participants and thanked them for being part of the study. The participants introduced themselves. Information packets (see Appendix2) relating to the workshop were passed out to each participant. 2. Introduction: The purpose of the study was explained and this was also included in the information packet and participants were encouraged to read it. 3. Agenda: The agenda will be presented to the users with the outline of the tasks being given to give the users an impression of what they will be doing without giving out information on that would affect the outcomes of any of the tasks. 4. Consent form: In view of the provided information participants who are no longer willing to take part in the study are given an opportunity to leave and the facilitator informs the 24

32 participants that they will be allowed to leave at any stage. Consenting participants are then required to sign the consent form. 5. Task Based Questions: The tasks were then conducted Results Objective 1 The results for the tasks in Appendix 2 as performed by the users in the workshop are presented in this section. The aim of this task was to identify which tools the users would want to make use of on mobile devices. Though not indicated here the researcher noted that when students were asked they consistently wanted to use fewer tools from the Vula system on mobile as compared to the Desktop. Figure 12: Students preferred tools for mobile display The results from the four users present in the workshop for this objective were combined with results from eleven other Science students who gave feedback in impromptu interviews using the tool familiarity section only from Appendix 2. This was done to allow the researcher to be able to see the points of view of a wider group of students than just the four who were at the workshop (the description of each tools function can be found in Appendix 1). The results of these surveys are presented in Figure 12. Objective 2 This objective was not meant to assess the entire SAKAI mobile access portal but just to get an impression of what the participants reactions to it were. As a result, this analysis did not go into detail. The screens that were shown represented the screen transitions from when the first screen of SAKAI portal is accessed to when the course tools for a course are presented. The course screen to be presented in this sequence was picked arbitrarily and is shown in Appendix 2 in the Sakai Mobile Portal section. 25

33 Users were asked to describe the screens of the SAKAI mobile access portal shown in Appendix 2. The users were to put particular emphasis on things they liked or did not like and what they thought the functions displayed did. Some of the comments are shown below ordered by screen title: Mobile Access Point: Most users felt that this screen was unnecessary as the functions it accessed apart from the login option were seldom used. A participant is quoted here commenting to that effect Please put text boxes for quick login. This type of feeling was mirrored by other participants who thought this screen could just be replaced with the login screen. Participants did however mention attempts at using the Reset Password option and commenting on how it did not work. Login screen: The participants thought this screen was well designed. Many thought that this screen should be the first screen the user meets when attempting to access Vula on mobile. Home screen: The participants thought this screen was also well designed. As it showed the courses that the user could access. Course screen: The participants thought this screen was well designed but they did note that it took time to access the tools they wanted as there were quite a number of tools on display and this necessitated scrolling down the page to find desired tools. They felt that some of the tools displayed could be classified as noise as they were tools they would probably never interact with but still needed to pass through to get to what they wanted. Objective 3 Participants were then tasked with performing a ground up design of the screens that would access the tools that they decided were most important. As far as was possible the participants were asked to collaborate and motivate the inclusion of each screen element and the layout chosen. The results of this task where put together and the researcher drew up low fidelity prototypes based on what the participants had designed on the white board. Figure 13 below shows participants collaborating to design screens as part of this task. Figure 13: Participants collaborating to design screens The section that follows documents the screens that were designed by the focus group the rationale behind their layout and their purpose. Figure 14 shows all the prototypes obtained. These prototypes will be explained in the following section in the order that they appear in Figure

34 Figure 14: Paper prototypes showing the login screen, home page and course site shown from left to right Login screen: This was intended to be the first screen the user encountered when using the system. This screen asked for the user details that would grant a user the access to the Vula system. The prototype of this screen is shown in Figure 14. The page was stripped of all tools that are not directly related to the task of logging in. The system was reduced to username field, password field, a login button and an option to change password. Home page: This screen is reached after user successfully logs in. It gives a broad overview of the functions that are available to the user. The screen designed is shown in Figure 14. The page shows all the broad definitions of the functions that the interface will be able to support. The page shows list elements representing the following functions: announcements, grades and a list of all available courses below that. Announcements: This was added by the participants as they felt it would be a good function to be able to see an aggregated view of the announcements across all courses on one list. The goal of this was to allow the user to have an overview of all course announcements in one location. Grades: This was added by participants as they felt it would help to have an aggregated view of all the grades the user had obtained across all courses. Course name: Selecting a course name for instance MAM1000 would expose all the tools of interest available for that course. In Figure 14 MAM1000 is shown with the Announcement and Grades tools available for selection Summary In the first iteration of the design process the designer sought to get an understanding of the kind of tools students would like to use on the mobile device. The most popular tools for considered for addition in our interface were found to be Announcements, Grade Book and Resources. After tasking the user with evaluating the SAKAI mobile access portal the researcher managed to get insight on the kind of experiences did not like, for example the researcher learnt that users do not enjoy looking through long lists of tools on mobile devices. 27

35 The participants were then asked to design an outline of the kind of screen interactions they would expect from an ideal system. These designs were carried out collaboratively on the white board and the results were transferred to paper prototypes by the researcher. 4.5 Second Design Iteration Objectives 1. To decide which tools to focus on for the new interface. 2. To decide on how to integrate the two extensions being developed by group members into the interface. 3. To decide an approach for the first high fidelity prototype using real data Preliminary In addition to the feedback from the participants from the First design iteration, on what users would want to use on mobile devices the designer continued questioning students from the Science faculty about what they would want to do on mobiles using quick impromptu interviews. The results of these shorter interviews closely matched the results of the first. So the researchers continued with the insights gained from the first design iteration This design iteration was limited to only the group members. The three other team members needed to coordinate efforts with the backend designer to successfully implement a horizontal prototype using real data Results Objective 1 The researcher met with the backend designer to try and assign priorities to the tools that the interface would need to support. These priorities were assigned by how important the users thought these tools were and how difficult the backend designer thought they would be to implement on the intermediary server. The results are structured as was discussed in section of this report and are stated in the following section (the description of each tools function can be found in Appendix 1). Essential: Announcements and Grade Book Desired : Resource and Assignments Initially Unnecessary: Chat Room Unnecessary : Calendar Objective 2 As previously stated the interface being produced should incorporate two additional tools being developed by group members. The researcher purposefully avoided adding them to the first iteration to avoid confusing participants of the first iteration. 28

36 The team agreed that the other tools should be available to the user after login to the system. This would affect the home screen previously designed by users. A paper prototype showing the change from the initial design is shown below Figure 15. Figure 15: Shows change to Home screen to accommodate group member tools Looking at the screen on the left of Figure 15 and comparing it to the one on the left its visible that two elements have been added namely Gallery and Feedback which represent the two additional tools being developed by other group members. Selecting either one of these options would access the functionality related to that tool. The Grades item was removed as the researcher wanted to keep the number of options on the screen kept to a minimum this was based on earlier insights on how users had commented on having too many options available on the screen. The Grades option was removed instead of Announcements as the users had indicated it as being less important than announcements. Also adding to this was the assumption that more new announcements were made over a period as compared to new course grades being posted and from the researchers own experience most grades postings were accompanied by announcement indicating their availability so users could get notification on grades from announcements. Another change is that courses are no longer listed on the home page but have all been replaced by an option Active Sites, selecting this option displays all the courses available in a list on a new screen. From this screen selecting a course behaves much as before. Fig 16 shows how this new active sites option would be used to navigate to a course screen. 29

37 Figure 16: Redesigned home screen This was done to reduce clutter and the amount of scrolling a user would have to do find a desired option. Objective 3 To allow users to get a feel for how the system would work with real data the team needed to agree on a prototyping plan. The consensus was that the two components that would be included in the interface would need to be developed using the same language and libraries to allow for a uniform look and feel to the application. The team agreed to develop the interface and tools using HTML5[33], CSS[24], Javascript[26] and jquerymobile[30]. The format for requests to the intermediary server and the responses from the intermediary server also had to be decided and the team settled for regular HTTP[31] requests and responses using GET and POST methods Summary This design iteration made changes to the interface that the users had designed in the first iteration to allow the interface to accommodate the two additional tools designed by group members. Group members also agreed upon the development languages for the first high fidelity prototype namely HTML5, CSS, Javascript, HTTP and jquerymobile. Justifications for these choices are given in Chapter 5 Interface Implementation. 30

38 4.7 Third Design Iteration (Interactive prototype) Objectives 1. To allow users to get a feel for how the system performs with real data by interacting with a high fidelity prototype and to allow users to suggest changes after interacting with the prototype. 2. To receive general feedback on the system 4.7.2Preliminary This prototype was to be viewed as a vertical prototype to give the user a feel of the screen transitions associated with the system and the kinds of navigation that the user would go through to complete certain tasks. Users were told that this was not a representation of the final system but a step in the design process and that their input was of great importance and if they noted any features or lack thereof that made the experience less fluid and or unusable they should bring it to the attention of the researcher as they went along with the interaction design session. For the purposes of the test a dummy Vula account with fictional courses was created to allow the user to explore the system after being given a set of access credentials. At this early stage the gallery and feedback tool were only added for aesthetic value and had no functionality yet Results For this report the researcher will go through each of the screens designed and present the most significant comments relating to each screen. If the participants called for a screen to be changed, the screen and the changes suggested would are presented Objective 1 Figure 17: Showing login screen, Home screen, Courses screen and Course page 31

39 Loginscreen: Users thought this screen (shown in Fig 17) was well designed but there were requests to have a function to change the account password from the login page. However, the researcher decided to overlook this as it was not directly related to course related information. Homescreen: Users thought the homescreen (shown above in Fig 17) was also well designed but had a problem with the way elements on the homescreen operated after being selected. i. After selecting the Announcement element After selecting this option available announcements were displayed in a list showing their titles only. Users felt that this did not give the user enough information to clearly classify and differentiate announcements. Based on their comments the changes shown in Figure 18 below were suggested. The diagram shows the structure of the information the users would like to be included with each announcement. Figure 18: Box to the right shows the information users wanted to accompany assignments Active sites When participants selected this option they were met by the Courses screen shown in Figure 17 which lists all the courses available to the user. When a user selected any of the courses listed a view of the supported tools associated with that course would be given this is also shown in Figure 17 in this instance the CSC1016 option has been selected. The functionality beyond the course page was not available yet at this stage in the design process. 32

40 Objective 2 The users felt that the system was coming along well and liked the way on screen elements were kept to a minimum. Users did complain about the speed of the application as it took time to load. These lengthy loading times were exacerbated by the lack of loading screens as the interface was loading new screens. This left the users confused as they did not know whether the application was working or not. The researcher then noted the need for loading screen to indicate when the interface was loading Summary This design iteration was an interactive session where participants were able to sample how the system would work with real data. There were some concerns raised with how the system did not give the user feedback when performing long tasks this prompted the designer to consider adding load screens. This session raised an important issue of how to display content on the phone as exemplified with the announcements display covered in this design. The researcher noted that the next iteration would need to explore how information for some of the tools would need to be structured to allow users to classify and access it efficiently. 4.8 Fourth Design Iteration 4.81 Objective 1. To check the system for usability issues using a heuristic evaluation. 2. To present custom views for tool screens that will be supported in the system as identified in the previous iteration Preliminary The evaluators for this phase were the researcher and two other Honours students who were not part of the project team. The evaluation was done using Nielsen s Ten Usability Heuristics[32](see Appendix 3) Results Objective 1 The team of evaluators went through the design and found that the design was lacking in some areas. The shortfalls of our design where in the following areas: Visibility if Status: This was because the system did not have loading screens to inform the user that something was happening. This was resolved by adding loading screens to the application. User control and freedom: The user was limited to using in browser controls to move back and forth within the application this was remedied by adding a back button to the interface. 33

41 Flexibility and efficiency of use: The interface did not have controls that gave users quicker ways of navigating the interface. As a result the designers added a Home button to the bottom of the screen that sent the user to the Home screen no matter where the user was. The other heuristics were either inapplicable or the evaluators lacked the technical ability to apply them. The results of this design are shown in Figure 19 below. Figure 19: Sample screen showing interface additions All the changes could be added to the interface in paper prototype form but it was challenging for the researcher to include a loading screen on the drawing. Objective 2 Up until this point the design process had focused on page transitions the interface would go through when accessing tools but no significant effort had been put into customizing the views that were presented to the user after they had accessed a tool. As already indicated in section of the report the implementation of the SAKAI mobile access portal forces the mobile to share the same view as would be seen on desktop. 34

42 This approach would not be sufficient as a result we put forward designs for new screen layouts for as shown in Figure 20 below and discuss thereafter. Figure 20: Showing Grades displayed on Vula mobile SAKAI(left) and Announcements displayed on Vula mobile SAKAI As shown in Figure 20 we opted to display Grade and Announcement information in column to reduce the amount of horizontal scrolling the user would have to perform to get details on announcements and Summary In this section we performed a heuristic evaluation on our interface to make sure that we had not made any usability omissions. The interface was assessed with the heuristics to the best of the teams ability and changes were added in an attempt to address these issues. The layout of tools that would be used in the new interface was also designed this design was aimed at reducing the amount of side scrolling the user needed to perform to get details on the system. 4.9 Chapter Summary This chapter presented details on how the interface for the new mobile interface came to be the process was documented from requirements gathering to final interface design. The interactions with the users were documented and all related feedback which shaped the design process was noted. In the next chapter we give some detail on how the design of the high fidelity prototype and the final interface was carried out. 35

43 5. Interface Implementation 5.1 Introduction This chapter documents how the interactive prototype and final interface were implemented. This project mainly deals with design so this section will not go into implementation detail but will provide a listing of the technologies used to implement both the interface and motivation for why these tools were used. 5.2 First Prototype The first prototype which was more of a proof of concept was written mostly in HTML5[33] we chose this to be the main language as there is wide belief that it can be viewed as the future of mobile development. Our system would communicate with Vula via an intermediary server this server would pass our requests on to the main Vula system and the responses would be sent back to the phone as HTML5. The styling and theming of the HTML was done inline using jquerymobile[30]. In- lining the styling meant that whenever the code on the server was changed the styling information was lost. This represented a very messy implementation which was slow. That was acceptable as this was still considered to be just a prototype. We chose jquerymobile initially is it was easy to use and was well documented the main aim was to get something we could learn to use early. The system at this stage used HTTP GET and POST[31] requests which are not very efficient 5.3 Final Design From the initial prototype a considerable number of problems had surfaced which needed to be addressed. The most important amongst these was our initial assumption that we would be able to give users access to our extended features through a conventional phone browser. The problem came in that the components that were being developed by group members to extend Vula needed access to desktop API which were not available through the mobile browser. Our initial work around for this was to have the two tools as separate entities and provide access to the results of their use on the integrated interface. This would result in us having 3 separate systems running. When the second reader did a progress check on our progress she did mention that our aim was to join all the functions into one system as we had proposed in the project proposal. To allow this to happen we had to make the system a native app[34]. To make this possible we had to adopt new architectures which allowed for HTML5 to be packaged into applications this was done using phone gap[35] and sencha touch[36]. To make the system faster we decided to use AJAX[37] which help make applications faster by loading content on delivery back to the system. We also started using JSON objects from the server as responses which helped speed up the system further. The main advantage of all these changes for the interface designer was the adoption of Sencha Touch which uses a model- view- controller structure which allows implementation and styling to be kept separate. 36

44 6. Interface Evaluation 6.1 Introduction This chapter seeks to determine to what extent the project met the second project aim of improving student interaction with the most useful Vula tools on mobile devices. This evaluation will be done using usability testing combined with user satisfaction interviews. These evaluations will give a broad picture of how well the interface supports the user in performing tasks of interest. 6.2 Test Platform and Environment The final interface was implemented as a native Sencha Touch 2[36] application running on the Samsung Galaxy ACE running the Android(Gingerbread) Operating system. The application running on the specified device is shown in Figure 21 below, other screenshots of the final design can be found in Appendix 5. Figure 21: Final interface running on test phone The usability tests and user satisfaction interviews were performed in a quiet controlled environment. These ranged from the Honours lab or a classroom in the CS department depending on the time of day. The evaluation tasks were run on dummy accounts for ethical and security reasons so at no point did the user need to surrender their personal details. The Vula accounts were dummies in the sense that they were not linked to a registered UCT student but could match the functionality of one that was. 6.3 Test Participants As was earlier indicated this application was designed with the help of Science students enrolled here at the University of Cape Town. In the design process we needed the designers to be domain experts and this was done by stipulating that all participants in the design process be at least in their 37

45 second year of schooling and should be familiar with the Vula SAKAI mobile portal. For these tests we wanted testers to be Science students as the system was designed for Science students. This time however there was no requirement for them to have had a minimum number of years of study. The testers were also not required to have used the Vula SAKAI mobile interface before. In total 10 University students were used for the testing this was mainly because of time constraints. The user centered design approach used in this project required large amounts of user time which also needed to be scheduled ahead of time, coordinating 4-6 people to meet for a day proved to be quite challenging. The researcher settled on 10 because of time constraints. However based on findings made by usability experts researchers found that using 5 users finds 85% of the usability problems related to a design[38]. The researcher would have endeavoured to find all the usability errors but felt that the 85% would have to be sufficient. The one problem the small number of testers does pose is that the results of the tests could not be said to have statistical significance as the sampling was too small. Of these testers 4 were 2 nd year, 3 were 3 rd year and 3 were 4 th year from different programs in the Science Department. Students were recruited from in and around the Computer Science building. Users who had been involved in the design process were not permitted to be evaluators as they had prior knowledge of the system 6.4 Evaluation Structure and Content The evaluation of the interface was done in three parts. The first part involved asking participants to answer questions about their academic and technical background and general competency with the Vula system and the test operating system interface would be accessed from. The second part involved watching participants performing a set of carefully designed tasks that are representative of what the new interface was designed for. These tasks were performed both on the SAKAI mobile interface and on the new interface. While the tasks were being performed the researcher took note of time taken to perform tasks, outcome of tasks in terms of success or failure and errors made. The participant was offered no assistance unless the problems were related to the test platform and not the actual task for example some users were unfamiliar with the android environment and needed assistance accessing the virtual keyboard. The third involves asking users satisfaction interviews relating to the users experience with the two interfaces. Participants were paid R30 for taking part in the evaluation. The tasks were focused around interacting with what the researcher had found to be the most useful tools as identified in the requirements gathering portion of this project. The tasks not only tested accessing these tools but the ability of the participant to understand where they were in relation to the whole application and how they would navigate the interface to do certain tasks. The user satisfaction questions that come after the tasks came in two sections. The first section of questions are based on the USE questionnaire[39] which is a freely available list of usability questions which can be put to the user to assess how useful they thought the product in our case interface was. The responses to these questions were accompanied by a seven- point Likert rating scale from 1 (totally disagree) to 7 (totally agree). The second section contained free response questions that would help the researcher identify areas of improvement in the interface or directions for further work. 38

46 6.5 Test Procedure Participants were told about what the evaluation was about this information was also included on the evaluation form and participants were encouraged to do so. If the users felt they still wanted to be part of the test they were asked to sign the consent form attached to the test form (see Appendix 4). Since both interfaces will be performing the same tasks it was necessary that we switched the order in which the participants performed the task. This was done in an attempt to eliminate the effect of task familiarity on the results. From our 10 testers 5 were asked to perform the tasks on the SAKAI mobile interface before using the new interface and the remaining 5 used the interfaces in reverse order. After responding to the background questions the participant was asked to attempt the interface tasks the researcher then made note of all the steps the user took when attempting the various tasks. The researcher was looking for user errors, system errors, time taken to perform tasks and hesitation from the user when attempting any task. If the researcher noticed any of the first two occurring i.e. user errors or hesitation the researcher would ask follow up questions after that task as to why the user made that choice or hesitated. If the problem was not related to the interface but more towards the operating platform e.g. the android OS, the researcher asked the user to move on to the next question. If the problem was related to the platform the researcher explained where possible to allow the participant to continue. 7. Findings and Results This section documents the results for the final evaluation carried out as described in the previous chapter. The results will be presented as follows, the task completion rate will be presented then the errors that were made will be discussed with their related task An investigation into the responses of the user satisfaction interviews will then be analysed and insights on how usable the system was will be made based on the responses given by users. The new interface was addressed to as Vulamobi for the tests and will be reported on using that name. The free response questions will be used to get an idea of how the users felt about the system and will also be used in addition to the researcher s experiences whilst developing the interface to find areas of improvement for future work. 7.1 Task Completion The participants were able to complete all the tasks in the evaluation. Some users did make a few errors or did show hesitation when making choices but all the participants did manage to recover from any errors they made. In the researchers opinion the 100% completion rate can be attributed to the familiarity of the themes checked in the evaluation. The evaluation was designed to access familiar tools that students had not doubt interacted with on desktop before. Users could use this familiarity to help them complete tasks even if they had initially made errors for example from past 39

47 experience a participant knew that if they wanted to access say an announcement for example they would first need to access the course concerned. (All the tasks are located in Appendix 4) 7.2 Task Errors The testers did perform errors while using both systems and in some cases the error counts were high. Here we just noted the occurrence of an error not the number. Figure 22 shows how many testers performed when performing a task on an interface. Task CompleWon Errors Task7 Task6 Task5 Task4 Task3 Task2 Task1 Sakai Vulamobi Number of Students Figure 22: Number of students that performed errors with either interface on a task. Comments on tasks performed on the interface is shown below, if the task had a special condition associated with its completion on that task it was: SAKAI interface Task1: Was easy to perform and was done without any errors (misspelling credentials was not considered an interface error) Task2: First screen displayed on logging in so users automatically completed task. Task3: The course view displayed a number of tools some of which were unfamiliar to the user and were closely positions so user would click the wrong one. The navigation between windows was also difficult for users as the system would sometimes zoom out instead of going back one page Task4: Same issue as above in task 3 Task5: Same issue as above in task 3 Task6: Same issue as above in task 3 40

48 Vulamobi interface Task2: Some confusion over the function of the home button (see Appendix 5) as some users expected course information to be there Task3: Done well apart from when user got confused with the use of home button 7.3 Completion Time Completion time was initially collected but as the tests went on it was noted that at times the interactions with both systems were affected by some long delays due to network delay problems. With this aspect the researcher was an unable to tell if the low performance was a result of poor network or even poor access design on the part of the system. If the evaluator had thought of this beforehand and taken video recordings he could have later assessed how long the user took actively engaged in completing a task. As no recording could be taken the results could not be considered all that reliable. However the evaluator did note that in general the users of the SAKAI mobile interface spent longer periods of time when they needed to complete course related tasks. This in the researcher s opinion is related to the number of options available which made it difficult for them to pinpoint the exact tool they were looking for at any particular instant. Upon reflection the researcher came to the conclusion that measuring completion time on the mobile phone can be a bit uncertain. This was found to be the case by Zhang et al[40], who proposed using an emulator to run completion time tests on mobile devices as they had no less network connectivity uncertainty. Unfortunately there was no time for the researcher to set up such tests as the project had almost run its course. 7.4 User satisfaction Feedback (based on responses for Appendix 4 usability scale forms) The average scores of all the questions relating to usefulness namely 5,10,11 where higher for the new interface 5.9 compared to SAKAI mobile access which got an average score of 5.4. The average scores of all the questions relating to ease of use namely 3,7,9 where higher for the new interface 6.4 compared to SAKAI mobile access which got an average score of 4.7. This shows that according to our participants more users were satisfied by the new interface. This researcher took this with a pinch of salt as the interface we had designed was very minimal and was bound to be more usable for example as it had fewer things that could go wrong. The assessment also does not show that the questions we designed for the evaluations were those that showcase the best function of our system. If the systems had been reviewed over a wider set of functionality the SAKAI mobile interface would have probably won. This however does not overcome the fact that the new interface was received better by the users. 41

49 7.5Free Response feedback on new interface (based on responses for Appendix 4 post test interview section) All the users gave the interface above average ratings. All the users felt that the application was very easy to use and in the mind of the researcher that can largely be attributed to the simplicity of the tasks that the users had to perform. This was not seen as a negative as that was part of what the researcher set out to do. To develop an interface that allowed doing small tasks of interest very quickly and very well. The users also commented about how they enjoyed the simplicity and straightforward nature of the interface interactions. Even though most of the users liked the system there was still a feeling of it being incomplete and not doing enough. This in the opinion of the researcher can be attributed to how in the evaluations the users were only able to interact with the course tool base materials but were not really able to go into the Gallery or Feedback tool which were assessed separately by the other group members. The one of the two participants who had tried to access courses by pressing the Home button shown in Figure 21 felt that the designer had failed to name the button properly as he believed that Vula was synonymous with courses. This proved to be a good point as the researcher noted that according to the desktop counter part of our Vula interface home had a different function which broad the user to a screen containing all active courses. As our interface was designed to be an extra and not a replacement of the desktop Vula interface the researcher felt that a better name would have to be suggested. Another user said that the researcher s decision to not support aggregated views of information meant that the user had to keep going back and forth to perform different tasks. This function had initially been included in the design but as time went on it fell away in an attempt to preserve the need for clutter free interfaces. Another user suggested that the researcher consider having recently accessed pages tabbed on the application to help the user save time and network data by having to access the same pages all over and over again. There was however some aesthetic criticism some users felt that the use of the colour green for the interface buttons was not in keeping with UCT and Vula theming which is usually blue. This was seen as a small problem by the researcher who felt this could be easily remedied by applying different colour schemes. Overall the system was well received and users felt that it was a good starting point in moving towards adding more functionality. 7.6 Chapter Summary In this chapter the results from the test of the interface were presented and based on the feedback received, the majority of the test subjects felt that new interface was well conceived and was useful and usable more so than the existing SAKAI mobile portal. 42

50 8. Conclusion and Future Work The goal of this project was to develop a mobile interface for the Vula system that allowed the students to get access to information in a more effective and usable way. The first step in this design was to ascertain what the interface was supposed to support. This was done by doing requirements gathering through interviews, questionnaires and workshops. When the question of what to design had been answered the researcher then approached the question of how. There is literature that reports on the peculiarity of mobile interface design[29] and how it can be tricky. The researcher chose to tackle this problem using a user centered design approach making use of two methods namely participatory design and interaction design which put end- users in the role of co- designers. This approach was chosen as it has been found to lead to highly usable systems. With the aid of users the researcher was able to design a high fidelity prototype which was then evaluated with users again to make further design improvements. After that the final interface design was reached and then evaluated. The success criteria of the project when it was first proposed was to identify which tasks users of Vula would want to perform on mobile devices and how these chosen tools could be presented better on mobile devices. The evaluations that were undertaken by the researcher showed that users were generally pleased with the way tools were presented to them. In this regard the interface can be seen as a success and so with it the project. However the number of tools supported was too minimal for there to be a comprehensive feeling of achievement initial design restrictions and time constraints did not allow the researcher to fully explore the user centered aspect of design. Upon further reflection the researcher realised that though the project was a moderate success it could have been done much better. Mobile design experts agree that the most challenging aspect of mobile interface development is the mobility aspect[40]. All the tests and evaluations where performed with users in a controlled environment but how often do users interact with mobile devices in controlled environments where they are able to devote their full attention to them. I think given more time an ethnographic study could be conducted into how students interact with their mobile devices given that a study has already been done into how student work with Vula. I also think that maybe another angle would be to reconsider how we try to use LMSes on mobile phones this project looked at which desktop tools we could successfully implement on mobile phones why not try to develop tools that function just for phone and provide a new experience on the phone so that users aren t compelled to use the phone because it can do it just as well as the desktop but because it can do it better. From my work on this project I have learnt that mobile design is an issue of compromises and I think the best way to build effective interfaces is to work closely with users with a full knowledge of their capabilities and strengths. The design principles covered in user centered design in my opinion are not sufficient if applied individually but should be applied in combination with other techniques. The problem comes in deciding which techniques work best for which problems and how do you combine them. This design process has taught me to appreciate the finer details and how every little thing counts in design. 43

51 44

52 9. References [1] C. Juma, Net access for African universities would boost continent, [Online]. Available: would_boost_continent.html. [Accessed: 29- Oct- 2012]. [2] L. Chittaro, Designing Visual User Interfaces for Mobile Applications, pp. 1 2, [3] S. Esther, One- Stop Shopping With Learning Management Systems, THE Journal, vol. 35, no. 6, p. 18, [4] Sakai Project collaboration and learning - for educators by educators. [Online]. Available: [Accessed: 13- May- 2012]. [5] A. Dix, J. Finlay, G. Abowd, and R. Beale, Human- Computer Interaction, 3rd ed. Prentice Hall, [6] Mobile_learning_reviewed_final.pdf (application/pdf Object). [Online]. Available: [Accessed: 13- May- 2012]. [7] Vula : Gateway : Welcome. [Online]. Available: [Accessed: 13- May- 2012]. [8] Projects Centre for Educational Technology. [Online]. Available: [Accessed: 29- Oct- 2012]. [9] icts- for- education.pdf (application/pdf Object). [Online]. Available: for- education.pdf. [Accessed: 13- May- 2012]. [10] Y. Liu, H. Li, and C. Carlsson, Factors driving the adoption of m- learning: An empirical study, Computers & Education, vol. 55, no. 3, pp , Nov [11] E. Wagner, Enabling mobile Learning, EDUCASE Review, vol. 40, no. 3, pp , [12] G. Conole, Contemporary Perspectives in E- Learning Research: Themes, Methods and Impact on Practice (Open and Flexible Learning Series) [Paperback], 1st ed. Routledge, 2006, p [13] G. Woodill, The Mobile Learning Edge: Tools and Technologies for Developing Your Teams, 1st ed. McGraw- Hill, 2010, p. 53. [14] Mobile Usability in Educational Contexts: What have we learnt?, The International Review of Research in Open and Distance Learning, vol. 8, no Jun [15] A. Serrano- santoyo and J. Organista- sandoval, Challenges and Opportunities to Support Learning with Mobile Devices, vol. 1, pp ,

53 [16] Moodle.org: open- source community- based tools for learning. [Online]. Available: [Accessed: 13- May- 2012]. [17] Blackboard Technology and Solutions Built for Education. [Online]. Available: [Accessed: 13- May- 2012]. [18] Desire2Learn Mobile Solutions - Hear One, Do One, Teach One. [Online]. Available: mobile- solutions.html. [Accessed: 11- May- 2012]. [19] Pearson ecollege. [Online]. Available: [Accessed: 13- May- 2012]. [20] moodle, Moodle4iPhones Project. [Online]. Available: [Accessed: 13- May- 2012]. [21] Mobile Oxford. [Online]. Available: [Accessed: 23- May- 2012]. [22] Vula. [Online]. Available: [Accessed: 11- May- 2012]. [23] Y. Huang and C. Chang, Applying Mobile Technology to Portal- Based Course Management Systems, [24] Cascading Style Sheets. [Online]. Available: [Accessed: 29- Oct- 2012]. [25] HTML iframe tag. [Online]. Available: [Accessed: 29- Oct- 2012]. [26] JavaScript Tutorial. [Online]. Available: [Accessed: 29- Oct- 2012]. [27] C. Abras, D. Maloney- krichmar, and J. Preece, User- Centered Design, pp. 1 14, [28] J. Preece, Y. Rogers, and H. Sharp, Interaction Design - Beyond Human - Computer Interaction, 2nd ed. John Wiley & Sons, [29] M. Jones and G. Marsden, Mobile Interaction Design. John Wiley & Sons, [30] jquery Mobile jquery Mobile. [Online]. Available: [Accessed: 29- Oct- 2012]. [31] T. Berners- lee, R. Fielding, and Frystyk, Hypertext transfer Protocol - - HTTP/1.0, [Online]. Available: [32] J. Nielsen, L. A. Blatt, J. Bradford, and P. Brooks, Usability Inspection, pp , [33] HTML5 Introduction. [Online]. Available: [Accessed: 30- Oct- 2012]. [34] Mobile applications: native v Web apps what are the pros and cons? 46

54 [35] PhoneGap Home. [Online]. Available: [Accessed: 30- Oct- 2012]. [36] Mobile App Development Platform Sencha Touch Products Sencha. [Online]. Available: [Accessed: 29- Oct- 2012]. [37] AJAX Introduction. [Online]. Available: [Accessed: 30- Oct- 2012]. [38] J. Nielsen, How to Conduct a Heuristic Evaluation. [Online]. Available: [Accessed: 29- Oct- 2012]. [39] M. Arnold, Measuring Usability with the USE Questionnaire, Usability & User Experience, vol. 8, no. 2, [40] D. Zhang and B. Adipat, International Journal of Human- Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications, no. October 2012, pp ,

55 10. Appendix 1: Standard Vula Tools The tools available for a course vary depending on what the course convenor expects students enrolled in the course to do. The possibilities include custom- built tools (e.g. the automated marker in Computer Science) and the standard Sakai tools as shown in the list below. Home Activities Announcements Assignments Blogs Calendar Chat Room Course Evaluation Course Outline Drop Box Archive External Tool Forums Glossary Gradebook Group Manager Lessons Maps For viewing recent announcements, discussion, and chat items. For designing, managing and delivering collaborative learning activities in LAMS For posting current, time-critical information For posting, submitting and grading assignment(s) online For course or project blogging or journals For posting and viewing deadlines, events, etc. For real-time conversations in written form For course evaluations or other surveys or feedback For posting a summary outline and/or requirements for a site. For private file sharing between instructor and student For viewing sent to the site For external tools supporting the IMS BasicLTI standard. Display forums and topics of a particular site OSP Glossary Tool For storing and computing assessment grades from Tests & Quizzes or that are manually entered. For managing Adhoc Groups and their memberships [Lessons] Create content modules and sequences; can be organized by week or unit For using interactive Google Maps 48

56 Messages Modules News Participants Podcasts Polls PostEm Q&A Resources Search Section Info Sign-up Site Setup Site Stats Tests & Quizzes Web Content Wiki Display messages to/from users of a particular site For creating and organizing content modules For displaying news and updates from online sources (RSS feeds) For viewing the site participants list. For managing individual podcast and podcast feed information For anonymous polls or voting PostEm gradebook tool for sakai. For asking, answering and organising questions For posting documents, URLs to other websites, etc. For searching content within the site or across sites For managing sections (e.g. tutorial groups) within a site. For enabling online registration for meetings and other events. For updating worksite information and site participants. For showing site statistics by user, event, or resource. For creating and taking online tests and quizzes For displaying web content from Resources or an external website For collaborative editing of pages and content 49

57 11. Appendix 2: Participatory Design Information Packet VulaMobi Participatory Design Preliminaries i. Participation in this survey is voluntary ii. Research participants may withdraw from the research at any time without compromise to their well- being iii. The objective of this study is to investigate how Vula is used by UCT students on mobile phones. It also aims to find which aspects of Vula students would want to use most frequently from a mobile phone. iv. Users will also be asked to share their opinion on the current mobile SAKAI access portal for interacting with Vula from a mobile device v. The tool will not be used to evaluate ones academic ability vi. Research participants are required to sign consent forms vii. Research participant s identities will be coded and as such their responses will be unidentifiable / anonymous viii. Data collected during this research will be treated / handled in a confidential manner ix. Feel free to comment at any time as this is meant to be a free flowing discussion amongst the participants. x. If any instructions are unclear please feel free to ask me. xi. Please feel free to share any information that you think may be relevant to the discussion. xii. This evaluation is limited to the access of course websites which a student may be enrolled in. Societies and clubs are not included. If you are still willing to be part of this study please see the attached consent form overleaf 50

58 Consent form Researcher: I agree to participate in this experiment. I agree to my responses being used for education and research. I understand that my personal information will only be used in aggregate form, so that I will not be personally identifiable. I understand that any material captured by this experiment is to be treated Confidentially and none of it will be released to the public. I understand that I am under no obligation to participate in this project. I understand that I have the right to withdraw from this experiment at any stage. I have read this consent form and the information it contains and had the opportunity to ask questions about them. Name of Participant: Signature of Participant: Date: 51

59 Vula Tools Familiarity Please indicate which Vula tools you have used on the desktop and which tools you would want to access from a mobile device? Tool name Desktop Mobile Announcements Assignments Calendar Chat Room Drop Box Archive Forums Gradebook Post 'Em Presentation Resources Schedule Section Info Site Info Syllabus Tests & Quizzes Web Content Wiki 52

60 SAKAI MOBILE PORTAL Mobile Access Point Comments 53

61 Login screen Comments 54

62 Home Screen Comments 55

63 Course screen Comments 56

64 12. Appendix 3: Nielsen s Ten Tsability Heuristics Visibility of system status Keep users informed about system status. Match between system and the real world The system should speak the users' language. User control and freedom Give users clear "emergency exits" to leave the unwanted states. Support undo & redo. Consistency and standards Give users standard set of words, situations, and actions. Error prevention Careful design is better than good error messages. Recognition rather than recall Minimize the user's memory load. Make objects, actions, and options visible. Flexibility and efficiency of use Give accelerators to speed up the interaction for the expert users. Aesthetic and minimalist design Do not give information that is irrelevant or rarely needed. Help users recognize, diagnose, and recover from errors Error messages should be expressed in plain language. Help and documentation Documentation should be easy to search and focused on the user's task. 57

65 13 Appendix 4 Interface Usability Testing VulaMobi Usability Testing Preliminaries xiii. xiv. xv. xvi. xvii. xviii. xix. xx. xxi. xxii. xxiii. Participation in this survey is voluntary Research participants may withdraw from the research at any time without compromise to their well- being The objective of this testing is to evaluate usability and usefulness of the VulaMobi tool The tool will not be used to evaluate ones academic ability Research participants are required to sign consent forms Research participant s identities will be coded and as such their responses will be unidentifiable / anonymous Data collected during this research will be treated / handled in a confidential manner Feel free to comment on any aspect of the evaluation in the comments section provided should you require additional paper please let me know. If any instructions are unclear please feel free to ask me. Please note any peculiarities or useful functions you encounter during this evaluation. This evaluation is limited to the access of course websites which a student may be enrolled in. Societies and clubs are not included. 58

66 VulaMobi Usability Testing Consent form Researcher: I agree to participate in this experiment. I agree to my responses being used for education and research. I understand that my personal information will only be used in aggregate form, so that I will not be personally identifiable. I understand that any material captured by this experiment is to be treated confidentially and none of it will be released to the public. I understand that I am under no obligation to participate in this project. I understand that I have the right to withdraw from this experiment at any stage. I have read this consent form and the information it contains and had the opportunity to ask questions about them. Name of Participant: Signature of Participant: Date: 59

67 Introductory Instruction Form Thank you for agreeing to participate in this study! This form explains the experimental procedure, so please read it carefully. The duration of the experiment is approximately 30 minutes. The experiment will present you with a set of tasks to perform using the Vulamobi app and the SAKAI mobile access portal. You will be told which of the two interfaces to start with. At the end of each set of tasks you will be required to answer some questions on the experience you had with each interface. You are encouraged to make any comments relating to the system in between tasks. Please do not commence a task until told to do so and continue until either you are asked to stop, feel you have completed the task or you feel the need to stop for any reason. Please answer all questions and feel free to share comments on any aspect of the test. 60

68 Pre Test Questions Please select your choice with an X where appropriate (use mark the area of your choice with an X ): 1. What academic program are you registered for (e.g. BSc Computer Science) 2. Which year of study are you in? 3. How often do you use Vula on a desktop computer? Never Daily 4. How often do you access Vula on a mobile device? Never Daily 5. How familiar are you with the android phone environment? Very Unfamiliar Very Familiar 61

69 Participant Tasks Section A: Vulamobi App Task 1 Please login to the service using the following details: Username: test Password: test Please make an indication when you feel you have successfully completed the task. How difficult do you feel this task was to perform? Very difficult Very easy Task 2 Where would you go to view the courses you are enrolled in? When you have identified the option please select it? How difficult do you feel this task was to perform? Very difficult Very easy Task 3 Can you please view an announcement from a course? How difficult do you feel this task was to perform? Very difficult Very easy Task 4 62

70 Can you view an announcement from a different course? How difficult do you feel this task was to perform? Very difficult Very easy Task 5 Can you go to a view showing all your enrolled courses? How difficult do you feel this task was to perform? Very difficult Very easy Task 6 Check the grades for Cookery101? How difficult do you feel this task was to perform? Very difficult Very easy Task 7 Can you logout of the system? How difficult do you feel this task was to perform? Very difficult Very easy 63

71 Vulamobi interface usability scale form 1. I think that I would like to use this system frequently Strongly disagree Strongly agree 2. I found the system unnecessarily complex Strongly disagree Strongly agree 3. I thought the system was easy to use Strongly disagree Strongly agree 4. I think that I would need the support of a technical person to be able to use this system Strongly disagree Strongly agree 5. It is useful? Strongly disagree Strongly agree 6. I thought there was too much inconsistency in this system Strongly disagree Strongly agree 64

72 7. I would imagine that most people would learn to use this system very quickly Strongly disagree Strongly agree 8. I felt very confident using the system Strongly disagree Strongly agree 9. Is it user friendly? Strongly disagree Strongly agree 10. It helps me be more effective? Strongly disagree Strongly agree 11. It makes the things I want to accomplish easier to get done? Strongly disagree Strongly agree 1 65

73 Section B: SAKAI MOBILE ACCESS Task 1 Please login to the service using the following details: Username: test Password: test Please make an indication when you feel you have successfully completed the task. How difficult do you feel this task was to perform? Very difficult Very easy Task 2 Where would you go to view the courses you are enrolled in? When you have identified the option please select it? How difficult do you feel this task was to perform? Very difficult Very easy Task 3 Can you please view an announcement from a course? How difficult do you feel this task was to perform? Very difficult Very easy 66

74 Task 4 Can you view an announcement from a different course? How difficult do you feel this task was to perform? Very difficult Very easy Task 5 Can you go to a view showing all your enrolled courses? How difficult do you feel this task was to perform? Very difficult Very easy Task 6 Check the grades for Cookery101? How difficult do you feel this task was to perform? Very difficult Very easy 67

75 Task 7 Can you logout of the system? How difficult do you feel this task was to perform? Very difficult Very easy 68

76 SAKAI mobile access interface usability scale form 1. I think that I would like to use this system frequently Strongly disagree Strongly agree 2. I found the system unnecessarily complex Strongly disagree Strongly agree 3. I thought the system was easy to use Strongly disagree Strongly agree 4. I think that I would need the support of a technical person to be able to use this system Strongly disagree Strongly agree 5. I found the various functions in this system were well integrated Strongly disagree Strongly agree 6. I thought there was too much inconsistency in this system Strongly disagree Strongly agree 69

77 7. I would imagine that most people would learn to use this system very quickly Strongly disagree Strongly agree 8. I felt very confident using the system Strongly disagree Strongly agree 70

78 Post Test Interview 1. What are your overall impressions of the VulaMobi app? Very bad Very good 2. If you had to give the app a grade, from A to F, where A was exemplary and F was failing, what grade would you give it, and why? Name three words or characteristics that describe this app? 5. What are the three things you like best about the app? 6. What are the three things you like least about the app? 7. If you could make one significant change to this app, what change would you make? 8. Are there tools or functionality you would like to see added to the Application? Which ones? 9. Would you recommend this app? To a friend? 10. Do you have any other questions or comments about the app or your experiences with it? 71

79 14. Appendix 5: Final Interface screenshots 72

80 73